TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Ask HN: Studying large code bases

57 点作者 rvalue大约 10 年前
Hi HN,<p>Suppose you had to understand a large distributed system. The modules which interact together can be written in different programming languages and all expose a REST endpoint along with a documentation of what each endpoint can do.<p>For the simplicity of discussion lets consider these languages(Java, C++, Scala, Python and PHP). They could use a distributed database like solr, couchbase<p>Some of these modules can be 12factored and some with pretty bad code practices. There is no existing mechanism for monitoring but logging.<p>How would go into analyzing this vast system ? (Not just one at a time, elaborate your techniques)<p>To what degree would you trust the documentation, infrastructure, existing testing tools, environment setup tools in the process of understanding a module ?<p>What would you make sure you definately do NOT do?<p>What language specific strategy would you suggest ? (Entrypoints, configs, static analysis etc.)<p>Would you think of creating a mechanism for cloud debugging ?<p>How do you keep track of the knowledge you learn from one module and apply that as you go along ?<p>How do you keep your life together and still swim in so much complexity ?<p>Share your stack! http:&#x2F;&#x2F;stackshare.io&#x2F;<p>How do you apply these techniques to learn, update and maintain this system?<p>PS: I am asking this question in general and not for an existing system. But if you would like to give an answer with some open-source project as example, that would be great.

14 条评论

zgm大约 10 年前
1. Start at the simplest subsystem, and work from there.<p>This is especially true when working with third-party, legacy, or undocumented code. In the absence of documentation, your only way forward is to read the source code[0]. Find the main() of the simplest&#x2F;smallest module in the system, and begin tracing through the code. This could be with a debugger, print statements, or the search functionality in your editor&#x2F;IDE (Find Usages and Go to Declaration in Intellij are lifesavers).<p>2. Don&#x27;t be afraid to break things apart.<p>If the simplest module in the system is overwhelmingly complex, start commenting out parts of the code. Go until you have effectively reduced it to &quot;Hello, World&quot;, if you have to. From there, you can gradually add features back.<p>3. Constantly test your assumptions.<p>Don&#x27;t assume comments do what they say they do.<p>Don&#x27;t assume that config flag produces the behavior the documentation claims it does.<p>Do take inventory of you assumptions whenever the behavior of the system contradicts your current understanding of how it works.<p>You should be able to back up any claims about the system with empirical evidence e.g. when I change A to B, X happens; if I change A to C, Y happens.<p>[0] <a href="http://blog.codinghorror.com/learn-to-read-the-source-luke/" rel="nofollow">http:&#x2F;&#x2F;blog.codinghorror.com&#x2F;learn-to-read-the-source-luke&#x2F;</a>
评论 #9061514 未加载
评论 #9061589 未加载
chubot大约 10 年前
Shell scripts!<p>I would write a crapload of shell scripts to start every component locally, in the foreground. Hopefully you can just pass flags to specify ports, rather than having to specify config files. (Though you can easily generate config files from shell scripts)<p>I use tmux, so I can have all servers up in the foreground and logging to stdout. You can arrange the windows so it&#x27;s easy to get a global view.<p>Identify which servers are stateful and which are stateless. For the stateful ones, it&#x27;s nice to be able to save and restore their state (hopefully just with cp and mv).<p>Then I would write some more shell scripts to poke at every port. For REST services, just use curl, and possibly some JSON&#x2F;XML helpers. Stateless servers should be testable exhaustively.<p>Then pick some code change you want to make. Write the shell snippets to tickle that code path.<p>Then write some more shell scripts to rebuild every component binary from source. Make your code change, build, restart, and then test to see if it works. Look at the logs for clues.<p>This is basically the &quot;dynamic&quot; approach to debugging systems, rather than the &quot;static&quot; approach of staring at code. Sometimes staring at code is the right thing to do, but if as you say the sheer volume of code is too much, then figuring out the behavior as a black box -- tickling it with shell scripts -- is often a faster way to understand the system.<p>They should be scripts because you need to be able to repeat this process many times.<p>Other strategies are:<p>- Understand the architecture from the build system. Don&#x27;t look at the source code, look at how it&#x27;s compiled. What libraries go in what binaries is a strong indicator of their function.<p>- Understand the architecture from the deployment. In this case I&#x27;m suggesting to rewrite the deployment yourself with shell scripts (to a single host, your dev box). But you can also try to just look at whatever the ops&#x2F;devops team does to bring up the service in a new cluster.<p>- Look at any existing system tests &#x2F; regression tests.
评论 #9061549 未加载
warcode大约 10 年前
To understand the system start by ignoring the code.<p>1. Start with all the components as black boxes.<p>2. Draw the system design&#x2F;connections&#x2F;interfaces between them.<p>3. Follow the data and uncover black boxes by reading the code.<p>4. (If possible) Write tests and refactor code so that testing is possible.
thewarrior大约 10 年前
Try to understand the data model. Dump all the SQL schemas , diagram them if required and study them.
devnonymous大约 10 年前
That&#x27;s a whole lot of questions, I&#x27;ll attempt to answer just the primary one -- how I approach a new large code base.<p>For me, the absolute first thing to do is <i>not</i> look at the codebase. I prefer to understand the system itself (ie: understand the stack and how everything fits in). IOW, understand what layers exist and how exactly are they stacked. Again <i>without</i> looking at the codebase, rather by interacting with the system.<p>I do this by trying to understand the main entry points for a specific task and follow the &#x27;request&#x27; (for lack of a better term, but it doesn&#x27;t mean a http request) along the stack. In real terms what this means usually, is that I&#x27;d hit the logs and see what happens when I do something.<p>Next (ie: by which time I know at least the major layers), I would look at the codebase and identify sections&#x2F;modules that seem like they provide the entry-points discovered above. Next, depending on the language and tools that are available, I start &#x27;playing&#x27; around with the code.<p>I know this sounds a bit vague but all that I can tell you is this is what works for me and I have gotten better at it over time (ie: experience counts).
joeld42大约 10 年前
How I do it is play with the system as a user first, and think of small stuff to change. It might be real features or bugfixes, but at this stage, it&#x27;s often thowaways like &quot;make that button report the current frommet count&quot;, or &quot;add a new token to the file format&quot;, etc.. small changes that require communicating from disparate parts of the system or require touching deeply buried parts of the codebase are best. Often in a large codebase trivial changes can be quite hard to track down. The act of digging through and finding and making these changes give me an overall map of the codebase.<p>Fixing bugs is a great way to do this. If I&#x27;m new to a project, I will spend a few days just going through the bug backlog if I can. Half the time (at least) the bugs are not even real, no longer applicable or otherwise non-fixable, but just tracking that down is useful (and it feels good to close tickets)<p>Also, learning quickly what parts of the codebase you can simply ignore and treat as a black box, and where you need to change.
sudeepj大约 10 年前
I recently changed my job and I am trying to figure out how the product works there so here is my strategy:<p>1. Knowing the actual purpose of the product(or system) is must.<p>2. Using the product yourself helps a lot. This gives you end-user perspective.<p>3. Reading the code in a context is far more effective. A context here could be a test case or a use case. More granular the scenario the better it is.<p>4. If you are not able to understand a piece of code, revisit it later and move on. In the meantime note the input and output of &quot;that specific&quot; code. Many times after your move on and read other parts one starts to connect the dots.<p>5. Never assume anything!<p>Example: I do not know much of Linux kernel so lets say I want to know when I do &quot;fopen&quot; in C where (&amp; how) does the kernel make entries in its own data structures.
parados大约 10 年前
Have a look at David Beazley&#x27;s video for a real life example: <a href="http://www.pyvideo.org/video/2645/discovering-python" rel="nofollow">http:&#x2F;&#x2F;www.pyvideo.org&#x2F;video&#x2F;2645&#x2F;discovering-python</a>
评论 #9072511 未加载
zzzcpan大约 10 年前
Even though you are asking general questions I&#x27;m going to assume your goal is to work on a large unfamiliar code base.<p>Go with a call tracing tool, write one yourself if there is no such tool. This will speed up your learning process tremendously compared to any other approach.<p>And in general, if learning feels hard - you are doing it wrong, try something different.
Nate75Sanders大约 10 年前
OpenGrok has been very useful when I want to understand a new codebase. The distributed &#x2F; REST aspect is going to make it harder to use code tools, though.
评论 #9063078 未加载
Stephn_R大约 10 年前
I would trust documentation with a grain of salt. Very rarely do I find intensive and accurate documentation for distributed systems. However, documentation may be helpful for things such as stand-alone components and REST endpoints. Those would be taken with possibly 2 grains of salt.<p>The one thing that you would never want to do is actually read through the code line by line*. HOWEVER, let it be noted that I mean this as an initial point of insertion. As a Computer Scientist, your first step should be in attempting to ask the right questions.<p>Think of it like a game of 20 questions ( N questions in this case...). To do this, one of the best things I have done is to follow the process. All distributed systems attempt to solve one or several use cases. Define these use cases and then analyze the code basis from there. For example, a use case that involves a REST Endpoint can be updated your profile photo. The use case would define a point of entry (where&#x2F;how to submit a new photo) and a point of submission (the REST Endpoint). From this, we can search for two components in the system that facilitate this task.<p>=== A good word of advice in understanding any code base is that, nine times out of 10, it is ALWAYS easier to read code once you understand its purpose. ===<p>&gt; Language Specific Strategies<p>As for some good language specific strategies, be knowledgeable of what Paradigm the language falls into. For example, Java code tends to have a single point of entry for most cases. Whereas Javascript (as a functional language) can have several entry points (or even none at all). For functional languages, be mindful of how they are structured. Functional languages are not object-oriented by nature but they can be if implemented properly. As such, try to notice the key differences (i.e. prototyping).<p>&gt; Cloud Debugging?<p>I would not consider it. Debugging should be a constant process of development. Not even the best developers can write perfect code from the start but with enough work and constant QA testing&#x2F;Unit testing during the development process, any code base can become easily manageable for debugging. In addition, learning how to read Stack Traces can save HOURS of debugging.<p>&gt; Tracking knowledge from Module to Module<p>Personally, I keep notes. I pretend as if I am writing an API document. Nine times out of ten, most modules take an input and provide an output after some calculation. So long as the calculation is sound, I do not worry about the calculation and only record the call to the module itself and its return. Other cases like classes can be tracked by the same notion.<p>&gt; OH THE COMPLEXITY<p>It&#x27;s a necessary evil. No one ever said being a developer was easy. And no one ever said the best things in life are free and easy. I work hard for what I do and for what I want. I love my job. I don&#x27;t think I would ever consider changing careers. Sometimes the complexity does become overwhelming and its hard to handle. But if you wish to know my little secret, schedule yourself some mandatory free time. And stick to the schedule. This doesn&#x27;t mean try breaking away from the computer, but just do things that help you relieve stress. Talk to a friend and have a drink or two. Developing is tough on the mind but it is an occupational hazard in the least.<p>&gt; Updating&#x2F;Maintenance<p>If it&#x27;s open source, let the users dictate when the updates should happen. I believe in Agile development and trusting the users. But when it comes to updates and maintenance on a corporate system, always try and set some goals for the change. Why are you updating&#x2F;performing maintenance? Is it to improve the speed&#x2F;response time? Is it worth the cost of time&#x2F;labor?<p>&gt; GOLDEN RULE: Ask Questions. Be Skeptical and always tread carefully. We are Developers. We are Problem Solvers. Not Problem Makers.<p>BG: I have 6 years of experience working with Corporate level architectures and distributed systems. And earned my BA in CS minor in Information Systems. In my time, I have worked with multiple functional, object-oriented, and web based languages. I have also worked as a Full-Stack Developer for roughly 3 years now.
评论 #9061470 未加载
adamnemecek大约 10 年前
setup a debugger and step through common scenarios.
评论 #9061196 未加载
igorgue大约 10 年前
Also, read this book: <a href="http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052" rel="nofollow">http:&#x2F;&#x2F;www.amazon.com&#x2F;Working-Effectively-Legacy-Michael-Fea...</a><p>It helps a lot and teaches you how to use grep and other tools (a lot more others that I no longer remember) to search and find your way through legacy code.
igorgue大约 10 年前
My regular way to go would be to go bug fixing, the more bugs you fix the more you&#x27;d be familiar with the source code, this is what most open source projects do, but code quality is better so it takes less time to get familiar with it.<p>Then if I don&#x27;t have any bugs to fix, it would be to get very familiar with the application and read the logs on every action you do, take note of where they say they are and study and go on rabbitholes.<p>Whatever you do, don&#x27;t try to rewrite it, they almost never work unless you have a way too much time in hand.