TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Things I learned from building a production database

551 点作者 dangoldin超过 3 年前

23 条评论

physicles超过 3 年前
&gt; [10] Make your project robust to re-orgs. A company management hierarchy is inherently fragile (a tree is a 1-connected graph, after all); socialize the project continuously with managers who might take over in the future. Do whatever it takes to make sure that manager churn does not result in unfair career outcomes for ICs.<p>&gt; [17] For storage systems, bias heavily in the beginning towards consistency and durability rather than availability; these are harder to measure and harder to fix if broken. Because availability is easier to measure, there will be external pressure to prioritize it first; push back.<p>It&#x27;s kind of crazy how well the author is able to move from the very top to the very bottom of the technical project management stack.
评论 #29327404 未加载
评论 #29334727 未加载
turbocon超过 3 年前
I really think this is a great list however I think one of the major factors why this project succeeded isn&#x27;t listed but is mentioned in the intro.<p>&gt; We hit production with a 3-person team in less than a year; and subsequently scaled the team to 30+ engineers spanning multiple sub-teams<p>Starting with a small skilled team to build the core and actively plan for a larger team at some point is absolutely critical for project success in my experience, especially with complex projects like this. That small team can have a much tighter communication loop and all keep a single unified picture of what the end goal is. When the team grows this single unified picture is no longer feasible so it has to be broken down into sections, which is a good thing, but if the core isn&#x27;t setup so that the sections are sufficiently decoupled things fall apart quickly.
评论 #29325736 未加载
评论 #29340072 未加载
vishnugupta超过 3 年前
Lot of good, actionable learnings there. I liked this one in particular.<p>&gt; [26] In a transparent codebase with quick review cycles, APIs will leak implementation details unless you gate-keep.<p>It&#x27;s tempting to make quick fixes to APIs but please take it slow. Think through existing customers as well as how could new ones will use it.<p>Also, something I learned while building&#x2F;maintaining public APIs was to <i>always</i> clearly document (along with examples) the context in which an API should (ideally) be used and how it combines with other APIs to enable common use cases.
nicoburns超过 3 年前
This is a really excellent list. Some of it is specific to the kind of low level work this person was doing, but a lot of it isn’t. I would love to work on a team with the author of this blog post.
cmg超过 3 年前
Assuming I&#x27;m not the only one who&#x27;s never heard of Delos or Cubby: <a href="https:&#x2F;&#x2F;engineering.fb.com&#x2F;2019&#x2F;06&#x2F;06&#x2F;data-center-engineering&#x2F;delos&#x2F;" rel="nofollow">https:&#x2F;&#x2F;engineering.fb.com&#x2F;2019&#x2F;06&#x2F;06&#x2F;data-center-engineerin...</a>
评论 #29325542 未加载
评论 #29324889 未加载
_pastel超过 3 年前
&gt; [16] Design as a team; implement as individuals.<p>This is pithy. I&#x27;m curious - does anyone here follow this philosophy? How do you actually design as a team? A meeting for every single design decision?<p>I&#x27;m used to the team finding general alignment on the problem in meetings, but then a single person coming up with the design, writing a doc, and circulating it for comments.
评论 #29328227 未加载
评论 #29327521 未加载
评论 #29327000 未加载
评论 #29326852 未加载
评论 #29327536 未加载
scrubs超过 3 年前
Wow! One hell of a good list. And well explained. You bet I book marked it. Heck even the 1-connected comment was worth the read. Because it&#x27;s true. I&#x27;m dealing with that right now. +5 and another reason I love HN. Now, if it you would have made it 43 items, a nice prime number, then I would be jealous to the point of angry (that&#x27;s a riff on John Daily).
评论 #29327975 未加载
评论 #29326989 未加载
krisrm超过 3 年前
Very good list.<p>[22] Avoid using real-time for correctness guarantees or comparing clocks across machines unless you have (and understand) error bounds on the clock.<p>I shudder to think of the hours spent head-scratching that went into this Lesson Learned :)
评论 #29324268 未加载
bertr4nd超过 3 年前
I’m reading this list, and it’s a great list but it also makes me sort of depressed. I think I have the combination of time and intellect to achieve maybe half of what’s proposed here. What do you do when you can’t possibly achieve writing throwaway prototypes to get design input while engaging deeply with your customers and keeping up with the research in your field while designing your APIs for 2+ implementations? And oh by the way shipping this major product in under a year?
评论 #29326236 未加载
评论 #29326888 未加载
评论 #29326216 未加载
评论 #29329569 未加载
bbulkow超过 3 年前
There is a critical element the post doesn&#x27;t state: a successful small project has (has) very limited scope of customers and use patterns. That is where complexity in writing databases arises: limited scope databases are far easier.
评论 #29327641 未加载
sjansen超过 3 年前
Although the author says his advice is for people &quot;building new infra at large companies&quot;, there&#x27;s certainly a number of points that apply to a larger audience.<p>For example:<p>&gt; Measurement is a means, not an end.<p>Reminds me of a lesson I learned early in my career. Bad things happen when you lose sight of your purpose and focus just on measurement.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Goodhart%27s_law" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Goodhart%27s_law</a>
swuecho超过 3 年前
Interface directly with customer ICs ==== what&#x27;s IC?
评论 #29326885 未加载
greenSunglass超过 3 年前
Is it what people do on a sabbatical?
throwthrow564超过 3 年前
&gt; [15] Design APIs with migration to new implementations as a first-class consideration; custom migrations are huge time-sinks and sources of unreliability. Every major API should have a single CLI-driven call for switching implementations.<p>I&#x27;ve never actually thought about doing this as an initial design principle. I wonder how it materially affects the design.
评论 #29326439 未加载
评论 #29326414 未加载
throwaway984393超过 3 年前
<i>&quot;[1] Keep your customers happy&quot;</i> - It&#x27;s hard to actually have a team that respects this lesson. Even if one or two people on the team believe in this rule, others won&#x27;t, and if they have equal say in the final product, it means the customers will suffer. Dogfooding is really the only way to fight this. Become your own most demanding customer.<p><i>&quot;[4] But realize that customers may not express what they really need; don’t take requirements at face-value, instead spend the time to understand their use case in detail. Read their code.&quot;</i> - Yep - Users Are Liars. They will lie right to your face, about requirements, deadlines, use cases, functionality, costs, etc, for no reason at all. Never believe anything a user says. And obviously, never tell them you think this.<p><i>&quot;[9] If you get good and&#x2F;or aligned managers, be as understanding, supportive, and accommodating as you can. If you don’t get such managers…&quot;</i> - Become the shadow manager; Team Lead, Lead Developer, Delegate Architect, etc. Ask the manager if you can get &quot;hands-on experience&quot; with various tasks like: running ceremonies, taking meeting notes, coordinating work, planning, resolving blockers. Reach out to other teams in private to resolve problems without involving any management, and be careful that the people you reach out to don&#x27;t widen scope to their managers. Hide problems and bubble up accomplishments to the bad manager. You&#x27;ll get none of the credit, but the manager won&#x27;t have the opportunity to make everyone&#x27;s lives bad.<p><i>&quot;[10] Make your project robust to re-orgs.&quot;</i> - Of these, the most resilient I&#x27;ve seen are the ones that promise the moon, and then deftly delay everyone&#x27;s realization that it doesn&#x27;t live up to the hype until they&#x27;re all deeply bought in. Don&#x27;t make the mistake of trying to get an executive to buy in, because they&#x27;ll be gone in 3 years and everyone will ditch your project.<p><i>&quot;[15] Design APIs with migration to new implementations as a first-class consideration&quot;</i> - Yes, yes, yes!! Literally all technology is replaced eventually, most of it in 3-5 years. Plan the exit strategy at the outset.<p><i>&quot;[21] Have the right number of abstractions&quot;</i> - The correct number of abstractions is the minimum possible. Every abstraction should be a solution to a problem. You have to define the problem that the abstraction was meant to solve, and then ask yourself if there is a simpler way to solve that problem.
alex64超过 3 年前
This is a good list, but I&#x27;m trying to understand the following in better detail:<p>&gt; [12] Be conservative on APIs and liberal with implementations.<p>&gt; [18] Maintain multiple implementations in test for APIs; compare results between them. The cost is worth it (it will help with correctness, and also prevent leakage of implementation detail).<p>The word implementation is thrown out a lot. What does implementation mean in these quotes? The databases used (ie. dependency injection), the endpoints (ie. &#x2F;user, &#x2F;user&#x2F;123), or something else?<p>I&#x27;m trying to visualize this in my head and am having trouble.
评论 #29328210 未加载
评论 #29327973 未加载
评论 #29328063 未加载
vincnetas超过 3 年前
This list is amazing for most software that i worked on. Dropping &quot;database&quot; from list name would only remove couple of items from the list. Most of the items are applicable much broader than just for database projects. Favourited for sure for future references.
lysecret超过 3 年前
The main thing i learned in my first production db work is that you pay for every bit of complexity you introduce. As a noob you are somehow much more afraid to not &quot;model the data right&quot; so you tend to create much to many tables much to many n to m relationships you make everything a string a uuid a relationship, never use bools etc. But then you have to iterate and live with those systems and then you pay and every bit of complexity makes everything harder. To give you some simple examples:<p>1. We don&#x27;t want to delete old data so we just use a &quot;deleted&quot; column and do that instead. Now every single time you have to filter out those results. 2. You just made this thing a relationship now every time you have to join.<p>In the end, if you can keep it simple even if you don&#x27;t model the data &quot;perfectly&quot;. But of course sometimes the complexity is really necessary. To know when is being a good Backend &#x2F; Db Engineer.
评论 #29328832 未加载
评论 #29328735 未加载
dalanmiller超过 3 年前
&gt; [12] Be conservative on APIs and liberal with implementations.<p>Curious how people interpret this? Is it:<p>* Attempt to keep our API small while also... * thinking widely about the way it&#x27;s used?<p>Or how does this work in practice?
评论 #29328024 未加载
e12e超过 3 年前
In this kind of structure, what role does managers fill? It sounds like ICs can manage themselves, and the project manager manages the project... What&#x27;s left for managers to do?
评论 #29330855 未加载
nyanpasu64超过 3 年前
Is a diff Facebook&#x27;s equivalent of a feature branch or PR?
评论 #29326837 未加载
评论 #29326658 未加载
评论 #29326656 未加载
tyingq超过 3 年前
There was a comment asking what an &quot;IC&quot; was, but the comment was flagged and now dead.<p>Guessing by the context, I&#x27;m pretty sure it means &quot;individual contributor&quot;, or &quot;person that is not a manager&quot;.
评论 #29325135 未加载
评论 #29324664 未加载
评论 #29324650 未加载
评论 #29324784 未加载
评论 #29324909 未加载
nzgrover超过 3 年前
Nowhere does it say what an &quot;IC&quot; is. Is it so hard to provided the Full Name(FN) and the abbrevation the first time you use it?
评论 #29324538 未加载