TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Normalization of Deviance (2015)

447 点作者 akshaykarthik超过 2 年前

28 条评论

aj7超过 2 年前
“Let&#x27;s say you notice that your company has a problem that I&#x27;ve heard people at most companies complain about: people get promoted for heroism and putting out fires, not for preventing fires.”<p>My first day at work at big-laser-company. Manufacturing engineer for a laser (then) so complex, it required a PhD to solve problems to get units out the door. The product was a ring laser. What that means is that the laser beam travels around in a race track pattern inside the laser before getting out, not a back-and-forth bouncing between two mirrors. Now this laser could be tuned to any wavelength by suitable setups and machinations, and once there, would “scan” a small amount about this wavelength, enabling scientists to study tiny spectral features in atoms and molecules with great precision. I knew all this shit. I was a Berkeley-trained physicist that built precision lasers out of scrap metal for my thesis. First day of work. I walk into the final test lab. The big laser was happily scanning away. The bright yellow needle-like output beam was permitted to hit the lab wall. As the laser scanned, the beam was MOVING on the wall. Whereupon, first day of work, I exclaimed the most obscene four words in manufacturing, for all to hear, “You can’t ship that!” (“Beam pointing instability” is detrimental to almost any laser application. It turns out that during scanning, an optical element was rotating, on a shaft, inside this laser. This mechanical motion caused beam motion.”) Well, I got an immediate reputation as a negative guy. (You can tell it’s deserved.) The solution was to retrofit 28 lasers in the field, mostly in Europe, with a component that cancelled the movement, on an expensive junket by a service guy. Who was hailed as a ”hero.”
评论 #34798049 未加载
评论 #34796093 未加载
评论 #34796063 未加载
评论 #34810664 未加载
评论 #34835685 未加载
评论 #34801943 未加载
chaps超过 2 年前
These stories ring so, so true. Once worked at a company whose infrastructure issues were so deep and festering that after fighting a fire, my boss told me, &quot;If you go to the press about this, the client will sue us and everyone who works here will lose their jobs.&quot;
评论 #34792229 未加载
dang超过 2 年前
Related:<p><i>Normalization of Deviance (2015)</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22144330" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22144330</a> - Jan 2020 (43 comments)<p><i>Normalization of deviance in software: broken practices become standard (2015)</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=15835870" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=15835870</a> - Dec 2017 (27 comments)<p><i>How Completely Messed Up Practices Become Normal</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10811822" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10811822</a> - Dec 2015 (252 comments)<p><i>What We Can Learn From Aviation, Civil Engineering, Other Safety-critical Fields</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10806063" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10806063</a> - Dec 2015 (3 comments)
评论 #34801112 未加载
PaulHoule超过 2 年前
I like the bit about<p><pre><code> Let&#x27;s look at how the first one of these, “pay attention to weak signals”, interacts with a single example, the “WTF WTF WTF” a new person gives off when the join the company. </code></pre> and kinda wonder if a company that prioritized not getting this reaction from new hires might find it is the most impactful thing they can do in terms of culture.
评论 #34792846 未加载
评论 #34792672 未加载
评论 #34791797 未加载
评论 #34792020 未加载
评论 #34792944 未加载
评论 #34795982 未加载
kurthr超过 2 年前
There&#x27;s been a lot of work on reliability of complex systems and how they operate. What has been found is that it is almost always necessary to have failure (degraded operation) modes that prevent system failure, and the more complex and more hazardous failure is the more modes develop.<p>In these systems it is found that they are almost always operating (or transitioning between) failure modes. Often multiple operational failure modes are simultaneous. It becomes very important to test the system in each of it&#x27;s failure modes and their combinations to maintain high up time.<p><a href="https:&#x2F;&#x2F;how.complexsystems.fail&#x2F;" rel="nofollow">https:&#x2F;&#x2F;how.complexsystems.fail&#x2F;</a> is an example, but there are many.<p>Human work, development, and maintenance is itself a system that interacts with these critical systems. Frankly, failure to fail causes failure (thus chaos monkey). The mythical man month is almost a sub category of these failures as are HR hiring processes and other BS. Being too successful and not having competition (or similarly sclerotic competition) can be as much of a hazard as &quot;move fast, break things&quot;.
评论 #34801817 未加载
stevehawk超过 2 年前
This is a big term in aviation, because in most cases in order for something catastrophic to happen it requires a lot of things to have failed. And one way to ensure that enough things fail is to start deviating from your maintenance, inspections, or general responsibilities. Related: the swiss cheese models.
评论 #34792587 未加载
somat超过 2 年前
A thought experiment.<p>When is it &quot;Normalization of Deviance&quot;? and when is it a &quot;Efficiency Optimization&quot;?<p>I mean, the difference is pretty clear after something has failed, But very murky before.
评论 #34794369 未加载
Logans_Run超过 2 年前
I&#x27;m not sure if this link has already been posted but have a look at <i>How I Almost Destroyed a £50 million War Plane and The Normalisation of Deviance.</i><p><a href="https:&#x2F;&#x2F;www.fastjetperformance.com&#x2F;blog&#x2F;how-i-almost-destroyed-a-50-million-war-plane-when-display-flying-goes-wrong-and-the-normalisation-of-deviance" rel="nofollow">https:&#x2F;&#x2F;www.fastjetperformance.com&#x2F;blog&#x2F;how-i-almost-destroy...</a>
renewiltord超过 2 年前
In <i>The Field Guide to Human Error Investigations</i> by Sidney Dekker, he quotes someone else saying something like:<p>&gt; <i>Everything that can go wrong will go right.</i><p>Murphy&#x27;s Law then manifests from escaping disaster through repeated iterations of taking risks where most things play out well anyway.<p>I have to laugh at the &quot;append z to the end&quot; strat at Google, though. That&#x27;s a good one.
overengineer超过 2 年前
Here are a few examples from real-world history that reflect problems discussed in the article:<p>- The Space Shuttle Challenger disaster in 1986 was caused by the normalization of deviance, where engineers became accustomed to problems with the O-ring seals and began to accept them as normal. This led to the eventual catastrophic failure of the shuttle&#x27;s launch, killing all seven crew members.<p>- The 2008 financial crisis was caused in part by a normalization of deviance in the banking industry, where risky and complex financial instruments were routinely used without proper oversight or understanding of the potential risks. This led to a widespread collapse of the financial system and a global economic recession.<p>- The Volkswagen emissions scandal in 2015 was caused by a normalization of deviance in the automotive industry, where engineers and executives became accustomed to cheating emissions tests and misleading customers about the true environmental impact of their vehicles. This led to significant financial and reputational damage to the company.<p>- The Theranos scandal in 2018 was caused by a normalization of deviance in the healthcare industry, where the company&#x27;s leaders became accustomed to misrepresenting the capabilities of their blood testing technology and misleading investors and customers about its accuracy. This led to significant legal and financial repercussions for the company and its executives. (ChatGPT)
deanCommie超过 2 年前
Reality: It is true that EVERY organization is broken in some way or another.<p>You have to find the one that is broken in the way that is tolerable to you.<p>Arguably the closest we know to a panacea in terms of engineering culture and best practices is Google. And what are they now known for? An inability to ship anything meaningful anymore. Spinning around in circles launching and re-launching new chat apps.<p>These are not unrelated. High engineering standards are always in tension with product delivery. As a security engineer once told me, &quot;the most secure system is the one that never gets launched into production.&quot;<p>So while Dan is right, and all the examples are right, and things like non-broken builds and a fast CI&#x2F;CD pipeline are totally achievable, don&#x27;t learn the WRONG lesson from this which is that when you arrive to a company and notice a bunch of WTFs, the first thing you must do is start fixing them in spite of any old timers who say &quot;Actually that&#x27;s not as bad as it seems&quot;. Sometimes they&#x27;re wrong. USUALLY, they&#x27;re right.
kerblang超过 2 年前
Great stuff - I think this goes in the &quot;required reading&quot; list.<p>The tech industry tends to revolve around &quot;I&#x27;m a super-rational robotic genius&quot; thinking that can&#x27;t accept the existence of its own irrational tendencies, to the point that it becomes ridiculous.
评论 #34792623 未加载
评论 #34792761 未加载
justin_oaks超过 2 年前
I welcome others to share stories of the normalization of deviance in their companies.<p>One company I worked had no unit tests, no infrastructure as code, and no build server. This held strong for a while until enough developers implemented some unit tests, infrastructure as code (e.g. terraform), and a build server as skunkworks projects. Eventually management tolerated them, but never endorsed them. Some teams at the company still never embraced good practices because it wasn&#x27;t forced on them.<p>I guess I&#x27;ve never worked at a company that valued unit tests across the whole of the engineering team. I introduced them and implemented them on my own team, but others ignored it.
评论 #34796817 未加载
superpope99超过 2 年前
Has Dan Luu ever explained why he doesn&#x27;t put dates in his blog posts?
评论 #34792413 未加载
aeturnum超过 2 年前
If you enjoyed this - I highly recommend watching Adam Curtis&#x27; <i>Can&#x27;t Get You Out of My Head</i>: <a href="https:&#x2F;&#x2F;thoughtmaybe.com&#x2F;cant-get-you-out-of-my-head&#x2F;" rel="nofollow">https:&#x2F;&#x2F;thoughtmaybe.com&#x2F;cant-get-you-out-of-my-head&#x2F;</a>
theptip超过 2 年前
As a new hire there is a line to walk between on one hand, using your outside&#x2F;fresh perspective to provide valuable insight to the org, and on the other, complaining (or appearing to complain) about decisions where you don’t have full context.<p>Many of the examples in the OP are probably closer to the former, but my general advice here is to keep lots of notes about what seems broken, and revisit in a month or two. Sometimes you gained context that explains why something is actually sensible. If it still seems crazy with context, you can now bubble up the feedback with confidence, and also having hopefully built some respect and trust from the team to make the message land better.
bluedino超过 2 年前
Write total shit for code, then look like a &#x27;genius&#x27; for &#x27;fixing&#x27; bugs, only to have them come back again in the future (further looking like a clown to the rest of the team)
dec0dedab0de超过 2 年前
I&#x27;m still reading, but I just got to the part about flaky, and I got annoyed because there are clear use cases for flaky or pytest-retry.<p>If you have an integration test that relies on an unreliable system you do not control. Sure you can mock it out for a unit test, but if you want to make sure you catch breaking API changes, you need to hit the actual system. And if it works after retrying it a few times, then so be it. no need to throw shade.
评论 #34793338 未加载
评论 #34792071 未加载
shadytrees超过 2 年前
formatted for wide monitors <a href="https:&#x2F;&#x2F;ddanluu.com&#x2F;wat" rel="nofollow">https:&#x2F;&#x2F;ddanluu.com&#x2F;wat</a>
pizzaknife超过 2 年前
i routunely remind everyone,&quot;we&#x27;re all mercenaries.&quot;<p>i have marginal control over who i manage. The Product isnt saving the world, but it is allowing us to live reasonably and with a clear soul at the end of the sprint. The reason i say the &quot;mercenary&quot; bit is simple: weigh your dreams against blood and gold and compromise.
Wistar超过 2 年前
AOPA: Normalization of Deviance in Aviation<p><a href="https:&#x2F;&#x2F;www.aopa.org&#x2F;news-and-media&#x2F;all-news&#x2F;2015&#x2F;december&#x2F;07&#x2F;the-normalization-of-deviance" rel="nofollow">https:&#x2F;&#x2F;www.aopa.org&#x2F;news-and-media&#x2F;all-news&#x2F;2015&#x2F;december&#x2F;0...</a>
calmdown13超过 2 年前
This rang so true for me. I’m constantly rediscovering things that I already understood well at my previous job. Once you get acclimatised to the current system, so many previously obvious learnings fall by the wayside.
irsagent超过 2 年前
&quot;rictus of horror&quot; - What a set of words to describe a response.
afahad超过 2 年前
I couldn’t finish reading this. Horror film. A screenplay to dystopian dread.
thomastjeffery超过 2 年前
Our society places an enormous value on &quot;competitive obfuscation&quot;.<p>As an obvious result, our society does an <i>incredible</i> amount of work maintaining that obfuscation.<p>---<p>I&#x27;ve heard estimates that 20% (1&#x2F;5th) of all healthcare-related spending in the US is overhead from insurance determinations, paperwork, etc., and that that 25% (25%&#x2F;125%=1&#x2F;5th) extra spending (relative to 100% of the rest of healthcare expenditure) <i>does not exist</i> in single-payer healthcare systems, like those used in Canada, Germany, and every other developed nation in the world.<p>What do we get from that extra spending? What substantive difference does that obfuscation provide?<p>The main difference I see is &quot;explicit opportunity cost&quot;. Instead of deciding ahead of time that we will pay for any arbitrary healthcare need (as a single-payer program), the <i>opportunity</i> for each individual healthcare act is given a price, and groups of priced opportunities are provided by subscription-based insurance plans.<p>Every person has to find, apply for, and pay for an insurance plan that will meet their current and future healthcare needs.<p>Because that is <i>explicit</i>, there is leverage available to manipulate each opportunity cost, and even the opportunity of each person to have that opportunity provided to them.<p>So what does that leverage even look like, and who is using it, and for what purpose?<p>Politics. Instead of care being determined by your doctor, access to each type of care is explicitly made available (or unavailable) by your insurance plan. That&#x27;s a huge attack surface for political motivation.<p>There is currently a dextroamphetamine (Adderall) shortage in the US. The other day, I went to my pharmacy to pick up my prescription for 30 generic Concerta (methylphenidate extended release, another stimulant medication used for ADHD), and learned that all they had left were 16 brand-name Concerta. I was lucky enough to have that covered by my insurance. Many different insurance plans would not have provided me that opportunity.<p>Why is there a shortage? Despite a significant increase in ADHD diagnosis last year, the DEA refused to raise the limit of Adderall that can be legally manufactured. Why? Because there is a long-standing political conflict between stimulant addiction prevention and ADHD treatment, and the DEA is positioned at one side of it.<p>That same political conflict is why some insurance companies have outright refused to include coverage for stimulant medications. Even without a nationwide shortage, some people have found themselves stuck in a position where the opportunity for medication is held just out of reach by the political decision of their insurance company, or the lack of access to insurance at all.<p>The same pattern can be found with practically every type of medical care that is politically controversial: contraceptives, abortions, hormones, etc. Even if you can&#x27;t get a legislative ban, there is still leverage available to obfuscate <i>opportunity itself</i>.<p>When conservative politicians argue that a single-payer program would be &quot;too socialist for America&quot;, the substantive difference they intend to preserve is the political leverage that is baked into the system we have; the political leverage that allows politics to restrict our medical care without a single vote.<p>---<p>That&#x27;s just one example. This pattern is everywhere. The only answer is social objectivity. It&#x27;s a hard problem.
AlbertCory超过 2 年前
This guy needs to organize &amp; format his writing better, since he does have really interesting things to say.
评论 #34795582 未加载
评论 #34792419 未加载
评论 #34792158 未加载
评论 #34794421 未加载
评论 #34794724 未加载
评论 #34792980 未加载
martopix超过 2 年前
&gt; Have you ever mentioned something that seems totally normal to you only to be greeted by surprise?<p>Once some (foreigner) person was surprised at my dipping toast with Nutella in my latte. I was equally surprised by his surprise.
quickthrower2超过 2 年前
&gt; It&#x27;s technically possible to use @flaky for that, but in practice it&#x27;s used to re-run the test multiple times and reports a pass if any of the runs pass<p>This is useful and fine. Someone wrote a test and it now hits a race condition or something and occasionally fails. Let’s assume we are very confident it is problem with the test not the product.<p>Choices:<p>Spend a sprint trying to fix it right now regardless of priority.<p>Turn it off and lose that coverage.<p>Buy some time.<p>In this context it makes sense. As long as their is a procedure to address these in some sane timeframe.<p>Maybe that is an example of normalization of deviance. But I think if it is discusses and trade offs thought through it is an OK thing to do at times. Remember most development is not green field. You inherit a system when you start a job.