Tacit Knowledge Is Dangerous

Tacit knowledge is inevitable... but it&#x27;s also different from what this article is about. Tacit knowledge is the kind of knowledge (and skill!) that cannot be fully explained or taught in words—think mechanical skills you can only pick up through physical practice or very context-specific expertise you only get through experience. In programming <i>good taste</i> is crucial and entirely tacit: we can <i>try</i> to distill taste into words and write books about it, but it will never be enough.<p>Tacit knowledge is not the same as explicit knowledge that just happens to not be documented. Tacit knowledge is not documented because it is undocumentable. You cannot avoid tacit knowledge. Human language fundamentally cannot express all the knowledge that experts develop through years of practice and experience. Even if an expert can express their experience, there are simply ideas and skills that people cannot learn exclusively from words.<p>This is an important distinction. Trying to eliminate undocumented explicit knowledge is useful for teams. Trying to eliminate tacit knowledge is disastrous. I&#x27;ve seen attempts to make tacit knowledge legible to management—it is one of the most direct ways to sabotage your own experts because experts fundamentally <i>cannot</i> make <i>all</i> their knowledge explicit. Expertise <i>is</i> tacit knowledge and tacit knowledge <i>is</i> expertise.<p>People generally understand that design by committee or design by regulation has awful results; one of the main reasons for this is that committees require decisions to be fully explicit and explainable—which hamstrings creative problem-solving and pushes for poor design decisions. Exclusively explicit processes get you lowest-common-denominator results: not the lowest common denominator of the expertise present, but the lowest common denominator of <i>what people can explicitly communicate and understand</i>, which is even lower! If you don&#x27;t let experts <i>be experts</i>, you won&#x27;t get expert-level work.

At a previous company where I had a lot of this &quot;tacit knowledge&quot;, I had made a policy for myself. Whenever somebody asked me a question over IM or e-mail, I would look in the wiki. If the answer was not in the wiki, I would write a new wiki article or update an existing article with the answer, and I would respond with a link to the article, and a statement like &quot;if this doesn&#x27;t answer your question let me know what is missing or could be described better.&quot; The reasons for this were for me to dump my tribal knowledge out of my brain into something everybody could search and read, and to promote usage of the wiki as a place where people found answers. Unfortunately it did not appear to have another desired effect of getting everybody to contribute content. I think this was probably because people came to see the wiki as a place to find knowledge, not to put knowledge.

Per other comments here, this is not what tacit knowledge is. There is an entire body of research devoted to understanding tacit knowledge. Anyone interested in tacit knowledge can read the works of those who have spent years researching the topic. Here are some sample papers. Nonaka is perhaps the most well-known one in this space.<p>Hadjimichael, D., and Tsoukas, H. 2019. “Toward a Better Understanding of Tacit Knowledge in Organizations: Taking Stock and Moving Forward,” Academy of Management Annals (13:2), pp. 672–703. (<a href="https:&#x2F;&#x2F;doi.org&#x2F;10.5465&#x2F;annals.2017.0084" rel="nofollow noreferrer">https:&#x2F;&#x2F;doi.org&#x2F;10.5465&#x2F;annals.2017.0084</a>).<p>Huang, X., Hsieh, J. P.-A., and He, W. 2014. “Expertise Dissimilarity and Creativity: The Contingent Roles of Tacit and Explicit Knowledge Sharing,” Journal of Applied Psychology (99:5), pp. 816–830.<p>Nonaka, I., and Von Krogh, G. 2009. “Perspective—Tacit Knowledge and Knowledge Conversion: Controversy and Advancement in Organizational Knowledge Creation Theory,” Organization Science (20:3), pp. 635–652. (<a href="https:&#x2F;&#x2F;doi.org&#x2F;10.1287&#x2F;orsc.1080.0412" rel="nofollow noreferrer">https:&#x2F;&#x2F;doi.org&#x2F;10.1287&#x2F;orsc.1080.0412</a>).<p>Nonaka, I. 1994. “A Dynamic Theory of Organizational Knowledge Creation,” Organization Science (5:1), pp. 14–37.

&gt; Tacit knowledge, often called “tribal knowledge” in tech, is prevalent in this industry.<p>Tacit knowledge is not the same as tribal knowledge. Tribal knowledge is undocumented stuff. Tacit knowledge is about things that cannot be learned from documentation.<p>Riding a bike is an example of tacit knowledge. No amount of reading about the theory of riding a bike will teach you how to ride a bike. You have to hop on the bike and fail a few times until you figure it out.<p>In software engineering, tacit knowledge are things like &quot;How to factor a system so 50 engineers can work together without stepping on each other&#x27;s toes&quot; and &quot;How to structure your files to naturally reduce architectural complexity&quot; and &quot;How do you write a useful test that isn&#x27;t just writing tests for the sake of tests&quot;. You can read about these things all you want, but you can only learn by experience. You have to get it wrong a bunch of times, see how it was wrong, and then you eventually start doing it less wrong.<p>Learning by example also works to speed up the process.<p>But that&#x27;s not the same as undocumented.

This looks like a really good use case for LLMs. When you&#x27;re small and starting out in person, record everything the dev team does -- meetings, whiteboads, pairing sessions, etc. (yeah, creepy, but I&#x27;m in brainstorming mode) and then as you start to scale beyond the ability to &quot;ask someone who knows&quot; trail up an LLM to serve as the 24x7 lore master.

I used to work with a smart guy who had what I think is the real answer to this problem. All documentation is maintained by the newest member of the team. The new guys job is to solve all of his issues with the documentation and if he has to ask someone then his job is to update the documentation.

This is an odd definition for tacit knowledge. My understanding is that it&#x27;s more caught up in the intuition of concepts&#x2F;systems that&#x27;s difficult to codify because it&#x27;s very contextual. This &quot;tribal knowledge&quot; perspective seems more like processes and facts that aren&#x27;t documented. Not because of the inherent difficulty but instead because of priorities. Not that there isn&#x27;t some intersection between the two.

Well, as in all of these absolutist black&#x2F;white, binary statements, &quot;it depends.&quot;<p>In my experience, tribal knowledge, which also includes cultural inculcation, can be the difference between success and failure. Sometimes, on the success side, and, sometimes, on the failure side.<p>The main thing about &quot;tribal knowledge,&quot; is that it relies on long-term association&#x2F;employment, and that is not something we see much of, these days.

Perhaps such products are in the works, but I would love a private company LLM that slurps all messages, docs, meetings, and becomes an oracle that can be relied upon. Perhaps it could even infer tacit knowledge even when it&#x27;s not explicit in any docs, since it could abstract principles and concepts from the maze of data.

I only once saw proper „knowledge“ documentation working in a german company which had a strict waterfall methodology. When project started, we spent at least 1&#x2F;3 of the time writing documentation and specification first, then code. Onboarding was a breeze and when corona hit - nothing has changed, we could operate as usual too. I also had to overtake a domain of another engineer who was leaving, the handover was minimal, as everything was written down.

&gt; You may really need to know about the whizbang service, but it’s been 4 years since anyone last worked on it and no-one remembers how it works. You’ve now fallen into the trap of tacit knowledge.<p>This article is bad. What it describes - as others have pointed out - is not &quot;tacit knowledge&quot;. It merely describes a problem with lack of documentation.<p>&quot;Whizbang service&quot; can be documented. You can write a diagram laying out the main components, write a small readme on how to build and run locally on against some staging environment, perhaps even having some internal wiki with a internal runbook of what to do when the service goes tits up.<p>None of this would touch on &quot;tacit knowledge&quot;. Tacit knowledge is there when I get a stack trace, and I already sort of suspect what happened without having to spend hours digging too deep on why. Or when I see some latency metrics for my service, and my first instinct is checking on Slack for deployments on dependencies. You can try to document those things, but it is a fool&#x27;s errand - you&#x27;ll end up with a huge mess of text that will work against the purpose of documentation.

The US Army has formal training about this and provides an Advanced Skill Identifier (ASI) to those training to practice it. See:<p>* Techniques for Effective Knowledge Management, ATP 6-01.1, <a href="https:&#x2F;&#x2F;armypubs.army.mil&#x2F;epubs&#x2F;DR_pubs&#x2F;DR_a&#x2F;pdf&#x2F;web&#x2F;atp6_01x1.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;armypubs.army.mil&#x2F;epubs&#x2F;DR_pubs&#x2F;DR_a&#x2F;pdf&#x2F;web&#x2F;atp6_01...</a><p>The Army technique follows the DIKW model. <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;DIKW_pyramid" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;DIKW_pyramid</a><p>When I did that job as NCOIC for a major sustainment command in Afghanistan I learned how to mix documentation, automation, and empathy to create a shared vision amongst the 24 shops that comprised the unit. I never would have matured into this writing corporate software.<p>Since being laid off this year I am choosing to regress and aggressively embrace tacit knowledge. In the past I used to publish things I had learned to GitHub and many people found those writing extremely beneficial. As a JavaScript developer, however, I have finally been beaten into submission by the neglectful immaturity of my profession and so many of my peers.<p>I have grown tired of debasing myself when so many people are hostile to any concepts of helpful guidance. The name of the game is attain employment by writing the same framework glue over and over thereby pandering to the desires of the least competent and least employable of the workforce. Any deviation is met with hostility first and possibly curiosity later. As a result I abandoned that line of work until I found a career change and now keep my learning about automation, performance, and architecture to myself.

Speaking of tacit experience, I love watching YouTube videos made by professionals working in industries I’m just a rank amateur in.<p>One thing that occurred to me is that I’ll observe these people casually and repeatedly applying “methods of the art” without a second thought, but then spend the entire episode talking about some specific obscure issue they’re trying to solve.<p>In my mind these have equal difficulty — and in some sense they are equal! It’s just that professionals may have done basics so often they don’t even think about it any more. It’s become a habit, mere tacit knowledge.<p>Some random examples:<p>Correctly centering something in a lathe — done in incorrectly can lead to this: <a href="https:&#x2F;&#x2F;www.forbesadvocate.com.au&#x2F;story&#x2F;907281&#x2F;faulty-oil-pipe-to-blame-for-qantas-incident-says-report&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.forbesadvocate.com.au&#x2F;story&#x2F;907281&#x2F;faulty-oil-pi...</a><p>I can think of thousands of examples like that: tacit knowledge is the knowledge you gain when you do something many times a day. That doesn’t make it less important, less valuable, or more obvious to beginners.

Nice article and good points, but perhaps the author wasn&#x27;t aware that &quot;tacit knowledge&quot; is a term already used for something else.<p><a href="https:&#x2F;&#x2F;commoncog.com&#x2F;the-tacit-knowledge-series&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;commoncog.com&#x2F;the-tacit-knowledge-series&#x2F;</a><p>Tacit knowledge is undocumentable because it&#x27;s knowledge borne out of experience. It&#x27;s tacit because articulating it would be too tedious, impractical, and most probably incomplete. Think learning to play a sport by simply watching videos or reading instructions. Michael Jordan famously explained that although all players are taught the same theory, there&#x27;s a point in a veteran&#x27;s career where they can <i>read</i> the game at a different level. Closer to programming, it&#x27;s the reason we see more articles repudiating well packaged ideas, such as &quot;Clean Code&quot; (capitalization intended), as past beginners who bought into them, are becoming experienced enough to recognize them as leaky abstractions.

I find that &quot;figuring it out on my own&quot; takes a bit longer, but I learn a LOT of answers to alot of other things I didn&#x27;t know I needed.<p>&quot;If I walk into a bookstore and you help me find the book I&#x27;m looking for, then You&#x27;ve robbed me of the opportunity to find all the books I wasn&#x27;t looking for&quot; -- paraphrasing Neil Tyson

Public everything inside the company. No private rooms. Redirect questions in dms to rooms. Public docs for everything. Everyone with full access to Google drive.<p>Now your &quot;question to the other side of the world&quot; becomes a slack search, or a docs search, and is even better than in person.

I think it is good to start by defining the concept because I first hear about &quot;tacit knowledge&quot; in the context of knowledge management [1]. In that context tacit knowledge is inherent to social systems. It is something that exists and could not be prevented.<p>At the end what the author should criticize is not the tacit knowledge but how the organization manages the knowledge. And that is, knowledge management.<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Knowledge_management" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Knowledge_management</a>

Tacit knowledge is inevitable. Some people, like Turing Award winner Peter Naur, claim that the knowledge in the programmer&#x27;s head is actually the main &quot;product&quot; of programming. He calls this &quot;Theory Building&quot;. There&#x27;s an excellent paper on it, but it&#x27;s pretty verbose, here&#x27;s [1] a summary with some additional case studies.<p>[1] <a href="https:&#x2F;&#x2F;hiringengineersbook.com&#x2F;post&#x2F;autonomy&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;hiringengineersbook.com&#x2F;post&#x2F;autonomy&#x2F;</a>

Sometimes it&#x27;s best to get answers from the horses mouth - meaning the resident expert. This is ideally the person who created a thing, but can also be the person currently responsible. If no such person even exists for critical infrastructure, your company is IMO in trouble.<p>Yes, documentation is important and should be kept up to date. People should consult documentation, and update it if it turn out to be old. But real, up to date, or deep understanding does not exist outside of human brains.

Intelligent people understand that knowledge can not ever be dangerous, because it&#x27;s the human who makes things dangerous.<p>Intelligent people understand that when lesser intelligent people call knowledge dangerous, they will eventually, but definitely inevitably, ask for the banning of said knowledge, because &quot;the evil knowledge makes people do things they shouldn&#x27;t&quot;, or a variation thereof.

The place I&#x27;ve just joined has an extremely big problem with this. There&#x27;s a lot of documentation, but it all relies on an immense amount of background knowledge to be useful. It&#x27;s difficult to strike a balance between excessive detail in docs versus wide readership, but I&#x27;m inclined to prefer the former as it helps alleviate situations like this.

There is hard knowledge and there is tacit knowledge. It’s not black and white. It’s a continuum.<p>Tacit knowledge documentation can take form of pro tips, best practices, anecdotes, postmortems, retrospectives, values, and philosophy statements.<p>It’s possible to capture a bit of it in written form. It will never be fully documented. A little goes a long way though.

One easy and very useful habit we have where I work is to post all questions (even the stupidest questions) in public slack channels. That&#x27;s also where most technical conversations happen. This helps us to hide less information in private chats.

I agree with this, but I also mourn the loss of &quot;just talk to someone 20 ft away.&quot; The better the docs get, the worse the social ties get. And that&#x27;s coming from someone who was nearly useless from distraction in an open office.

Scaling organizations requires work organizing the information about how things work. Tacit knowledge allows you to get stuff done with small groups without having to pay the extra cost of investing in organization.

Even if you could express all tacit knowledge, which yoy can&#x27;t because you may not know you hold it or it&#x27;s hollistic, sharing it takes and resources.<p>And no projects have an infinite amount of those, so choices are made.

I really dislike when companies represent their knowledge bases as a list of videos. I have seen it a few times and I look forward to videos becoming searchable more easily over time.

Please leave clues as close to the scene of the crime as possible. You can&#x27;t fix everything, and some fixes are too big and take a long time, but please just leave me a clue.

Some tacit knowledge maybe is impossible to document, but in a lot of circles there is a distinct lack of ability by those who are skilled in some activity to describe how they do things. There is often a split between those who can execute at a world-class level, and those who aren&#x27;t as good but can teach others effectively. I&#x27;ve seen this dynamic play out in competitive videogames, figure skating, musicianship, etc.<p>Take figure skating for example. I was being taught how to do a basic two foot spin by someone I know who is&#x2F;was a competitive figure skater. Not a complicated move. They completely failed to explain to me a critical part of how to do the spin effectively (namely, where on my blades I should keep my balance and how I should angle them). So did several instructors leading classes on figure skating. I floundered for a while until one day I managed to figure it out by random chance.<p>I was quite annoyed, because to me it is a perfectly explainable thing that everyone was apparently either just figuring out through trial and error or just didn&#x27;t think was worth explaining. But these kinds of pedagogical &quot;blind spots&quot; pervade nearly all human fields of study.

It can also be lucrative: witness Joe Flacco, who three weeks ago was sitting on his couch watching games on TV, now with a chance to lead the Cleveland Browns to the playoffs.

No knowledge can be fully explained in words

Does this mean tacit knowledge only exists within in-person teams? So projects like the Linux kernel don&#x27;t have any tacit knowledge?

There’s less knowledge needed to manage these systems than we want to believe.<p>Tech is mired in hustle culture, which means it’s generating a lot of useless “knowledge” to role-play hustle.<p>Over engineering re-engineering to justify business headcount, importance, and prestige is a bigger danger in tech than the one this article points out