The Opposite of Documentation is Superstition (2020)

I&#x27;ve called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there&#x27;s no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works.<p>Now, for some interfaces, this isn&#x27;t too bad. Most people don&#x27;t know why US AC power plugs are polarized, or what the ground prong is for. Electricians have to, but users do not.<p>For more complex interfaces, it means that many functions will either be misused or undiscovered. This is the source of the plaint that only 10%-20% of a product&#x27;s features are used.<p>On user interface design, the classic is &quot;Tog on Interface&quot;, Bruce &quot;Tog&quot; Tognazzini, 1992. That&#x27;s from the Mac UI era. A more modern take is &quot;The Gamer&#x27;s Brain&quot;, by Celia Hodent, designer of Fortnite&#x27;s UX.<p>Software internal documentation seems to suffer today from a mindset that comments are unnecessary and waste space. Especially in the Javascript era. It&#x27;s worst in languages that don&#x27;t have data declarations. There&#x27;s no place to properly document the data.<p>Rust has good conventions and tools for documenting data. There&#x27;s a well defined place where the documentation for each structure and field goes, and reasonable tools for checking it and turning that into documentation. If you fail to do this, when you publish your crate on crates.io, the documentation pages will come up blank, which screams &quot;loser&quot;.<p>Rust is weak on function parameter documentation. There should have been a defined place where each formal parameter gets a comment, which then appears in the documentation of the call.<p>Most other languages don&#x27;t take such a hard line. More should.

I can&#x27;t remember the number of times I have given up on software because there is no “Concepts and Facilities” document that gives me a mental model of what the program will do and how I operate it. Instead, one sees a website with a mass of unrelated documents that I&#x27;m supposed to read in some order, and divine how everything works.<p>Maybe because I&#x27;m a child of the 20th century that I value documentation that gives me a path from understanding what and why to learning how I can use it, with a master index so I can look up exactly what the “Mung Until No Good” command does, and see which other commands relate to it.<p>Perhaps one day generative AI will be good enough that we can feed in one of these websites, with its pages with titles like “Migrating from V4.7.2 to V4.7.3” and “Building for OS&#x2F;2 Warp” (I exaggerate, but only slightly) to documentation that is useful for learning, use, and troubleshooting. I live in hope.

He&#x27;s really talking about the user&#x27;s mental model, here.<p>I&#x27;ve found that it is sometimes a good idea to reinforce mental models that don&#x27;t actually reflect the operation of the software (in my case) under it.<p>Documentation is good, but, in my experience, almost no one ever reads it. I can tell, because of all the questions that I get, that are in the docs.<p>That&#x27;s totally my fault. A really good tech writer could probably do a better job of conveying information.<p>That said, I think that it&#x27;s important to present good, discoverable UI, that doesn&#x27;t punish users for exploring.

&quot;Substition is the Discworld opposite of superstition. Superstition being the belief in the things that everyone believes in, but which are not true. Substition is the belief in things which are true but most people do not believe in.&quot;<p><a href="https:&#x2F;&#x2F;wiki.lspace.org&#x2F;Substition" rel="nofollow">https:&#x2F;&#x2F;wiki.lspace.org&#x2F;Substition</a>

Seems to me documentation can also be superstition if it&#x27;s out of date.<p>I wonder if LLMs can generate docs at any reasonable level of usefulness and accuracy<p>Speaking of which, I just tried using ChatGPT to create a React app with typescript and rollup. What it generated didn&#x27;t work (got errors). Of course I could tell it the errors and it fixed them but it suggests they aren&#x27;t yet ready to write accurate docs

Right now, I&#x27;m working on a SwiftUI project that displays charts of various statistics for our app system.<p>The docs for SwiftUI are ... <i>less than ideal</i>.<p>I&#x27;m pretty sure that it can do what I need, but the docs don&#x27;t cover it <i>at all</i>. I&#x27;ll need to keep playing with code completion (which kinda sucks, right now, because Xcode keeps making up nonexistent APIs), and looking at the actual system exported headers.<p>I&#x27;ll get it, eventually, but this is pretty crazy bad.

&quot;Superstition&quot; results from any number of things.<p>A) insufficient docs B) programmer can&#x27;t be bothered to read the docs C) programmer doesn&#x27;t trust the docs D) programmer had a bug once, blames the underlying library, not his code E) programmer had a bug, changes code, bug goes away, jumps to conclusion.<p>Trust in the docs is a big thing. If the docs are wrong a lot the the programmer is skeptical even when they&#x27;re right.<p>But I&#x27;ve also noticed that most developers simply don&#x27;t read them. &quot;I&#x27;m too busy (fixing bugs) to read the docs.&quot; Or &quot;my backlog is so long I don&#x27;t have time to learn how to do it right, I just want it done fast &quot;<p>Superstition occurs with the absence of knowledge - the existence of docs is necessary, but not sufficient, in overcoming that absence.

I have also used the phrase &quot;tribal knowledge&quot; in the documentation-vacuum context.

The pigeon experiment is a great one to learn from not just about programming or software, but about life in general. Where are you getting your next dopamine hit? Is it random? Maybe that’s where our idiosyncrasies come from.

&gt; By the end of the experiment, six of the eight pigeons developed nonsensical &quot;superstitions&quot; to get food<p>Almost every human will do this for something IRL. As semi-irrational creatures, we tend to forcibly seek out patterns where there are none.<p>Some people become so encrusted with such behaviour that it legitimately impacts their ability to operate effectively in modern society.<p>Not surprisingly, the religiously-afflicted tend to pack the upper end of this spectrum.

One also could say that the opposite of open source is superstition.<p>It works with the example given in the article: OneNote is proprietary software, and it is documented, though incompletely. Had it been open source, one could see exactly how shape recognition works, bugs, quirks and all. No documentation will give that amount of detail.<p>Sure, not everyone has the time and skills for that, but maybe someone on stackoverflow does and have answered the question <i>with proof</i>, that is, not a superstition.

This is an interesting case, since the pigeon study is about what happens when the underlying process is random.<p>But if the shape drawing process isn&#x27;t random, I think the author&#x27;s experience of feeling unable to articulate the rules AND gravitating to a set of behaviors is a good example of procedural memory (implicit vs explicit).<p>Explicit rules would probably help speed things up, though!

The opposite of docs is also expertise: learn through experience, then have a career moat. It’s why Microsoft and Apple get away with secret API, which also gives them some leverage. Docs tend to be good only in the post-prototype, pre-dominance phase of rapid growth. Or when the author cares about users.

This refers to user docs rather than your team&#x27;s README.md.<p>Programmers have the luxury of documenting the previously undocumented for their repos (assuming management allocates time).<p>End users would need to reverse engineer but that assumes predictability. Without that you get superstition as per article.

Functionality doesn&#x27;t exist if it&#x27;s not communicated, whether through intuitive UI or documentation. Much like laws don&#x27;t exist if they&#x27;re not enforced.

Superstition plus documentation equals religion.

Isn&#x27;t this more likely an example of LLM-crappification?<p>As in, the reason it isn&#x27;t documented is that no one has a clue...

I thought the opposite was called cargo cult.

the fun of that study is that the pigeon behavior they think is to get food is superstition it self.

Technically the opposite of documentation is anecdote.<p>Literal interpretation of anecdotes lead to superstition.<p>Ironically this leads to them becoming documented.

Some of the best technical documentation I’ve ever written has straight up lies in it. Granted they were placed there in the service of distilling the most germane portions of the API I was trying to explain. But it made a huge difference once I decided my code in the library didn’t have to be my code in its documentation. I felt a little like a biographical writer who makes a composite character. But honestly I recommend trying it. Lie in your docs if it serves the greater goal of communicating what’s most important. Let the computer labor in the “truth” and its tediums.