A three-page paper that shook philosophy, with lessons for software engineers

&gt; A philosopher might say that these aren’t bona fide Gettier cases. True gettiers are rare. But it’s still a useful idea...<p>At least as presented, I see the idea being used to do more harm than good.<p>Take the first example, with the form not autofocusing. We&#x27;re already not in a Gettier case, because the author didn&#x27;t have JTB. The belief that he caused the bug obviously wasn&#x27;t true. But it wasn&#x27;t justified, either. The fact that he had rebased before committing means that he knew that there were more changes than just what he was working on between the last known state and the one in which he observed the defect. So all he had was a belief - an unjustified, untrue belief.<p>I realize this may sound like an unnecessarily pedantic and harsh criticism, but I think it&#x27;s actually fairly important in practice. If you frame this as a Gettier problem, you&#x27;re sort of implying that there&#x27;s not much you can do to avoid these sorts of snafus, because philosophy. At which point you&#x27;re on a track toward the ultimate conclusion the author was implying, that you just have to rely on instinct to steer clear of these situations. If you frame it as a failure to isolate the source of the bug before trying to fix it, then there&#x27;s one simple thing you can do: take a moment to <i>find and understand</i> the bug rather than just making assumptions and trying to debug by guess and check.<p>tl; dr: Never send philosophy when a forehead slap will do.

I think it helps to look at the mind as a probabilistic survival engine than some truth engine.<p>If there appears to be a cow in a random field the odds are extremely low that someone put a papier mache cow there. If there’s something that has 50 % chance of being a snake you panic and run because that’s a 50 % chance of dying.<p>In the case of the authors bug yes the change he introduced had a good probability of being the cause. However he could have increased the probability by going back over commits and confirming that his exact commit introduced the bug. Now the probability goes even higher. But it could still be machine specific a cosmic ray or whatever but the odds are over whelmingly low.<p>In practice causal reasoning also works in a probabilistic fashion.<p>I have a simple model saying that if a planes engine is on then it’s flying. Its a single step probability and so it’s not very accurate in the real world.<p>I do a bunch of experiments and say a plane is flying if the engine is on and air is going over it’s wings faster than a certain speed.<p>Now we have two correlations connected by a causal model that works in many other cases. Hence the probability of it being correct rises.<p>But at the same time we should never mistake direct correlation for causality. But in daily life it’s “good enough”.

This is a lesson learned well from open-ended systems. An open-ended system is one where input is received to process, but the input is not well defined. The more accepted unknown input becomes the more open the system must be in its rules to process it. The results of processing are:<p>* expected output from a known input (intention)<p>* unexpected output from a known input (defect)<p>* expected output from an unknown input (serendipity)<p>* unexpected output from an unknown input (unintentional)<p>For example I maintain a parser and beautifier for many different languages and many different grammars of those languages. In some cases these languages are really multiple languages (or grammars) imposed upon each other and so the application code must recursively switch to different parsing schemes in the middle of the given input.<p>The more decisions you make in your application code the more complex it becomes and predicting complexity is hard. Since you cannot know of every combination of decisions necessary for every combination of input you do your best to impose super-isolation of tiny internal algorithms. This means you attempt to isolate decision criteria into separated atomic units and those separated atomic units must impose their decision criteria without regard for the various other atomic decision units. Provided well reasoned data structures this is less challenging than it sounds.<p>The goal in all of this is to eliminate <i>unintentional</i> results (see forth bullet point above). It is okay to be wrong, as wrong is a subjective quality, provided each of the atomic decision units are each operating correctly. When that is not enough you add further logic to reduce the interference of the various decision units upon each other. In the case of various external factors imposing interference you must ensure your application is isolated and testable apart from those external factors so that when such defects arise you can eliminate as much known criteria as rapidly as possible.<p>You will never be sure your open-ended system works as intended 100% of the time, but with enough test samples you can build confidence against a variety of unknown combinations.

The propensity for mistaking belief for facts certainly take daily hits as a software developer. &quot;How come this simple thing isn&#x27;t working? I thought of everything didn&#x27;t I?&quot;. After a while you are forced to realize that belief isn&#x27;t the same as reality.<p>It seems insights like this don&#x27;t easily translate into other domains though, like relationships, dearly held political views etc. We prefer to think of them as based on facts, when in all probability they are merely beliefs fraught with assumptions.<p>Some people might be good at being skeptics in all areas, but I sense most share my own ineptitude here, the reason probably being that any such (incorrect) beliefs don&#x27;t immediately come back and bite us, as in programming.

The &quot;cow in the field&quot; example reminds me of two heuristics I like: am I right for the wrong reason?; am I wrong for the right reason?<p>Being right for the wrong reason is dangerous: it&#x27;s not easy to spot, and it perpetuates false sense of security leaving &quot;black swan events&quot; unanticipated. This might occur during debugging as the article points out, or e.g. during A&#x2F;B testing of a product.<p>Bring wrong for the right reason is just plain frustrating.

&gt;&gt; He called them “gettiers” for short. So we used to talk about gettiers all the time, no doubt in part just because it felt clever to talk about them, but also because when you’re a programmer, you run into things that feel like Gettier cases with unusual frequency.<p>Sometimes I think that is what philosophers are doing - feeling clever - perhaps as a defense against some negative inner problem (psychology is an outgrowth of philosophy after all). The whole cow story stinks of telling someone &quot;you&#x27;re right, but you&#x27;re also WRONG! Your perception of reality is BROKEN!&quot;. To me knowledge is simply having a model of the world that can be used to make useful predictions and communicate (and some other things). Aside from that, it doesn&#x27;t matter if your model is &quot;grounded in reality&quot; until it fails to work for you, at which time it can be helpful to realize your knowledge (model) needs adjustment.<p>One way to resolving the authors first software issue would be to check a diff between what he committed and the previous production revision - this would quickly uncover the changes he &quot;didn&#x27;t make&quot;. This is an old lesson for me - what I changed may not be limited to what I think I changed. It&#x27;s a lesson in &quot;trust but verify&quot;. There are any number of ways to view it, but in the end we only care about ways that lead to desired outcomes weather they&#x27;re &quot;right&quot; or not.<p>On a related note, I&#x27;ve found that software is one of the <i>only</i> places where there is a &quot;ground truth&quot; that can be examined and understood in every detail. It&#x27;s completely deterministic (given a set of common assumptions). I&#x27;ve found the real world - and people in particular - to not be like that at all.

There&#x27;s a much closer analogy from software development to the cow story. The cow story is confusing because the cow that you see (A) is fake, but the real one (B) you don&#x27;t know about. So your belief is not a justified true belief because although the real cow exists, the one your knowledge refers (A) to isn&#x27;t the real one (B).<p>An intertemporal variant of this is race conditions. There have been lots of problems of the form &quot;(1) check that &#x2F;tmp&#x2F;foo does not exist (2) overwrite &#x2F;tmp&#x2F;foo&quot;; an attacker can drop a symlink in between those and overwrite &#x2F;etc&#x2F;password. The file that you checked for is not the same file as you wrote to, it just has the same <i>name</i>. This is an important distinction between name-based and handle-based systems.

&gt; <i>I could have had a JTB that the code change had caused the emails to stop delivering, but still we wouldn’t want to say I “knew” this was the cause, because it was actually the service outage that was directly responsible.</i><p>He is wrong and this is not a gettier in any way. &quot;The code change had caused the emails to stop delivering&quot; is not a JTB, because it is not true. Rather it was that the email server went down.

I don&#x27;t see any of his examples as Gettier cases. He thought his code caused the autofocus problem; it didn&#x27;t. He thought someone else&#x27;s push had broken email, but instead the service happened to go down at the same time. A proper Gettier case would be when you write code that you believe to be correct and it does work, but not for the reasons you think it does. Often this eventually bites when some edge case arises.<p>I run into this fairly often playing chess. I calculate out some tactic, decide it works, and play the first move. But my opponent has a move I overlooked which refutes the line I was intending to play. Then I find a move that refutes his, and the tactic ends up working in the end anyway, just not for the reasons I thought it would.

&gt;<i>A philosopher might say that these aren’t bona fide Gettier cases. True gettiers are rare.</i><p>I beg to differ. Besides the examples in programming the author gave, I can very easily think of examples in medicine, police work (e.g. regarding suspects), accounting, and so on...

I can imagine a future where what&#x27;s true generally describes itself (like terraform on drugs for software :p) Imagine software that is fully self descriptive and would no longer require engineers to individually interpret what&#x27;s happening because the software would tell us. The system would be a graph of every single component and all possible connections between them, and all variants of each component and state that it could be in. When we introduce a change we would be aware with perfect information about the affect to the states and the paths between them.<p>In the example the Mental Model was at a level too shallow, it should have only affected the paths between the autofocus and the user. But the bug necessitated a larger mental model (the author was considering too small subsection of the graph).<p>I&#x27;d hope in the future we could reach a state where the program could have detected that the frame refactor would have an affect on the autofocus and all other components instead of being an implementation detail.

Although many folks have raised the &quot;not Gettier&quot; objection, I would propose that the premise of applying the test to <i>debugging</i> is wrong. Debugging means that your assumptions were already faulty: otherwise the system would not have bugs.<p>That is, the act of programming means working on an unfinished thought, something that can reflect some beliefs but compromises on being an exactly true expression of them. And so the weight of philosophical reasoning should appear at design time. What occurs after that is a discovery phase in which you learn all the ways in which your reasoning was fallacious - both bugs and feature decisions.

&gt; (Yes, I should have caught the bug in testing, and in fact I did notice some odd behavior. But making software is hard!)<p>How often have I noticed some &quot;odd behavior&quot; in testing, and later wasn&#x27;t able to reproduce it? Some nagging feeling that I broke something remained, but since I&#x27;ve deployed a new version (that fixed something else), and I couldn&#x27;t reproduce the &quot;odd behavior&quot;, I tricked myself into ignoring it.<p>And then I deployed to production, and shit hit the fan.<p>Now I try to pay more attention to those small, nagging feelings of doubt, but it takes conscious effort.

This reminds me of a recent event at WrestleKingdom 13, a Japanese professional wrestling event where, as you might imagine, pretty much everything is planned and choreographed ahead of time.<p>In the opening match, Kota Ibushi suffered a concussion. Some doctors came out, carried him out on a stretcher, and took him to the back. As it turns out, this was all planned. The doctors were fake, and this course of events was determined ahead of time. But coincidentally, Ibushi _actually_ suffered a real-life concussion in the match.<p>Wrestling always has an interesting relationship with reality.

This is precisely why when dealing with bugs I advise juniors to avoid asking the question, &quot;what changed?&quot; Gettier cases are just one problem that you can face when asking that question.<p>Instead I usually tell them to do it the proper way: start from the bug, and work backwards to understand why that bug is happening. At that point the change that caused the bug becomes obvious, and most of the time we realize that we probably wouldn&#x27;t have come to that conclusion by looking just at what changed.

This are called false positives and are a normal occurrence in the life of a software developer. That&#x27;s why when testing the root-cause you should test for a false positive as well.

I haven&#x27;t read the original paper, so maybe the example is better, but it seems the cow example fails the justified condition. The knowledge is justified if it derives from the evidence, but once we know the evidence is faulty it can no longer be used for justification by definition. It seems by extension that any justified true belief can become unjustified by the addition of new information that invalidates the justification on which the alleged knowledge is based upon.

Sounds like (from the cases presented) an over intellectualisation of “coincidences.”

Another really great work that&#x27;s related to some of these concepts is &quot;Naming and Necessity&quot; by the philosopher Saul Kripke.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Naming_and_Necessity" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Naming_and_Necessity</a><p>It investigates how we assign names to things and what those names mean and how we can reason about them.

I know mathematical logic but I don&#x27;t know much about conventional philosophy. The Gettier argument seems to indicate that a system of false propositions used to arrive at a conclusion could be called &quot;justification&quot; in normal epistemology - that seems a bit disturbing. Being &quot;justified&quot; in believing X means just having some reason, maybe utterly bogus but existing and convincing to you, for believing X (and then X becomes knowledge even if X happens to be true entirely different reasons). How could no one have commented previously if that was how the system was?<p>Edit: the main practical implication of the argument seems to be that one cannot assume that when you have argument for X and when you then you get empirical evidence for X, you cannot take that a proof your argument for X is true. It might be suggestive of the truth of the argument but the structure of the argument also has to be taken into account. But that&#x27;s been a given scientific and statistical investigations for a long time.

I initially thought this article might be about programming by coincidence [1] or maybe about user experience superstitions [2], but after reading it I wonder if this isn&#x27;t just about the practice of debugging. Software is complex. When someone begins investigation into a bug, if the fix is not immediately found, it becomes a matter of running down a list of hypothetical causes. As one&#x27;s experience deepens, both the &quot;running down&quot; and the curating of the &quot;list&quot; become more efficient. IMHO this article is merely about a developer who was unaware of certain hypothetical causes.<p>1: <a href="https:&#x2F;&#x2F;pragprog.com&#x2F;the-pragmatic-programmer&#x2F;extracts&#x2F;coincidence" rel="nofollow">https:&#x2F;&#x2F;pragprog.com&#x2F;the-pragmatic-programmer&#x2F;extracts&#x2F;coinc...</a><p>2: <a href="https:&#x2F;&#x2F;www.nngroup.com&#x2F;articles&#x2F;quality-assurance-ux&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.nngroup.com&#x2F;articles&#x2F;quality-assurance-ux&#x2F;</a>

I actually disagree with the Gettier thought experiment and don’t believe it demonstrates anything interesting.<p>When you see the cow (but it’s really a convincing model), then in your mind, there should be some probability assigned to a variety of outcomes. The main one would be a cow, another one might be that you’re hallucinating, and so on down the list, and somewhere the outcome of cow-like model would be there.<p>From that point you can go in at least two directions, one would be something like a Turing test of the fake cow... beyond a certain point it’s a matter of semantics as to whether it’s a real cow or not, or you could say that your “justified true belief” had to apply to the total state of the field. If you believed there was both a cow model and a cow behind it, that woukd be justified, but the existence of the cow behind the model would not justify incorrect belief that the model was a real cow, in the sense of not admitting uncertainty over the things you see.

It&#x27;s a bit in the weeds, but I think the author has a wrong JTB. They author deployed multiple changes, and just incorrectly assumed that their PR was the one that introduced the bug. They author had incorrect knowledge about what was being deployed. If something in their deploy process indicated that in fact only their code was being deployed then perhaps it&#x27;s a JTB? But otherwise I think it&#x27;s just a bit off.<p>However, the gist of it is correct. We often update dependencies or deploy more than we think we do. We have an &quot;us&quot; focused view of our code, and keeping gettier cases in mind helps us break out of that.<p>Just recently I kept thinking that I didn&#x27;t know how to write a jest test, when in fact I was using a version of Jest which didn&#x27;t support a certain method. It&#x27;s easy to think it&#x27;s our fault, when in fact there can be deeper reasons.

Interesting to compare this with a simpler term - &quot;Red herring&quot; as something that throws you off course.

These cases seem to come up often (weekly?) in software development. I wonder how often they come up in other professions.<p>One common case is when you change or delete a comment, and suddenly something breaks. It couldn&#x27;t have been the comment... but it was working fine before my edit... wasn&#x27;t it?

I was surprised when I first learned that this was a novel philosophical concept, because I recall reading a Renaissance-age Italian story(maybe from the De Cameron?) that talks about this:<p>Basically, a man sees his wife walking in the town square with the hat of a friend of theirs, and this leads him to believe that she is cheating with that friend. It turns out that he just offered the hat to her in the market, to help her carry some eggs home, and she was going to return it. So, she goes and returns it, the husband follows her, and it turns out she actually is cheating on the husband with the friend, but the hat hat nothing to do with it.

This hits close to me as a possible reason why I could never get good at solving geometry problems, solid geometry especially. Most problems would be trivial when one assumes specific preconditions, but my mind was always wandering around, looking at all potential sides of a problem and I could never solve anything. To quote the author from my particular pov:<p><pre><code> a problem has multiple potential causes, and you have every reason to believe in one of them, even though another is secretly responsible. </code></pre> Reminds me that I need to pick up a book and re-learn the damn thing. It really saddens me that I suck at geometry.

I don&#x27;t understand how anyone could have ever thought &quot;justified true belief&quot; was a good definition of knowledge, since the question of &quot;what constitutes justified belief?&quot; is muddier than the question &quot;what constitutes knowledge?&quot; in the first place. Further, even without considering such absurd situations as a real cow hiding behind a fake cow, if you see something that, based on its appearance, almost certainly is a cow, the near certainty doesn&#x27;t change into absolute certainty just because what appears to be a cow is in fact a cow.

As usual, science has a more practical take on this. Occam&#x27;s Razor says that if you see a cow shape and hear a cow sound coming from that direction, the most likely explanation is that you&#x27;re seeing a cow. It retains the possibility this isn&#x27;t true; the belief can be falsified in several ways: by examining the cow more closely, by the real cow walking out from being the fake one, etc.<p>I think it follows that we never absolutely &quot;know&quot; something. We asymptotically approach knowledge. The scientific method is a way of approaching truth.

I don&#x27;t think the example of checking in someone else&#x27;s bug is a very good one. If I checked in something that had code from other people I would see it very easily, and if I did not think there was any way my code should have affected the autocomplete then I would assume the code I checked in which was not mine broke the autocomplete.<p>Matching it to the example of the papier mache cow doesn&#x27;t really work because the papier mache cow hides the real cow but it is very easy to see that your code was also checked in with other people&#x27;s code.

Somehow, the examples all sound more like a quality and&#x2F;or testing issue. The workflow seems prone to people rebasing to a buggy state and at that point, in a non-trivial system, all bets are basically off. Basically I need to be able to have a &quot;JTB&quot; about a pull request having undergone enough review and testing before being merged on the master that it doesn&#x27;t introduce such glaring regressions as cited in the examples. If that cannot be ensured, I&#x27;m setting myself up for one goose chase after the other...

I learned this word, but I&#x27;m scared and wondered by it every time. I think it fits here. There is two worlds: believed and true. And when they merge it&#x27;s called &quot;peripeteia&quot;

It might be true that even though I have a JTB about something I might be wrong.<p>Nevertheless, I think It would be resonable to act upon a JTB as it it was true. For all effects it is true to the best of my knowledge. This does not mean I shun down new information that might make me to change my JTB.<p>And if having a JTB is not knowledge, what is? What can we know? We can always imagine a world where even our most firm JTB might be false. If a JTB is not a good case to use the word knowledge I don&#x27;t know what is

Always bemused and not a little confused that anyone (most notably, Americans) still spend so much time and energy on analytical philosophy [and its Quine-y assertions about semantics] so many decades after its sort of formal semantics collapsed as a useful way of analyzing natural language.<p>Linguistics (not to mention, comp lit or continental philosophy) departments have an order of magnitude more to say about meaning in natural language and have had for... decades and decades.<p>I just don&#x27;t get it.

I don&#x27;t like this approach. If everyone were to approach a problem with this mentality it would conjure doubt in the entire process. Nothing would ever get done. Question: &quot;how do we know if anything exists?&quot; &lt;--(an extreme example). Answer: &quot;well we don&#x27;t but it doesn&#x27;t help us with the realities of the problem at hand.&quot; I think this idea introduces confusion and does more harm than good, in my opinion.

This reminds me of how some studies are proven to be true but the reason they are true are not the reasons the author of the study presents. Instead, the true-ness is either a coincidence or a correlation, not causation. These people go on to write books and entire industries are formed around these hypothetical truths and it takes years to undo the damage of the original study (e.g. marshmallow test, jam experiment)

So just to be clear, a &#x27;gettier&#x27; is when something you Believe and have Justification for turns out to be false?<p><pre><code> Actually True J B 1 1 knowing 1 0 denying 0 1 lucky hunch 0 0 sceptic, default position Actually False J B 1 1 mis-justification: un&#x2F;incorrectly-verified 1 0 lucky denial of mis-justified 0 1 superstition 0 0 sceptic, default position</code></pre>

Note most gettier cases are plays on our intuitions:<p>What feels like a pointer is actually a category. That is, it feels like it points to one, but it points to many. Like both examples given here: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Gettier_problem" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Gettier_problem</a> .

I had a philosophy lecture last year that included a lot of epistemology (Theory of Knowledge). We talked a fair bit about justified true beliefs, but Gettier only came up in a side note - the professor being more interested in skepticism and the responses thereto. Never would have dreamt of applying that lecture to programming, though.

The article was great. I also find philosophy and software correlates occasionally. The response of the author to the gettier cases are expected behavior. In fact it is a blessing. 99% of the time there is a cow, we experienced that before and that’s the reason of our confidence. We easily solve our problems with this approach.

The first example is exactly the reason why I hate rebases and prefer merges and complicated history instead. It may be more complicated, but it doesn&#x27;t swipe problems under the rug.

My own opinion is that knowledge is always relative to a perspective. It is only valid in a context.<p>Nothing is absolute.<p>Example:<p>1=1 is something i know is true cause i know the rules of mathematics. There is no absolute truth to that.

As software abstractions often put up facades in front of similar abstractions, this is bound to happen to us software developers.

Didn&#x27;t Descartes cover this with &quot;I think therefore I am.&quot;? Everything else is varying degrees of speculation.

My reaction Gettier cases as a philosophy minor was that J is the only philosophically interesting part of JTB.

“Justified true belief”? All our knowledge is subjective by definition. We don’t even know whether we’re living in a simulation.<p>Personally I doubt that we’re living in a simulation. But the fact that we could be, demonstrates that we don’t have objective knowledge. No cows needed in the field explain it<p>Philosophy might better be called “the history of flawed thinking”

So just more simply “red herring”

Isn’t this just a confounding factor?

Is not this same as coincidence?

To me this seems unhelpful. I&#x27;d say there is no &quot;knowledge&quot;; there&#x27;s only belief. And if you defined knowledge as &quot;justified true belief&quot; then you couldn&#x27;t apply the definition in practice in the real world because you don&#x27;t know when something is true. But that&#x27;s philosophy for you: fun (for some people) but not useful.