What separates humans from AI? It's doubt (2021)

&quot;One promising approach, known as “dropout”, runs a problem through a neural network algorithm multiple times, each with slightly different settings. We can then ask to what extent the resulting answers agree -- a proxy for how uncertain the network is about its choice.&quot;<p>Isn&#x27;t this just the good old consensus algorithm? And things get tense when you start to consider a real-world deployment or a distributed system with which some of the nodes trying to reach a consensus might be malicious; cf.<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39086775">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39086775</a>

LLMs that output text have this problem.<p>I use classification models all the time that are probability calibrated, it’s not that hard to do and works most of the time. The thing is you have to do it and most people don’t.