While this is a neat demo, I would in general be cautious about translating GPT-4's logits to probabilities. We know from the technical report that its confidences are not well calibrated. See Figure 8: <a href="https://arxiv.org/pdf/2303.08774.pdf" rel="nofollow">https://arxiv.org/pdf/2303.08774.pdf</a>