Hi HN,<p>Today I wrote a simple tool to visualize the log probabilities of OpenAI GPT completions.<p>It's designed for debugging and understanding GPT's behavior by showing the probability of each candidates, with which you can know when GPT is hesitating and when it is confident of what it's talking about.<p>Basically it's a remake of the OpenAI Playground but with all the models and all the API parameters available.<p>I would recommend forking the repo and check the code before inputing your credentials if you do not trust the version hosted by others.<p>Thanks for checking it out!
With this tool I find GPT firmly believes the second of three random number between 1 to 10 is 7 during my test. It believes in the law of large numbers more than I do.