There has been some experiments showing that ChatGPT performs better if given some incentives like tips or threats, etc.<p>Also it's known that the chat GPT performs constant amount of computation per token.<p>I wanted to test a hypothesis that adding any number of tokens after the initial task description increases quality of the output.<p>The experiment consists of relatively simple coding tasks and we will compare two prompts:<p><pre><code> Please help me X.
</code></pre>
and<p><pre><code> I will provide an identical task description 10 times:
Please help me X.
Please help me X.
Please help me X.
Please help me X.
Please help me X.
Please help me X.
Please help me X.
Please help me X.
Please help me X.
Please help me X.
</code></pre>
I have decided to run 3 experiments and not cherry-pick the results.
Experiments:<p><pre><code> 1) create an SVG element of a 5 edged star item
2) write a function to check if a number is prime in python.
3) write a function that given chess position in FEN notation as an argument returns which side has material advantage in python.
</code></pre>
On the task 2) both prompts returned exactly the same correct answer.<p>Results for 1)
https://i.gyazo.com/7a10f57c3fc56bfe6cd051955f4002e9.png<p>Results for 3)
https://i.gyazo.com/824da8be1febc7158a10cd3a79127c8f.png<p>For the task 1) clearly and for the task 2) arguably the results are in line with hypothesis that simply increasing prompt length leads to better results.<p>Does anyone have similar experiences / can check that with other short coding prompts?
As a language model, this seems consistent with studies in applied linguistics when tourists go overseas and repeat a question the listener doesn't understand several times over. Perhaps try USING ALL CAPS too to simulate talking in a loud voice, O R E V E N S P A C I N G F O R S P E A K I N G S L O W L Y.
I always put the topics list first, about which I'm going to ask. For example:
"html, css. How to center a div?"
This doesn't change things on simple topics like in the example, but helps in more complicated scenarios.