Understanding the Limitations of Mathematical Reasoning in Large Language Models

The paper confirms that even the biggest and most advanced models can still be misled by adding irrelevant information that can be easily filtered out by humans.

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra