Why LLMs Fail at Even Simple Math (And How to Fix It)

All Notes

Technology

A whiteboard full of scribbled math equations.

The Calculation Paradox

Large Language Models (LLMs) are the most complex reasoning engines ever built by humanity. They can debate Hegelian philosophy, debug cryptic Rust compiler errors, and write sonnets in the style of Kanye West. Yet, ask a frontier model to multiply two prime 5-digit numbers, and it will often confidently present you with a hallucinated disaster.

Why can an AI rewrite the Iliad but fail a 4th-grade math test? The answer lies at the intersection of architecture and probability.

1. The Tokenization Trap: Seeing Numbers as Text

The most fundamental reason LLMs fail at math is that they don’t actually see “numbers”—they see Tokens.

The Problem:

When you type 123,456, the model’s tokenizer might split it into arbitrary chunks like 12, 3,4, and 56. Unlike a human (who sees individual digits) or a calculator (which sees binary integers), the LLM sees a sequence of symbols.

Because the model is trying to predict the next token based on probability, it isn’t “calculating” 123 x 456. It is essentially asking its internal neural network: “Statistically speaking, what text usually follows the pattern 123 multiplied by 456?” For common numbers (like 12 x 12), the model just “remembers” the answer. For rare numbers, it takes a guess.

2. The Lacking “Scratchpad” (Working Memory)

When you solve a multi-step math problem, you use a piece of paper. You perform one step, store the intermediate result, and then proceed.

Standard LLMs are Autoregressive Feed-forward systems. Every time they generate a word, they are looking at the entire previous context and performing a massive parallel calculation to find the next most likely word. They don’t have a “private” place to do work. They have to output their thoughts into the chat box immediately.

3. Fixed Accuracy vs. Creeping Errors

In long-form math, one tiny error at the start destroys the final answer. Because LLMs are probabilistic, there is always a ~1% chance they will pick a slightly-less-ideal token. In a 50-step math problem, that 1% error rate compounds. By the time the model reaches the answer, it is mathematically identical to a “broken telephone” game.

4. How We are Fixing It in 2025

The AI community has developed three major “crutches” to turn LLMs into math geniuses:

A. Chain-of-Thought (CoT) Prompting

By telling the model to “Think step by step,” we force it to use its own output as a “Scratchpad.” By writing out “9 x 7 = 63, carry the 6,” it essentially creates a persistent memory state that it can refer to in the next step.

B. Tool-Use (Code Iteration)

The current gold standard. Instead of asking the AI to be a calculator, we give it an actual Python REPL.

Prompt: Calculate the square root of 9821.
AI Action: Generates math.sqrt(9821).
Result: Runs the code and reports the perfect result back to the user.

C. o1-style “Reasoning” Models

Models like OpenAI’s o1 (Strawberry) use “Reinforcement Learning through Search.” They generate thousands of internal “thoughts” and verify them against logical rules before showing you the final answer. This has effectively solved math for Frontier models.

Feature	Base GPT-4	GPT-4 with Tool-use	o1-Reasoning
Raw Math (AIME)	~12%	~50%	~83%+
Speed	Fast	Moderate	Slow (Thinking…)
Logic Type	Pattern Match	Outsourced Calculation	Internal Verification

Conclusion

LLMs were designed to speak, not to count. Their failure at math is not a sign of “stupidity,” but a mismatch between a probabilistic architecture and a deterministic task. By adding tools and external reasoning loops, we aren’t just making AI better at math—we are teaching it how to check its own work.