Arithmetic Without Algorithms

"Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics": an influential mechanistic-interpretability result showing that an LLM's arithmetic circuit is not a learned algorithm but a sparse set of heuristic neurons that each fire on particular operand patterns. By Yonatan Belinkov with Northeastern.