OLMo 2

"2 OLMo 2 Furious." Dense Transformers at 7B (4T tokens), 13B (5T), and 32B (6T tokens, 1.5 epochs). 4K context. Introduces Dolmino Mix late-stage curriculum training (specialized data during annealing) and model souping (merging 3 annealing runs). Training FLOPs: 1.3x10^24 for 32B.

First fully open model to outperform GPT-3.5 Turbo and GPT-4o mini (post-trained with SFT + DPO + PPO + RLVR). MMLU: 78.7 (32B base). COLM 2025. Apache 2.0.