OlmPool: Cracks in the Foundation

Studies how seemingly minor architectural choices impact long-context extension in language models. Provides a pool of 7–8B parameter models trained to 150B tokens (140B pretraining + 10B context extension) with varying architectural configurations, enabling controlled experiments on which design decisions matter for extending context windows.

Part of Ai2's tradition of fully open research infrastructure — all model checkpoints released in both OLMo-core and HuggingFace formats. Apache 2.0.