TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

Microsoft Research (Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan Ash, Akshay Krishnamurthy) asks whether standard pipelines produce the checkpoints best suited for subsequent RL — and answers with a stage-aware fix. Building on evidence that coverage and pass@K predict post-RL performance, TailSFT modifies supervised fine-tuning to filter out already-fit sequences during training, concentrating learning on the under-modeled tail of the data distribution, with the filtering criterion justified through controlled experiments (including a graph-navigation warm-up) and theoretical analysis.

On OLMo-3 7B, TailSFT improves pass@16 on math and coding evaluations by up to 17% absolute at minimal computational overhead, and the higher-coverage checkpoints consistently translate into up to 4% absolute pass@1 gains in subsequent GRPO runs — better RL initializations, not just better SFT scores. A lightweight coverage-ratio diagnostic identifies when TailSFT is likely to help. The broader thesis: judge intermediate checkpoints by how effectively they support the next training stage — the RL-readiness axis of the RL-scaling family.