Synthetic Persona Pretraining: Alignment from Token Zero

From Robert West's dlab at EPFL (co-first authors Julian Minder, Viktor Moskvoretskii and Raghav Singhal) with Toronto, Northeastern, SJTU, Saarland, Ontocord and others, under the Model Raising program. Synthetic Persona Pretraining (SPP) installs the assistant persona during pretraining instead of as a post-training overlay: (1) annotate ~10% of pretraining documents — every document the SafeLM classifier scores harmful, plus an equal-token benign sample — with first-person, constitution-grounded reflections generated by Qwen3.5-35B-A3B (~53 tokens each, +0.55% of the token mix); (2) pretrain with standard cross-entropy on documents and reflections; (3) post-train on SP-SFT, 300K persona-expressing dialogues, which binds that persona to the assistant identity (persona binding).

Data-matched runs at 1.7B/100B tokens and 3B/500B tokens (a Dolma 3 subsample) compare Vanilla, Filtered, SPP from token zero (T0), SPP at midtraining only (MT) and both. SPP improves constitution following, jailbreak robustness and behavior on out-of-distribution AIRiskDilemmas while preserving capabilities and avoiding over-refusal. Early intervention matters: T0 models beat MT on ConstitutionEval-Hard and AI Risk, shift value priorities toward Truthfulness and Justice, and the T0-over-MT gap grows with scale (AI Risk ≈4 → 19 points, ConstitutionEval-Hard ≈7 → 14 points from 1.7B to 3B); midtraining alone suffices for jailbreak robustness. All 20 base/instruct checkpoints, the 50M-reflection corpus, SP-SFT and the ConstitutionEval / AIRiskDilemmas evals are released on HuggingFace (dlab-spp); code under MIT.