Motif 3

Motif Technologies' frontier Korean/English foundation model: a 314B-total / 13.2B-active sparse MoE (384 routed experts, top-8, plus 1 shared expert; 53 layers, 2 dense + 51 MoE; 220K vocabulary) with a native 256K context, pretrained from scratch on ~12.5T tokens (knowledge cutoff March 2026; ~70% of the corpus is NVIDIA's public Nemotron pretraining releases, the rest in-house web, STEM, code, synthetic, Korean, legal and financial data). The architecture is fully in-house: Grouped Differential Latent Attention (GDLA) (grouped differential attention over an MLA-style compressed KV latent, with a query-dependent output gate), Expert-Specific PolyNorm activations, modified manifold-constrained hyper-connections (mHC, 4 residual streams), and a 1-layer multi-token-prediction head used for self-speculative decoding. Trained on NVIDIA B200s with the Muon optimizer (QK-Clip), selective MXFP8 expert compute and window-aware context parallelism. Post-training combines general SFT, six RL-trained specialist teachers plus a software-engineering teacher, and Multi-teacher On-Policy Distillation (MOPD) into one unified reasoning model.

Timeline. Motif first surfaced as Motif-3-Beta, an intermediate checkpoint released 14 July 2026 under a non-commercial research-only license (AA lists it separately as "Motif 3 (Beta)"; it scored 44 then 45 on AA Intelligence Index v4.1/v4.1.1, 37 on v4.2, and 32 on v4.3). The final release followed in August 2026: the post-trained Motif-3 and pretrained Motif-3-Base checkpoints under the MIT license (commercial use permitted, no access gate), the "Motif 3: Technical Report" (arXiv 2608.09119, 10 Aug), and an NVFP4-quantized build for Blackwell GPUs. AA lists the final model at AA Intelligence Index v4.3: 34 — the #1 Korean model on the index — with particular strength on agentic tool use (τ³-Banking 35.3, Terminal-Bench 2.1 74.9, ITBench 51.5) and calibrated abstention on AA-Omniscience (71.6 non-hallucination). Motif's torchtitan-based training framework (motif3-training-example) and a vLLM fork with Motif-3 support are open on GitHub. From a lab spun out of Moreh, trained on its GPU-optimization stack.