Sapiens AI's first open base checkpoint: a 202B MoE with about 16B active parameters under a custom agnes architecture (48 layers, hidden 4,096, 160 routed plus one shared expert with 6 active, MLA-style low-rank attention, a sparse top-512 indexer, alternating KV compression ratios, hyper-connections, and YaRN from 65K to a 1M-token context), shipped in FP8 under Apache 2.0 on August 29, 2026. The card gives no training tokens, benchmarks, or report, and the companion post-training repository is explicitly a pre-registered proposal rather than results. Provenance is unsettled: the shape matches no tracked checkpoint, but the 129,292-entry vocabulary and the noaux_tc routing, indexer, and ue8m0 scaling stack are DeepSeek-V3/V4 conventions, and the lab's earlier from-scratch claims for Agnes 2.5 Pro did not hold up. Recorded as not verified from scratch until a training record appears.

Model Details

Architecture MOE
Active params 16B
Context window 1,048,576
License Apache-2.0
moeopen-weight

Related