RIBOSPAN

A long-context RNA foundation model from SII's GAIR lab: 1.61B parameters, bidirectional, natively pretrained at context lengths up to 10,240 nucleotides so that full-length mRNAs can be modeled at single-nucleotide resolution with their 5' UTR, coding sequence, and 3' UTR together. Earlier dense RNA encoders were pretrained at about 1K tokens and had to truncate long transcripts. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing. The paper posted August 24, 2026 and the weights on September 1.