SEA-LION v1

model
AI Singapore

The origin of the SEA-LION line and the only generation trained from scratch: 3B and 7B decoder models on the MPT architecture, pretrained by AI Singapore's Products pillar on 980B tokens with a custom 256K-entry SEABPE tokenizer built for eleven Southeast Asian languages (English, Chinese, Indonesian, Malay, Thai, Vietnamese, Filipino, Tamil, Burmese, Khmer, Lao) on 256 A100 40GB GPUs, released under MIT in October 2023 with instruct-tuned versions following in early 2024. Later generations moved to continued pretraining of open foundation models, so v1 remains the reference point for what a from-scratch regional model of that era could reach.

HuggingFace (SEA-LION-v1-7B)HuggingFace (SEA-LION-v1-3B)GitHub

Model Details

Architecture DENSE
Parameters 7B
Training tokens 980B
Training hardware 256x A100 40GB
License MIT

Variants

Name Parameters Notes
SEA-LION-v1-3B 3B —
SEA-LION-v1-7B 7B Instruct-tuned SEA-LION-v1-7B-IT released February 2024.
multilingualopen-weightfoundational

Related