KoGPT

Kakao Brain's first LLM. 6.2B parameters, 28 layers, 4096 hidden dim, 16 attention heads, RoPE. Trained on 200B tokens of Korean text. 2048 context. One of the earliest large Korean language models.