Code Llama
model"Code Llama: Open Foundation Models for Code." Meta's first major code-specialized open-weight family, built on Llama 2 with continued pretraining on a code-heavy corpus and long-context fine-tuning to 100K tokens. Available at 7B, 13B, 34B, and 70B sizes, with base, Python-specialized, and Instruct variants.
Foundational for the open-coding-model lineage. Adopted as the base for many downstream code models (DeepSeek-Coder ancestry, Phind-CodeLlama, the Ziya-Coding-34B fine-tune, etc.) and one of the first open models to credibly approach GPT-4 on HumanEval at the 34B scale.
Model Details
Architecture DENSE
Parameters 70B
Base model llama-2
Variants
| Name | Parameters | Notes |
|---|---|---|
| Code Llama 7B | 7B | — |
| Code Llama 13B | 13B | — |
| Code Llama 34B | 34B | — |
| Code Llama 70B | 70B | — |