Meta releases the Code Llama model family
Meta releases Code Llama 7B, 13B, and 34B checkpoints based on Llama 2, with base, Python, instruction-tuned, and selected fill-in-the-middle variants.
What happened
Event details
On August 24, 2023, Meta released Code Llama 7B, 13B, and 34B checkpoints, inference code, and training recipes. The family was built on Llama 2 for code generation, completion, and debugging. The Python variants received additional Python-code training, while the Instruct variants were aligned for natural-language instructions. Selected 7B and 13B models supported fill-in-the-middle completion. Meta reported HumanEval and MBPP scores of 53.7% and 56.2% for the initial 34B model; these were vendor evaluations. The models were trained on 16,000-token sequences, and Meta reported improvements on inputs extending to 100,000 tokens. The weights could be used for research and some commercial purposes, but users had to accept the Llama 2 Community License and Acceptable Use Policy. That makes the release open-weight rather than equivalent to software under a standard open-source license.
Assessment
Why it matters
Code Llama gave developers a locally deployable and fine-tunable family spanning code completion, Python specialization, and instruction following. It helped expand the open-weight coding-assistant ecosystem, although the launch benchmarks came mainly from Meta and the custom community license was more restrictive than a standard open-source license.
Availability
Access notes
Initial weights and inference code were available after accepting the Llama 2 Community License and Acceptable Use Policy.