Google releases open Flan-T5 checkpoints
Google instruction-tuned the T5 family on 1,836 tasks and released five checkpoints ranging from 80 million to 11 billion parameters.
What happened
Event details
On October 20, 2022, Google researchers submitted “Scaling Instruction-Finetuned Language Models” and publicly released Flan-T5 checkpoints. The study scaled instruction tuning across 1,836 tasks and included nine chain-of-thought datasets. The open Flan-T5 family contains five sizes: Small at 80 million parameters, Base at 250 million, Large at 780 million, XL at 3 billion, and XXL at 11 billion. The paper reported broad improvements over the corresponding T5 checkpoints across zero-shot, few-shot, and chain-of-thought evaluations.
Assessment
Why it matters
Flan-T5 provided a complete, openly available demonstration of large-scale instruction tuning and had lasting influence on open instruction models, evaluations, and tooling.
Availability
Access notes
Official Small, Base, Large, XL, and XXL checkpoints are publicly downloadable under Apache 2.0.