Google publishes T5 and releases models, code, and C4
T5 cast a wide range of natural-language-processing tasks into a text-to-text format and released a model family ranging from 60 million to 11 billion parameters.
What happened
Event details
On October 23, 2019, Google researchers submitted the T5 paper, representing translation, summarization, question answering, and classification as text input followed by text output. The study systematically compared architectures, pretraining objectives, datasets, and transfer methods. The team trained five sizes on C4: Small, Base, Large, 3B, and 11B. The paper and official materials made the code, pretrained checkpoints, and C4 dataset public. Eleven billion parameters describes the largest original variant, not the entire T5 family.
Assessment
Why it matters
T5 provided a unified text-to-text interface for many NLP tasks and released C4 and multiple pretrained checkpoints, influencing transfer-learning research and open tooling.
Availability
Access notes
The paper, pretrained checkpoints, code, and C4 dataset are publicly available.