Google first introduces LaMDA
Google introduces LaMDA for open-domain dialogue at I/O 2021; the model remains an internal research system without public weights or a general-purpose API.
EVENT ARCHIVE
Events
20 English reports are shown on this page. Refine the keywords, date range, company, or event type below.
AI-assisted · reviewedImpact 79/100
EleutherAI releases GPT-Neo
EleutherAI releases an early open alternative to GPT-3.
Naver releases HyperCLOVA
Naver releases a large language model optimized for Korean.
Google publishes Switch Transformer, scaling a sparse model to 1.6T parameters
Switch Transformer routed each token to a single expert, scaling total sparse parameters to about 1.6 trillion without activating the full model for every token.
OpenAI introduces the 12B text-to-image model DALL·E
OpenAI introduces DALL·E, a 12B autoregressive Transformer that models text and discrete image tokens as one sequence.
OpenAI releases the CLIP vision-language model
OpenAI released CLIP, a vision-language model trained with natural-language supervision, together with its paper, code, and pretrained models.
Meta publishes BART research and pretrained models
Meta published the BART denoising sequence-to-sequence model and made code and pretrained weights available through fairseq.
GShard paper demonstrates a sparse-expert translation model with more than 600B parameters
Google researchers introduce GShard automatic sharding and use it to train a multilingual translation MoE with more than 600B total parameters on 2,048 TPU v3 accelerators.
OpenAI opens GPT-3 access through a private API beta
OpenAI launches a private beta for a general-purpose text-in, text-out API running GPT-3 family models, offering controlled access instead of open weights.
GPT-3 demonstrates in-context learning at 175B parameters
OpenAI publishes GPT-3 research showing that a 175B-parameter model can switch tasks from instructions and a few examples in the prompt without updating its weights.
Microsoft releases Turing-NLG as a limited academic demo
Microsoft introduced the 17-billion-parameter Turing-NLG and offered a private demo to a small group of academic users.
Google Research introduces the open-domain conversational model Meena
Google Research introduced the 2.6-billion-parameter open-domain conversational model Meena but did not release an external demo or model weights.
OpenAI releases the full GPT-2 1.5B weights
OpenAI releases code and model weights for the largest GPT-2 configuration, concluding its nine-month staged-release experiment.
Google publishes T5 and releases models, code, and C4
T5 cast a wide range of natural-language-processing tasks into a text-to-text format and released a model family ranging from 60 million to 11 billion parameters.
Facebook AI Research releases RoBERTa
Facebook AI Research retrains BERT with more data, larger batches, longer training, dynamic masking, and no next-sentence prediction, then releases the paper, weights, and fairseq implementation.
XLNet paper and XLNet-Large weights are released
Researchers from Google Brain and Carnegie Mellon University introduce XLNet and release the XLNet-Large checkpoint with TensorFlow code.
OpenAI introduces GPT-2 and begins a staged release
OpenAI introduces GPT-2, a family scaling to 1.5B parameters, but initially releases only a smaller model because of misuse concerns.
Google publishes BERT and releases pretrained models
BERT unified many language-understanding tasks around a bidirectional Transformer encoder, followed by a release of code and pretrained checkpoints.
OpenAI releases a generative pre-trained Transformer, later known as GPT-1
OpenAI releases a two-stage transfer-learning system: pretrain a Transformer with language modeling, then adapt it to several language-understanding tasks with limited task-specific changes.
ELMo introduces deep contextualized word representations
The AllenNLP team introduces ELMo, which derives context-dependent word representations from a bidirectional LSTM language model, and releases an implementation and pretrained weights.