OpenAI opens GPT-3 access through a private API beta
OpenAI launches a private beta for a general-purpose text-in, text-out API running GPT-3 family models, offering controlled access instead of open weights.
EVENT ARCHIVE
Events
Browse published English reports with original sources and content provenance.
AI-assisted · reviewed
GPT-3 demonstrates in-context learning at 175B parameters
OpenAI publishes GPT-3 research showing that a 175B-parameter model can switch tasks from instructions and a few examples in the prompt without updating its weights.
Microsoft releases Turing-NLG as a limited academic demo
Microsoft introduced the 17-billion-parameter Turing-NLG and offered a private demo to a small group of academic users.
Google Research introduces the open-domain conversational model Meena
Google Research introduced the 2.6-billion-parameter open-domain conversational model Meena but did not release an external demo or model weights.
OpenAI releases the full GPT-2 1.5B weights
OpenAI releases code and model weights for the largest GPT-2 configuration, concluding its nine-month staged-release experiment.
Google publishes T5 and releases models, code, and C4
T5 cast a wide range of natural-language-processing tasks into a text-to-text format and released a model family ranging from 60 million to 11 billion parameters.
Facebook AI Research releases RoBERTa
Facebook AI Research retrains BERT with more data, larger batches, longer training, dynamic masking, and no next-sentence prediction, then releases the paper, weights, and fairseq implementation.
XLNet paper and XLNet-Large weights are released
Researchers from Google Brain and Carnegie Mellon University introduce XLNet and release the XLNet-Large checkpoint with TensorFlow code.
OpenAI introduces GPT-2 and begins a staged release
OpenAI introduces GPT-2, a family scaling to 1.5B parameters, but initially releases only a smaller model because of misuse concerns.
Google publishes BERT and releases pretrained models
BERT unified many language-understanding tasks around a bidirectional Transformer encoder, followed by a release of code and pretrained checkpoints.
OpenAI releases a generative pre-trained Transformer, later known as GPT-1
OpenAI releases a two-stage transfer-learning system: pretrain a Transformer with language modeling, then adapt it to several language-understanding tasks with limited task-specific changes.
ELMo introduces deep contextualized word representations
The AllenNLP team introduces ELMo, which derives context-dependent word representations from a bidirectional LSTM language model, and releases an implementation and pretrained weights.
Google researchers publish the Transformer paper
The paper introduced an encoder–decoder architecture built entirely on attention, replacing recurrent computation with parallelizable self-attention.