Meta publishes BART research and pretrained models
Meta published the BART denoising sequence-to-sequence model and made code and pretrained weights available through fairseq.
EVENT ARCHIVE
Events
15 English reports are shown on this page. Refine the keywords, date range, company, or event type below.
AI-assisted · reviewed
GShard paper demonstrates a sparse-expert translation model with more than 600B parameters
Google researchers introduce GShard automatic sharding and use it to train a multilingual translation MoE with more than 600B total parameters on 2,048 TPU v3 accelerators.
OpenAI opens GPT-3 access through a private API beta
OpenAI launches a private beta for a general-purpose text-in, text-out API running GPT-3 family models, offering controlled access instead of open weights.
GPT-3 demonstrates in-context learning at 175B parameters
OpenAI publishes GPT-3 research showing that a 175B-parameter model can switch tasks from instructions and a few examples in the prompt without updating its weights.
Microsoft releases Turing-NLG as a limited academic demo
Microsoft introduced the 17-billion-parameter Turing-NLG and offered a private demo to a small group of academic users.
Google Research introduces the open-domain conversational model Meena
Google Research introduced the 2.6-billion-parameter open-domain conversational model Meena but did not release an external demo or model weights.
OpenAI releases the full GPT-2 1.5B weights
OpenAI releases code and model weights for the largest GPT-2 configuration, concluding its nine-month staged-release experiment.
Google publishes T5 and releases models, code, and C4
T5 cast a wide range of natural-language-processing tasks into a text-to-text format and released a model family ranging from 60 million to 11 billion parameters.
Facebook AI Research releases RoBERTa
Facebook AI Research retrains BERT with more data, larger batches, longer training, dynamic masking, and no next-sentence prediction, then releases the paper, weights, and fairseq implementation.
XLNet paper and XLNet-Large weights are released
Researchers from Google Brain and Carnegie Mellon University introduce XLNet and release the XLNet-Large checkpoint with TensorFlow code.
OpenAI introduces GPT-2 and begins a staged release
OpenAI introduces GPT-2, a family scaling to 1.5B parameters, but initially releases only a smaller model because of misuse concerns.
Google publishes BERT and releases pretrained models
BERT unified many language-understanding tasks around a bidirectional Transformer encoder, followed by a release of code and pretrained checkpoints.
OpenAI releases a generative pre-trained Transformer, later known as GPT-1
OpenAI releases a two-stage transfer-learning system: pretrain a Transformer with language modeling, then adapt it to several language-understanding tasks with limited task-specific changes.
ELMo introduces deep contextualized word representations
The AllenNLP team introduces ELMo, which derives context-dependent word representations from a bidirectional LSTM language model, and releases an implementation and pretrained weights.
Google researchers publish the Transformer paper
The paper introduced an encoder–decoder architecture built entirely on attention, replacing recurrent computation with parallelizable self-attention.