Google first introduces LaMDA
Google introduces LaMDA for open-domain dialogue at I/O 2021; the model remains an internal research system without public weights or a general-purpose API.
GO48 English reports
ORGANIZATION
Google DeepMind
Related English reports
English reports linked to this entity will appear here as their translations are published.
AI-assisted · reviewedImpact 79/100
Google publishes Switch Transformer, scaling a sparse model to 1.6T parameters
Switch Transformer routed each token to a single expert, scaling total sparse parameters to about 1.6 trillion without activating the full model for every token.
GShard paper demonstrates a sparse-expert translation model with more than 600B parameters
Google researchers introduce GShard automatic sharding and use it to train a multilingual translation MoE with more than 600B total parameters on 2,048 TPU v3 accelerators.
Google Research introduces the open-domain conversational model Meena
Google Research introduced the 2.6-billion-parameter open-domain conversational model Meena but did not release an external demo or model weights.
Google publishes T5 and releases models, code, and C4
T5 cast a wide range of natural-language-processing tasks into a text-to-text format and released a model family ranging from 60 million to 11 billion parameters.
XLNet paper and XLNet-Large weights are released
Researchers from Google Brain and Carnegie Mellon University introduce XLNet and release the XLNet-Large checkpoint with TensorFlow code.
Google publishes BERT and releases pretrained models
BERT unified many language-understanding tasks around a bidirectional Transformer encoder, followed by a release of code and pretrained checkpoints.
Google researchers publish the Transformer paper
The paper introduced an encoder–decoder architecture built entirely on attention, replacing recurrent computation with parallelizable self-attention.