Google publishes Switch Transformer, scaling a sparse model to 1.6T parameters
Switch Transformer routed each token to a single expert, scaling total sparse parameters to about 1.6 trillion without activating the full model for every token.
SW1 English report
MODEL
Switch Transformer
- Release date:
- 2021-01-11
- Input/output:
- Text → Text
- Openness:
- Open-source
- Organization:
- Google DeepMind
Official links | Chat | Code Plan | Agent tools | API |
|---|---|---|---|---|
| Global |
Related English reports
English reports linked to this entity will appear here as their translations are published.
AI-assisted · reviewed