Facebook AI Research releases RoBERTa
Facebook AI Research retrains BERT with more data, larger batches, longer training, dynamic masking, and no next-sentence prediction, then releases the paper, weights, and fairseq implementation.
RO1 English report
MODEL
RoBERTa
- Release date:
- 2019-07-26
- Parameters:
- 125M (base)/ 355M (large)
- Input/output:
- Text → Text
- Openness:
- Open-source
- Organization:
- Meta
Official links | Chat | Code Plan | Agent tools | API |
|---|---|---|---|---|
| Global | — |
Related English reports
English reports linked to this entity will appear here as their translations are published.
AI-assisted · reviewed