Meta researchers publish multi-token prediction
Meta FAIR researchers propose training language models to predict several future tokens at once, work that later informs DeepSeek-V3's MTP design.
What happened
Event details
The paper by Gloeckle and colleagues adds multiple prediction heads to a shared model trunk so training can predict several future tokens simultaneously. The authors reported gains in sample efficiency and code performance, plus a path to self-speculative decoding. DeepSeek-V3 later cited this research while implementing a sequential variant that preserves the causal chain between predicted tokens. The connection is best understood as open research being adapted, not copied unchanged.
Assessment
Why it matters
The roughly eight-month path from paper to V3 illustrates how quickly public research can be absorbed into frontier model engineering.