The Transformer paper replaces recurrence with attention
Google researchers introduce an attention-only architecture that trains efficiently in parallel and becomes the foundation of modern large language models.
What happened
Event details
Attention Is All You Need proposed the Transformer, an encoder-decoder architecture built around self-attention instead of recurrent or convolutional sequence processing. The design made training more parallelizable and improved machine-translation results while reducing the path information had to travel through a network. The paper did not invent today's language-model products by itself, but it supplied the scalable architecture on which most of them were later built.
Assessment
Why it matters
The Transformer became the common architectural foundation for the language-model scaling wave that followed.