OpenAI releases a generative pre-trained Transformer, later known as GPT-1
OpenAI releases a two-stage transfer-learning system: pretrain a Transformer with language modeling, then adapt it to several language-understanding tasks with limited task-specific changes.