OpenAI introduces GPT-2 and begins a staged release
OpenAI introduces GPT-2, a family scaling to 1.5B parameters, but initially releases only a smaller model because of misuse concerns.
What happened
Event details
OpenAI introduced GPT-2, a decoder-only autoregressive Transformer with a largest configuration of 1.5B parameters, trained on about 40GB of WebText from eight million web pages. With next-token prediction as its only objective, it showed some zero-shot transfer in question answering, reading comprehension, summarization and translation, although OpenAI noted that many downstream results still lagged specialized systems. Citing concerns about deceptive, biased and abusive uses, OpenAI initially released only a smaller model and sampling code, withholding the 1.5B weights, training code and full training dataset. The decision made responsible publication a public issue. Later work found bias, fine-tuning misuse and limits to content detection, but no strong evidence of real-world misuse at the time; OpenAI completed the staged release in November 2019.
Assessment
Why it matters
GPT-2 advanced zero-shot language-model capabilities while turning staged model release and misuse evaluation into a prominent governance case.
Availability
Access notes
The initial release included a smaller model and sampling code; the 1.5B weights followed nine months later.