Efficient training of language models to fill in the middle Read paperAbstractWe show that autoregressive language models can learn to infill text after we appl...
A hazard analysis framework for code synthesis large language models Read paperAbstractCodex, a large language model (LLM) trained on a variety of codebases, exceeds the...
DALL·E 2 pre-training mitigations In order to share the magic of DALL·E 2 with a broad audience, we needed to reduce the risks associa...
Learning to play Minecraft with Video PreTraining We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled vide...
Evolution through large models Read paperAbstractThis paper pursues the insight that large language models (LLMs) trained to genera...
AI-written critiques help humans notice flaws We trained “critique-writing” models to describe flaws in summaries. Human evaluators find flaws in ...
Techniques for training large neural networks Large neural networks are at the core of many recent advances in AI, but training them is a difficul...
Teaching models to express their uncertainty in words Read paperAbstractWe show that a GPT-3 model can learn to express uncertainty about its own answers ...
Hierarchical text-conditional image generation with CLIP latents Read paperAbstractContrastive models like CLIP have been shown to learn robust representations of im...
Measuring Goodhart’s law Goodhart’s law famously says: “When a measure becomes a target, it ceases to be a good measure.” Alt...
