Episode 15 – Inside the Model Specs
Episode 15 – Inside the Model Specs
Author: OpenAI – Duration: 00:37:27
The more AI can do, the more we need to ask ourselves what it should and shouldn't do. In this episode, OpenAI researcher Jason Wolfe joins host Andrew Mayne to talk about the Model Spec, the public framework that defines the model's intended behavior. They explain how the Model Spec works in practice, including how the chain of command handles conflicts between instructions and how OpenAI evolves it based on feedback, real-world usage, and new model capabilities. Learn more about our approach to model specification: https://openai.com/index/our-approach-to-the-model-spec/
Chapters 00:00 Introduction 01:10 What are model specifications? 03:55 How does Model Spec work in practice? 06:26 Transparency: where to read the model specifications and give feedback 07:51 How did the model specification come about? 10:02 How does the specification translate into the behavior of the model? 11:26 What is the hierarchy/chain of command? 13:35 Handle edge cases like Santa 17:41 How does the Model Spec change over time? 19:59 What happens when models disagree with specifications? 22:05 How do the smaller models follow the specifications? 23:16 Is chain of thought useful for alignment? 24:16 Spec Model vs Anthropic Constitution 26:28 What surprised you the most? 26:56 How do you define the scope of the specification? 27:44 What is the future of Model Spec? 31:16 How should developers think about the specification? 34:44 Asimov's laws versus model specifications 37:16 Could AI write a human specification?
