OpenAI Teaches o1 and o3 Models to Deliberate on Safety Policies
OpenAI introduces a "deliberative alignment" technique that trains its new o1 and o3 reasoning models to actively consider safety guidelines before responding. This method reportedly reduces unsafe outputs while improving the handling of harmless prompts.
OpenAI unveils a new safety paradigm called "deliberative alignment" for its o1 and o3 reasoning models. Instead of relying solely on standard training methods, the company instructs these models to actively "think" about OpenAI's safety policy during the inference phase. This approach takes place after a user submits a prompt, forcing the AI to consider safety guidelines before generating a final response.
The technique works by weaving the actual text of the safety policy into the model's chain-of-thought process. As the AI breaks down a user's query into smaller steps and re-prompts itself, it simultaneously evaluates the safety implications of its planned output. OpenAI reports that this internal deliberation significantly decreases the rate of unsafe answers while preserving or even enhancing the model's ability to handle benign questions.
Despite the human-like terminology used to describe this process, these models are still fundamentally predicting the next token in a sequence rather than possessing true consciousness. Furthermore, the implementation of such safety measures remains a subject of intense debate in the tech industry. Critics like Elon Musk and Marc Andreessen argue that these subjective safety boundaries often function as a form of censorship, highlighting the ongoing tension between AI safety and open expression.