OpenAI Unveils Codex, a GPT-3 AI Model Specializing in Code Generation
OpenAI introduces Codex, a 12-billion parameter AI model fine-tuned on public Python code to write software from natural language prompts. The new model successfully solves the majority of problems on a novel benchmark test, a significant leap over standard GPT-3.
OpenAI announces Codex, a 12 billion parameter AI model designed to generate program code directly from natural language descriptions. Built upon the foundation of the GPT-3 language model, Codex undergoes fine-tuning using 159 gigabytes of publicly available Python code sourced from millions of GitHub repositories.
To measure the capabilities of this new system, the OpenAI research team creates HumanEval, an open-source dataset consisting of 164 programming challenges. While the base GPT-3 model fails to solve any of these problems, Codex passes the unit tests for 28.7% of challenges on its first attempt and reaches a 77.5% success rate when generating 100 possible solutions per problem.
The development process reveals unique challenges in processing code compared to standard natural language, prompting researchers to adapt the text tokenizer to better handle Python's structural whitespaces. This specialized AI model already powers GitHub's Copilot tool and represents a major step forward in automated software development.