OpenAI Unveils Codex, a 12-Billion Parameter AI for Writing Software Code
OpenAI introduces Codex, a specialized GPT-3 model fine-tuned on public Python code that successfully solves over 70% of programming challenges in a new benchmark test.
OpenAI announces Codex, a 12-billion parameter AI model designed to generate program code directly from natural language descriptions. Built upon the foundation of the GPT-3 language model, this new system powers GitHub's Copilot tool and represents a major leap in automated software development.
To train Codex, OpenAI fine-tunes a pre-trained GPT-3 model using 159 gigabytes of Python code sourced from 54 million public GitHub repositories. The researchers also adapt the text tokenizer to better handle the unique structure of programming languages, specifically introducing new tokens to manage significant whitespace runs in Python.
The system undergoes testing on HumanEval, a newly created open-source benchmark consisting of 164 programming challenges graded by automated unit tests. While standard GPT-3 fails completely at this task, Codex passes 28.7% of the tests on its first try and successfully solves 77.5% of the problems when generating 100 possible solutions per challenge.