OpenAI Unveils Codex, a 12 Billion Parameter AI for Generating Code
OpenAI introduces Codex, a specialized version of GPT-3 trained on public Python code to write software from plain English. The new model successfully solves the vast majority of problems on a novel benchmark test.
OpenAI announces Codex, a 12 billion parameter artificial intelligence model that generates program code from natural language descriptions. Built upon the foundation of the GPT-3 language model, Codex powers GitHub's Copilot tool and represents a major leap in automated software development.
To create Codex, OpenAI fine-tunes a pre-trained GPT-3 model using 159 gigabytes of Python code collected from 54 million public GitHub repositories. The researchers also adapt the text tokenizer to better handle the unique word distribution and whitespace requirements found in programming languages.
The system undergoes testing on HumanEval, a new open-source benchmark containing 164 programming challenges verified by unit tests. While generating a single solution yields a 28.7% success rate, allowing the model to produce 100 attempts per problem results in at least one correct answer for 77.5% of the challenges, a massive improvement over standard GPT-3.