DeepMind AlphaCode Achieves Median Human Level in Coding Competitions
DeepMind's AlphaCode system writes computer programs at a competitive level, ranking in the top 54% of participants on Codeforces. This milestone marks the first time an artificial intelligence achieves competitive performance in complex programming contests.
DeepMind introduces AlphaCode, an artificial intelligence system that writes computer programs at a competitive level. By solving entirely new problems, the system achieves an estimated rank within the top 54% of participants in programming competitions. This accomplishment represents a major milestone, as previous AI advances remain limited to simple math or copying existing solutions.
The system utilizes transformer-based language models to generate code at an unprecedented scale and then smartly filters the output down to a small set of promising programs. DeepMind validates this performance using 10 recent contests hosted on Codeforces, a popular platform that attracts tens of thousands of human programmers. AlphaCode performs at about the level of the median competitor, making it the first AI code generation system to reach a competitive tier in these events.
To encourage further innovation in problem solving and code generation, DeepMind releases a comprehensive dataset of competitive programming problems and solutions on GitHub. This newly published dataset includes extensive tests to ensure program correctness, which is a critical feature that current datasets lack. The underlying research detailing these advancements now appears on the cover of the journal Science.