DeepMind's AlphaCode Achieves Median Human Level in Coding Competitions
DeepMind introduces AlphaCode, an AI system that writes computer programs at a competitive level, ranking within the top 54% of participants on Codeforces. The system uses massive-scale code generation and smart filtering to solve novel programming problems.
DeepMind unveils AlphaCode, an artificial intelligence system that writes computer programs at a competitive level, marking a significant milestone in AI problem-solving. The system achieves an estimated rank within the top 54% of participants on Codeforces, a popular platform that attracts tens of thousands of human programmers. This performance places AlphaCode at about the level of the median competitor, making it the first AI code generation system to reach a competitive threshold in these challenging events.
The system relies on transformer-based language models to generate code at an unprecedented scale before smartly filtering the output down to a small set of promising programs. AlphaCode tackles unforeseen problems that require a sophisticated blend of critical thinking, logic, algorithms, coding, and natural language understanding. Researchers validate these capabilities by evaluating the AI on 10 recent Codeforces contests that are intentionally newer than the system's training data.
DeepMind details this breakthrough in a paper published on the cover of the journal Science. To encourage further innovation in problem solving and code generation, the team releases a comprehensive dataset of competitive programming problems and solutions on GitHub. This newly available dataset includes extensive tests to ensure program correctness, addressing a critical feature that current machine learning datasets lack.