Salesforce Open-Sources CodeT5 to Improve AI Code Generation
Salesforce researchers release CodeT5, an open-source AI system that understands and generates code by incorporating structural programming knowledge instead of relying solely on natural language processing techniques.
Salesforce researchers release an open-source machine learning system called CodeT5 that understands and generates code in real time. Unlike existing AI-powered coding tools that often produce errors, this new system claims to achieve state-of-the-art performance on tasks like code defect detection and clone detection.
Previous AI coding systems typically adapt conventional natural language processing techniques to source code, which ignores the structural information inherent in programming languages. CodeT5 overcomes this limitation by incorporating code-specific knowledge, using both developer-assigned identifiers and accompanying comments to better comprehend code semantics.
The system builds on Google's T5 framework and trains on over 8.35 million instances of code pulled from open-source GitHub repositories across multiple programming languages. The largest version of the model contains 220 million parameters and requires 12 days of training on a cluster of 16 Nvidia A100 GPUs.