MIRI Highlights AI Safety Research and Growing Need for Infosec Professionals
The Machine Intelligence Research Institute releases a core paper on learned optimization and receives a massive Ethereum donation. Industry leaders also warn of a pressing need for information security experts in AI safety.
The Machine Intelligence Research Institute shares Hubinger et al.'s new paper on the risks of learned optimization in advanced machine learning systems, which serves as a core resource for understanding the AI alignment problem. The organization also receives a major Ethereum donation worth $230,910 from Vitalik Buterin, making him their third-largest all-time supporter.
Experts from the Open Philanthropy Project highlight a critical shortage of security professionals in AI safety and biosecurity. They predict that dozens of global catastrophic risk-focused information security roles will emerge within the next decade, and they urge individuals pursuing high-impact careers to gain relevant infosec expertise to fill these urgent positions.
Additional updates include Rohin Shah's analysis of mesa optimization, Stuart Armstrong's new research agenda on synthesizing human preferences, and a collaborative effort to stop the premature release of a GPT-2 replication. This GPT-2 situation sparks important conversations about ML publishing norms, reinforcing the idea that researchers must default to caution as technology becomes more complex and powerful.