Plutôt que de spéculer sur une auto-amélioration générale incontrôlée, cette étude démontre une application ciblée et mesurable de l'IA appliquée à sa propre sécurité.
L'automatisation de la R&D en IA franchit une étape concrète : les tâches complexes de post-entraînement et de réglage des garde-fous ne dépendent plus exclusivement de chercheurs humains.
Conduit par le chercheur Chen Yueh-Han, le projet Automated Alignment Researcher (AAR) analyse la littérature scientifique, propose des méthodes d'alignement et entraîne des modèles par cycles itératifs de 30 minutes. Sur 10 benchmarks d'alignement, l'agent a surpassé les propositions des chercheurs humains en moins de six heures, réduisant le coût de recherche de 150 dollars par heure humaine à 4 dollars en appels API.
Impact concret
Les responsables R&D et ingénieurs en apprentissage automatique peuvent concevoir des pipelines d'alignement automatisés pour tester des hypothèses d'entraînement à moindre coût.
Applications concrètes
Génération et sélection automatisées de techniques de fine-tuning, optimisation itérative des garde-fous de sécurité et évaluation accélérée d'hypothèses de recherche.
Training AI models with other AI models has become a very popular goal for neolabs — and now, a researcher in Anthropic’s fellows program has given us an early look at what it might look like in practice. On Friday, Anthropic published a new paper titled “ Automated Researchers Can Reliably Mitigate Alignment Failures ,” detailing how AI systems could reliably improve a model’s performance on a set of alignment benchmarks. When given 10 benchmarks for specific misaligned behaviors, the automated systems were able to improve performance on every single one without degrading overall performance. Led by Anthropic fellow Chen Yueh-Han, the system replicates much of the traditional approach to research. Each automated system searches the available literature, proposes a method, and trains the model using that method for 30 minutes, gradually increasing the benchmark over several iterations. Effective methods are preserved while ineffective ones are discarded, allowing the system to operate quickly and at a great scale. “Overall, these results provide early evidence that automated alignment post-training could become practical in the near term,” the paper reads. The paper is a step toward recursive self-improvement , which many see as the next significant step in AI progress. If models can improve their own alignment training, it’s plausible they could improve training practices more broadly — at which point, human AI researchers might soon become obsolete. The paper isn’t shy about addressing this idea, explicitly comparing the Automated Alignment Researcher (AAR) to its human equivalent. “The best AAR method beats what experienced humans propose, on average within six hours,” the paper reads. “Human guided research directions do not lead to stronger performance.” There’s even a cost comparison, in case anyone wasn’t convinced. “An AAR costs roughly $4 per hour in API inference against the $150 per hour we pay our human researchers.” In fairness, the paper also points out a few limitations to this approach. The automated system only works insofar as the benchmarks reflect the actual alignment goals, and even then there’s significant work to be done in establishing and maintaining those benchmarks — not to mention maintaining and expanding on the literature the automated researchers are drawn from. Topics AI , Anthropic When you purchase through links in our articles, we may earn a small commission . This doesn’t affect our editorial independence. Russell Brandom AI Editor Russell Brandom has been covering the tech industry since 2012, with a focus on platform policy and emerging technologies. He previously worked at The Verge and Rest of World, and has written for Wired, The Awl and MIT’s Technology Review. He can be reached at russell.brandom@techcrunch.com or on Signal at 412-401-5489. View Bio October 13 – 15 San Francisco Don’t miss out . The startup community will gather to answer a pivotal question: How do you build sustainably in the AI era? REGISTER NOW Most Popular Hugging Face is selling a cute $399 open source duck robot, Microduck Rebecca Bellan Nvidia closes in on Hugging Face acquisition Connie Loizos Viral AI startup Instinct has raised $350M at a $2.5B valuation Lucas Ropek Fitbit founders launch Luffu Link, an LTE health and safety band Aisha Malik India’s Airbound bags $37M to take on trucks with rocket-like drones Jagmeet Singh Hugging Face reportedly in talks to be acquired for $13B Rebecca Bellan Who’s behind the new ‘stealth model’ Ox Alpha? Anthony Ha