论文标题
使用代码生成语言模型进行自我编程的人工智能
Self-Programming Artificial Intelligence Using Code-Generating Language Models
论文作者
论文摘要
大规模语言模型的最新进展已在以前难以置信的计算机编程任务中取得了突破。元学习和神经体系结构搜索的先前工作已在各个任务领域取得了巨大的成功，从而产生了无数方法，以优化深度学习模型的设计和学习动态。在这些研究领域的交集中，我们实施了一个代码生成的语言模型，能够修改其自己的源代码。自AI本身黎明以来，自我编程的AI算法就引起了人们的关注。尽管已经提出了广义自我编程AI的各种理论公式，但迄今为止，在现实世界的计算约束下，尚未成功实施此类系统。将基于AI的代码生成应用于AI本身，我们开发和实验验证了自我编程的AI系统的第一个实际实现。我们从经验上表明，使用代码生成模型实施的自编程AI可以成功修改其自己的源代码，以改善性能和程序子模型以执行辅助任务。我们的模型可以自我修改各种属性，包括模型架构，计算能力和学习动态。
Recent progress in large-scale language models has enabled breakthroughs in previously intractable computer programming tasks. Prior work in meta-learning and neural architecture search has led to substantial successes across various task domains, spawning myriad approaches for algorithmically optimizing the design and learning dynamics of deep learning models. At the intersection of these research areas, we implement a code-generating language model with the ability to modify its own source code. Self-programming AI algorithms have been of interest since the dawn of AI itself. Although various theoretical formulations of generalized self-programming AI have been posed, no such system has been successfully implemented to date under real-world computational constraints. Applying AI-based code generation to AI itself, we develop and experimentally validate the first practical implementation of a self-programming AI system. We empirically show that a self-programming AI implemented using a code generation model can successfully modify its own source code to improve performance and program sub-models to perform auxiliary tasks. Our model can self-modify various properties including model architecture, computational capacity, and learning dynamics.
