论文标题
Autolr：一种进化的学习率政策方法
AutoLR: An Evolutionary Approach to Learning Rate Policies
论文作者
论文摘要
适当的学习率的选择对于良好的人工神经网络培训和性能至关重要。过去，人们必须依靠经验和反复试验才能找到足够的学习率。目前，存在多种最先进的自动方法，使寻找良好的学习率更容易。尽管这些技术有效，并且在多年来取得了良好的效果，但它们是一般解决方案。这意味着对特定网络拓扑的学习率优化在很大程度上尚未探索。这项工作介绍了Autolr，该框架是使用结构化语法演化来发展特定神经网络架构的学习率调度程序。该系统被用来发展学习率的学习率策略，这些策略已与学习率的常用基线值进行了比较。结果表明，使用某些进化政策进行的训练比已建立的基线更有效，并认为这种方法是改善神经网络性能的可行手段。
The choice of a proper learning rate is paramount for good Artificial Neural Network training and performance. In the past, one had to rely on experience and trial-and-error to find an adequate learning rate. Presently, a plethora of state of the art automatic methods exist that make the search for a good learning rate easier. While these techniques are effective and have yielded good results over the years, they are general solutions. This means the optimization of learning rate for specific network topologies remains largely unexplored. This work presents AutoLR, a framework that evolves Learning Rate Schedulers for a specific Neural Network Architecture using Structured Grammatical Evolution. The system was used to evolve learning rate policies that were compared with a commonly used baseline value for learning rate. Results show that training performed using certain evolved policies is more efficient than the established baseline and suggest that this approach is a viable means of improving a neural network's performance.
