论文标题
深玻尔兹曼机器中的退火和复制对称性
Annealing and replica-symmetry in Deep Boltzmann Machines
论文作者
论文摘要
在本文中，我们研究了多层旋转玻璃模型（人工智能术语中的深玻尔兹曼机器）淬灭压力的性能，该模型在相邻层中的旋转之间允许旋转之间的成对相互作用，而不是在相同层中，而不是在同一层中，在距离内的层之间的旋转相互作用。我们证明了一个定理，它可以根据K Sherrington-Kirkpatrick旋转玻璃杯的这种K层机的压力界定，并使用它来研究其退火区域。确定了淬灭压力的复制对称近似，并考虑其与退火的关系。本文还对与机器学习有关的模型的建筑结构进行了一些观察。由于逃脱了退火区域是有意义的训练，因此通过挤压此类区域，我们获得了对形式因素的热力学约束。值得注意的是，它的最佳逃脱是通过要求最后一层以在网络大小中进行亚线性扩展的。
In this paper we study the properties of the quenched pressure of a multi-layer spin-glass model (a deep Boltzmann Machine in artificial intelligence jargon) whose pairwise interactions are allowed between spins lying in adjacent layers and not inside the same layer nor among layers at distance larger than one. We prove a theorem that bounds the quenched pressure of such a K-layer machine in terms of K Sherrington-Kirkpatrick spin glasses and use it to investigate its annealed region. The replica-symmetric approximation of the quenched pressure is identified and its relation to the annealed one is considered. The paper also presents some observation on the model's architectural structure related to machine learning. Since escaping the annealed region is mandatory for a meaningful training, by squeezing such region we obtain thermodynamical constraints on the form factors. Remarkably, its optimal escape is achieved by requiring the last layer to scale sub-linearly in the network size.
