论文标题
RELU网络的限制等轴测：通过标准浓度的概括
The Restricted Isometry of ReLU Networks: Generalization through Norm Concentration
论文作者
论文摘要
虽然回归任务旨在插值整个输入空间的关系，但通常必须使用有限的培训数据来解决它们。不过，如果可以用数据对假设函数进行很好的绘制，那么人们可以希望识别概括模型。 在这项工作中，我们介绍了神经限制的等轴测属性（NEURIP）一个均匀的浓度事件，其中所有浅$ \ mathrm {relu} $网络都以相同的质量勾勒出。为了得出实现神经的样本复杂性，我们绑定了高斯次级指标中网络的覆盖数，并应用链接技术。在神经事件的情况下，我们随后提供了预期风险的界限，这对于任何一系列经验风险的网络都持有。我们得出的结论是，所有具有足够小的经验风险的网络都统一概括。
While regression tasks aim at interpolating a relation on the entire input space, they often have to be solved with a limited amount of training data. Still, if the hypothesis functions can be sketched well with the data, one can hope for identifying a generalizing model. In this work, we introduce with the Neural Restricted Isometry Property (NeuRIP) a uniform concentration event, in which all shallow $\mathrm{ReLU}$ networks are sketched with the same quality. To derive the sample complexity for achieving NeuRIP, we bound the covering numbers of the networks in the Sub-Gaussian metric and apply chaining techniques. In case of the NeuRIP event, we then provide bounds on the expected risk, which hold for networks in any sublevel set of the empirical risk. We conclude that all networks with sufficiently small empirical risk generalize uniformly.
