论文标题
半监督学习的学习
Semi-Supervised Learning with Meta-Gradient
论文作者
论文摘要
在这项工作中，我们提出了一种简单而有效的元学习算法，中的半监督学习。我们注意到，大多数现有的基于一致性的方法都具有过度拟合和有限的模型概括能力，尤其是在仅使用少量标记数据进行培训时。为了减轻此问题，我们通过利用标签信息并以元学习方式优化了问题，提出了一个学习对待正规化术语。具体而言，我们寻求未标记数据的伪标签，以便该模型可以很好地概括为标记的数据，该数据被称为嵌套优化问题。我们使用伪标签和正则化项之间桥接的元梯度解决了这个问题。此外，我们引入了一个简单的一阶近似，以避免计算高阶导数并提供理论收敛分析。对SVHN，CIFAR和Imagenet数据集的广泛评估表明，所提出的算法对最新方法的表现良好。
In this work, we propose a simple yet effective meta-learning algorithm in semi-supervised learning. We notice that most existing consistency-based approaches suffer from overfitting and limited model generalization ability, especially when training with only a small number of labeled data. To alleviate this issue, we propose a learn-to-generalize regularization term by utilizing the label information and optimize the problem in a meta-learning fashion. Specifically, we seek the pseudo labels of the unlabeled data so that the model can generalize well on the labeled data, which is formulated as a nested optimization problem. We address this problem using the meta-gradient that bridges between the pseudo label and the regularization term. In addition, we introduce a simple first-order approximation to avoid computing higher-order derivatives and provide theoretic convergence analysis. Extensive evaluations on the SVHN, CIFAR, and ImageNet datasets demonstrate that the proposed algorithm performs favorably against state-of-the-art methods.
