论文标题
在协变量偏移下无监督的校准
Unsupervised Calibration under Covariate Shift
论文作者
论文摘要
如果概率模型的预测概率与相应的经验频率相匹配，则可以校准概率模型。校准对于安全至关重要的应用中的不确定性量化和决策很重要。尽管对分类器的校准进行了广泛的研究，但我们发现校准是脆弱的，并且在最小的协变量偏移下很容易丢失。现有技术（包括域适应性）主要集中在预测准确性上，并且不能保证在理论上也不保证校准。在这项工作中，我们正式介绍了域转移下的校准问题，并提出了一种基于重要性抽样的方法来解决它。我们评估和讨论方法在现实世界数据集和合成数据集上的功效。
A probabilistic model is said to be calibrated if its predicted probabilities match the corresponding empirical frequencies. Calibration is important for uncertainty quantification and decision making in safety-critical applications. While calibration of classifiers has been widely studied, we find that calibration is brittle and can be easily lost under minimal covariate shifts. Existing techniques, including domain adaptation ones, primarily focus on prediction accuracy and do not guarantee calibration neither in theory nor in practice. In this work, we formally introduce the problem of calibration under domain shift, and propose an importance sampling based approach to address it. We evaluate and discuss the efficacy of our method on both real-world datasets and synthetic datasets.
