论文标题
神经排名模型是否会加剧性别偏见？
Do Neural Ranking Models Intensify Gender Bias?
论文作者
论文摘要
对信息检索（IR）系统中社会偏见的占地面积的担忧已在以前的几项研究中提出。在这项工作中，我们从性别偏见的检索结果的角度研究了各种IR模型。为此，我们首先提供了一个偏差测量框架，其中包括两个指标，以量化给定IR模型的排名列表中与性别相关概念不平衡的程度。为了通过框架检查IR模型，我们创建了由人类注释者选择的非性别查询数据集。然后，将这些查询应用于MS MARCO通道检索集合中，然后测量BM25模型的性别偏见和几种最近的神经排名模型。结果表明，尽管所有模型都对男性，神经模型，尤其是基于上下文化嵌入模型的模型有很大的偏见，从而显着加剧了性别偏见。我们的实验还显示了神经模型的性别偏见在利用转移学习时的性别偏见，即当他们使用（已经偏见）预训练的嵌入时。
Concerns regarding the footprint of societal biases in information retrieval (IR) systems have been raised in several previous studies. In this work, we examine various recent IR models from the perspective of the degree of gender bias in their retrieval results. To this end, we first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. To examine IR models by means of the framework, we create a dataset of non-gendered queries, selected by human annotators. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. The results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias. Our experiments also show an overall increase in the gender bias of neural models when they exploit transfer learning, namely when they use (already biased) pre-trained embeddings.
