论文标题
用随机软磁技巧的梯度估计
Gradient Estimation with Stochastic Softmax Tricks
论文作者
论文摘要
Gumbel-Max技巧是许多放松的梯度估计器的基础。这些估计器易于实现和较低的差异，但是将它们全面扩展到大型组合分布的目的仍然是出色的。在扰动模型框架内工作，我们引入了随机软磁技巧，将Gumbel-Softmax Trick推广到组合空间。我们的框架是对扰动模型现有放松估计量的统一观点，它包含许多新颖的放松。我们设计了用于子集选择，跨越树木，树木等的结构性放松。与结构较低的基线相比，我们发现随机软智能技巧可用于训练更好的可变变量模型并发现更多的潜在结构。
The Gumbel-Max trick is the basis of many relaxed gradient estimators. These estimators are easy to implement and low variance, but the goal of scaling them comprehensively to large combinatorial distributions is still outstanding. Working within the perturbation model framework, we introduce stochastic softmax tricks, which generalize the Gumbel-Softmax trick to combinatorial spaces. Our framework is a unified perspective on existing relaxed estimators for perturbation models, and it contains many novel relaxations. We design structured relaxations for subset selection, spanning trees, arborescences, and others. When compared to less structured baselines, we find that stochastic softmax tricks can be used to train latent variable models that perform better and discover more latent structure.
