论文标题
贝叶斯神经网络大规模：绩效分析和修剪研究
Bayesian Neural Networks at Scale: A Performance Analysis and Pruning Study
论文作者
论文摘要
贝叶斯神经网络（BNN）是获得神经网络预测的统计不确定性的一种有前途的方法，但具有较高的计算开销，可以限制其实际用途。这项工作探讨了高性能计算与分布式培训的使用，以应对BNNS的挑战。我们介绍了Cray-XC40群集上训练VGG-16和RESNET-18型号的性能和可伸缩性比较。我们证明，网络修剪可以加快推理而不会准确丢失，并提供开源软件包，{\ it {bprune}}以自动化此修剪。对于某些模型，我们发现，将多达80 \％的网络修剪仅导致精度损失7.0 \％。随着新硬件加速器的发展，BNN对基准测试性能引起了极大的兴趣。与常规神经网络相比，对BNN训练的分析概述了局限性和益处。
Bayesian neural Networks (BNNs) are a promising method of obtaining statistical uncertainties for neural network predictions but with a higher computational overhead which can limit their practical usage. This work explores the use of high performance computing with distributed training to address the challenges of training BNNs at scale. We present a performance and scalability comparison of training the VGG-16 and Resnet-18 models on a Cray-XC40 cluster. We demonstrate that network pruning can speed up inference without accuracy loss and provide an open source software package, {\it{BPrune}} to automate this pruning. For certain models we find that pruning up to 80\% of the network results in only a 7.0\% loss in accuracy. With the development of new hardware accelerators for Deep Learning, BNNs are of considerable interest for benchmarking performance. This analysis of training a BNN at scale outlines the limitations and benefits compared to a conventional neural network.
