论文标题
仔细观察多exit体系结构的分支分类器
A Closer Look at Branch Classifiers of Multi-exit Architectures
论文作者
论文摘要
多EXIT体系结构由骨干和分支分类器组成，可提供缩短推理途径以减少深神经网络的运行时间。在本文中，我们分析了不同分支模式在分支分类器的计算复杂性分配方面有所不同。恒定复杂性分支使所有分支保持不变，同时复杂性增强和复杂性降低分支的位置分别在骨干链中分别更复杂的分支。通过对多个骨干和数据集进行的大量实验，我们发现复杂性削弱分支比恒定复杂性或复杂性增强分支更有效，这可以实现最佳的准确性成本折衷。我们通过使用知识一致性来研究原因，以探测将分支添加到主链上的效果。我们的发现表明，复杂性降低的分支对骨干的特征抽象层次结构产生最小的破坏，这解释了分支模式的有效性。
Multi-exit architectures consist of a backbone and branch classifiers that offer shortened inference pathways to reduce the run-time of deep neural networks. In this paper, we analyze different branching patterns that vary in their allocation of computational complexity for the branch classifiers. Constant-complexity branching keeps all branches the same, while complexity-increasing and complexity-decreasing branching place more complex branches later or earlier in the backbone respectively. Through extensive experimentation on multiple backbones and datasets, we find that complexity-decreasing branches are more effective than constant-complexity or complexity-increasing branches, which achieve the best accuracy-cost trade-off. We investigate a cause by using knowledge consistency to probe the effect of adding branches onto a backbone. Our findings show that complexity-decreasing branching yields the least disruption to the feature abstraction hierarchy of the backbone, which explains the effectiveness of the branching patterns.
