# Dynamic Channel Pruning: Feature Boosting and Suppression

**Authors:** Xitong Gao, Yiren Zhao, Łukasz Dudziak, Robert Mullins, Cheng-zhong Xu  
**arXiv:** [1810.05331](https://arxiv.org/abs/1810.05331)  
**Date:** 2018-10-12  
**Summary page:** https://www.summarizepaper.com/en/paper/1810.05331v2/

## Key points

- Authors propose a method called feature boosting and suppression (FBS) to reduce computational and memory resources required for deep CNNs without sacrificing accuracy.
- FBS introduces small auxiliary connections to existing convolutional layers, allowing for dynamic amplification of salient convolutional channels and skipping of unimportant ones at runtime.
- FBS preserves the full network structures and accelerates convolution by dynamically skipping unimportant input and output channels, unlike permanent channel pruning methods.
- Experimental results show that FBS can provide up to 5 times savings in compute on VGG-16 and ResNet-18 models with less than 0.6% top-5 accuracy loss.
- FBS-augmented networks can be trained using conventional stochastic gradient descent, making it applicable to many state-of-the-art CNN architectures.
- This approach has potential implications for cost-sensitive cloud services and low-powered edge computing applications where reducing resource requirements is crucial.

## Summary

In this paper, the authors address the challenge of reducing the computational and memory resources required for deep convolutional neural networks (CNNs) without sacrificing accuracy. They propose a new method called feature boosting and suppression (FBS), which takes advantage of the input-dependent importance of features computed by convolutional layers. The FBS method introduces small auxiliary connections to existing convolutional layers, allowing for dynamic amplification of salient convolutional channels and skipping of unimportant ones at runtime. Unlike channel pruning methods that permanently remove channels, FBS preserves the full network structures and accelerates convolution by dynamically skipping unimportant input and output channels. To evaluate the effectiveness of FBS, the authors compare it to various existing channel pruning and dynamic execution schemes. They demonstrate significant improvements on ImageNet classification tasks, showing that FBS can provide up to 5 times savings in compute on VGG-16 and ResNet-18 models with less than 0.6% top-5 accuracy loss. The authors highlight that FBS-augmented networks can be trained using conventional stochastic gradient descent, making it readily applicable to many state-of-the-art CNN architectures. This approach has potential implications for cost-sensitive cloud services and low-powered edge computing applications where reducing resource requirements is crucial. Overall, this paper presents a novel method for reducing computational and memory costs in deep CNNs while maintaining high accuracy. The experimental results demonstrate the effectiveness of FBS in achieving significant resource savings without compromising performance on challenging image classification tasks.