AI summary · arXiv: 1810.05331v2 · cs.CV

Dynamic Channel Pruning: Feature Boosting and Suppression

Xitong Gao, Yiren Zhao, Łukasz Dudziak, Robert Mullins, Cheng-zhong Xu
Original paper Submitted to arXiv on 12 Oct. 2018 14 pages, 5 figures, 4 tables, published as a conference paper at ICLR 2019 License: CC BY-NC-SA 4.0

How to read this summary? Illustrated guide

The essentials

  • Authors propose a method called feature boosting and suppression (FBS) to reduce computational and memory resources required for deep CNNs without sacrificing accuracy.
  • FBS introduces small auxiliary connections to existing convolutional layers, allowing for dynamic amplification of salient convolutional channels and skipping of unimportant ones at runtime.
  • FBS preserves the full network structures and accelerates convolution by dynamically skipping unimportant input and output channels, unlike permanent channel pruning methods.
  • Experimental results show that FBS can provide up to 5 times savings in compute on VGG-16 and ResNet-18 models with less than 0.6% top-5 accuracy loss.
  • FBS-augmented networks can be trained using conventional stochastic gradient descent, making it applicable to many state-of-the-art CNN architectures.
  • This approach has potential implications for cost-sensitive cloud services and low-powered edge computing applications where reducing resource requirements is crucial.

Key points written by AI: they may contain errors, check them against the paper.

AI-generated keywords: Feature Boosting Suppression Convolutional Layers Image Classification Resource Savings

Read further

  • Comprehensive Summary
  • Layman's Summary
  • Blog article
  • Original abstract
In this paper, the authors address the challenge of reducing the computational and memory resources required for deep convolutional neural networks (CNNs) without sacrificing accuracy. They propose a new method called feature boosting and suppression (FBS), which takes advantage of the input-dependent importance of features computed by convolutional layers. The FBS method introduces small auxiliary connections to existing convolutional layers, allowing for dynamic amplification of salient convolutional channels and skipping of unimportant ones at runtime. Unlike channel pruning methods that permanently remove channels, FBS preserves the full network structures and accelerates convolution by dynamically skipping unimportant input and output channels. To evaluate the effectiveness of FBS, the authors compare it to various existing channel pruning and dynamic execution schemes. They demonstrate significant improvements on ImageNet classification tasks, showing that FBS can provide up to 5 times savings in compute on VGG-16 and ResNet-18 models with less than 0.6% top-5 accuracy loss. The authors highlight that FBS-augmented networks can be trained using conventional stochastic gradient descent, making it readily applicable to many state-of-the-art CNN architectures. This approach has potential implications for cost-sensitive cloud services and low-powered edge computing applications where reducing resource requirements is crucial. Overall, this paper presents a novel method for reducing computational and memory costs in deep CNNs while maintaining high accuracy. The experimental results demonstrate the effectiveness of FBS in achieving significant resource savings without compromising performance on challenging image classification tasks.

Was this synthesis useful?

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.

AI assistant instructions?

Tools for this paper

Explain it differently 1 free rewrite left today
Save to a collection Group papers in collections to compare them, chat with them and export them. Continue with Google Create a free account Log in

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero Zotero guide
Citation alerts 298 papers already cite it: be emailed each new one. Citation alerts guide
Who works on it?

Among the 31 papers around this one, the name that comes back most: Yi-Hui He (2 papers)

5 more people sign several of them. The full map adds their institutions and approaches, from the papers of the last years.

Continue with Google Create a free account: your first map is free
Weekly letter The latest letter of its field, "Artificial intelligence this week" of 16 November 2026: Artificial intelligence this week: What impressive scores leave out · Receive one by email for free

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Very close Builds on this paper Cites this paper Uses this paper in depth
    Understanding the Impact of Dynamic Channel Pruning on Conditionally Parameterized Convolutions
    Ravi S. Raju, Dibakar Gope, Urmish Thakker et al. · 2020 · AIChallengeIoT@SenSys · cited 4 times · Not summarized yet

    Understanding the Impact analyzes FBS’s runtime threshold gating and its effects on conditionally parameterized convolutions.

    Abstract

    Recent trends have shown that deep learning models have become larger and more accurate at an increased computational cost, making them difficult to deploy for latency-constrained applications. Conditional execution methods address this increase in cost while preserving the accuracy of the model by only performing the most important computation on important features. In this paper, we analyze a recent method, Feature Boosting and Suppression (FBS), which dynamically assesses which channels contain the most important input-dependent features and prune the others based on a runtime threshold gating mechanism. FBS is able to dynamically prune convolution filters with little loss in accuracy at …

  2. Very close Another approach to the problem
    Accelerating Convolutional Neural Networks with Dynamic Channel Pruning
    Chi-Liang Zhang, Tao Hu, Yingda Guan et al. · 2019 · Data Compression Conference · cited 21 times · Not summarized yet

    Accelerating Convolutional Neural Networks with Dynamic Channel Pruning proposes channel pruning to reduce CNN computation, a problem also addressed by FBS.

    Abstract

    Network acceleration has become a hot topic, for the substantial challenge in deploying such networks in real-time applications or on resource-limited devices. A wide variety of pruning-based acceleration methods were proposed to expend the sparsity of parameters, thus omit computations involving those pruned parameters. However, these element-wise pruning methods can hardly be efficiently used for accelerating without special-customized speed-up algorithms. Due to this difficulty, recent work has turned to prune filters or channels instead, which directly reduce the number of matrix multiplications. While Channel Pruning method reforms the original CNNs to a kernel-wisely or channel-wisely …

  3. Very close Another approach to the problem
    A Channel Pruning Method Using Subtraction-Average-Based Optimizer
    Zhou-Feng Liu, Wentao Wu, Shu-Min Ding et al. · 2024 · International Conference on Robotics, Intelligent Control and Artificial Intelligence · Not summarized yet

    SABOPruner uses a subtraction-average-based optimizer to select channels for pruning, rather than FBS’s input-dependent runtime selection.

    Abstract

    Convolutional Neural Networks(CNNs) require huge resources, which hindered the implement on light devices. However, existing pruning techniques primarily concentrate on choosing channels based on their importance or on regularization strategies that are crafted through empirical knowledge, the compressed networks obtained through these methods have unstable performance and require unnecessary human intervention. This paper proposes a new channel pruning method based on the Subtraction-Average-Based Optimizer, dubbed as SABOPruner. To facilitate deployment on FGPA, we constrain the pruning step size of each layer to be a multiple of 8. Finally, the search was transformed into an optimization …

  4. Very close Another approach to the problem
    Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networks
    Zhouyang Xie, Yan Fu, Sheng-Zhao Tian et al. · 2021 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet

    Pruning with Compensation proposes an efficient channel-pruning procedure to reduce deep CNN inference costs, unlike FBS’s dynamic channel skipping.

    Abstract

    Channel pruning is a promising technique to compress the parameters of deep convolutional neural networks(DCNN) and to speed up the inference. This paper aims to address the long-standing inefficiency of channel pruning. Most channel pruning methods recover the prediction accuracy by re-training the pruned model from the remaining parameters or random initialization. This re-training process is heavily dependent on the sufficiency of computational resources, training data, and human interference(tuning the training strategy). In this paper, a highly efficient pruning method is proposed to significantly reduce the cost of pruning DCNN. The main contributions of our method include: 1) pruning …

  5. Very close Another approach to the problem
    Pruning Very Deep Neural Network Channels for Efficient Inference
    Yi-Hui He · 2022 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet

    Pruning Very Deep Neural Network Channels selects channels using LASSO regression and least-squares reconstruction to accelerate CNN inference.

    Abstract

    —In this paper, we introduce a new channel pruning method to accelerate very deep convolutional neural networks. Given a trained CNN model, we propose an iterative two-step algorithm to effectively prune each layer, by a LASSO regression based channel selection and least square reconstruction. We further generalize this algorithm to multi-layer and multi-branch cases. Our method reduces the accumulated error and enhances the compatibility with various architectures. Our pruned VGG-16 achieves the state-of-the-art results by 5 × speed-up along with only 0.3% increase of error. More importantly, our method is able to accelerate modern networks like ResNet, Xception and suffers only 1.4%, 1.0% …

  6. Very close Another approach to the problem Cites this paper Uses this paper in depth
    Boosting the Performance of CNN Accelerators with Dynamic Fine-Grained Channel Gating
    Weizhe Hua, Yuan Zhou, Christopher De Sa et al. · 2019 · Micro · cited 48 times · Not summarized yet

    Dynamic Fine-Grained Channel Gating uses input-specific channel gating in feature-map regions and develops an accelerator to exploit that sparsity.

    Abstract

    This paper proposes a new fine-grained dynamic pruning technique for CNN inference, named channel gating, and presents an accelerator architecture that can effectively exploit the dynamic sparsity. Intuitively, channel gating identifies the regions in the feature map of each CNN layer that contribute less to the classification result and turns off a subset of channels for computing the activations in these less important regions. Unlike static network pruning, which removes redundant weights or neurons prior to inference, channel gating exploits dynamic sparsity specific to each input at run time and in a structured manner. To maximize compute savings while minimizing accuracy loss, channel …

  7. Very close Broader context
    Dynamic Neural Network Channel Execution for Efficient Training
    Simeon E. Spasov, P. Lio' · 2019 · British Machine Vision Conference · cited 6 times · Not summarized yet

    Dynamic Neural Network Channel Execution selects salient channels during training as well as inference, extending channel execution to training efficiency.

    Abstract

    Existing methods for reducing the computational burden of neural networks at run-time, such as parameter pruning or dynamic computational path selection, focus solely on improving computational efficiency during inference. On the other hand, in this work, we propose a novel method which reduces the memory footprint and number of computing operations required for training and inference. Our framework efficiently integrates pruning as part of the training procedure by exploring and tracking the relative importance of convolutional channels. At each training step, we select only a subset of highly salient channels to execute according to the combinatorial upper confidence bound algorithm, and r…

  8. Very close Builds on this paper Cites this paper Uses this paper in depth
    CDCP: A Framework for Accelerating Dynamic Channel Pruning Methods
    Zhaohong Xiang, Yi-Gui Luo, Yin Xie et al. · 2023 · International Conference on Parallel and Distributed Systems · Not summarized yet

    CDCP proposes a framework to accelerate dynamic channel-pruning methods, including the class of methods represented by FBS.

    Abstract

    Dynamic channel pruning is a technique aimed at reducing the theoretical computational complexity and inference latency of convolutional neural networks. Dynamic channel pruning methods introduce complex additional modules for dynamically selecting channels for images. Due to the additional modules, dynamic channel pruning methods never achieve optimal acceleration effect in real world. To address this problem, we propose Consecutive Dynamic Channel Pruning (CDCP), a novel dynamic channel pruning framework unified for almost all dynamic pruning methods designed for continuous image processing. The core idea of CDCP stems from our observation that adjusting the network for all frames in seman…

Prior work it builds on

  1. Close Another approach to the problem Used in depth by this paper A method it uses
    Pruning Filters for Efficient ConvNets
    Hao Li, Asim Kadav, Igor Durdanovic et al. · 2016 · International Conference on Learning Representations · cited 4,297 times · Not summarized yet

    Pruning Filters removes whole filters with small effects on output accuracy to accelerate CNNs, whereas FBS skips channels dynamically by input.

    Abstract

    The success of CNNs in various applications is accompanied by a significant increase in the computation and parameter storage costs. Recent efforts toward reducing these overheads involve pruning and compressing the weights of various layers without hurting original accuracy. However, magnitude-based pruning of weights reduces a significant number of parameters from the fully connected layers and may not adequately reduce the computation costs in the convolutional layers due to irregular sparsity in the pruned networks. We present an acceleration method for CNNs, where we prune filters from CNNs that are identified as having a small effect on the output accuracy. By removing whole filters in…

  2. Close Another approach to the problem A method it uses
    Discrimination-aware Channel Pruning for Deep Neural Networks
    Zhuangwei Zhuang, Mingkui Tan, Bo-Han Zhuang et al. · 2018 · Neural Information Processing Systems · cited 648 times · Not summarized yet

    Discrimination-aware Channel Pruning selects channels for their contribution to discriminative power, rather than FBS’s input-dependent runtime salience.

    Abstract

    Channel pruning is one of the predominant approaches for deep model compression. Existing pruning methods either train from scratch with sparsity constraints on channels, or minimize the reconstruction error between the pre-trained feature maps and the compressed ones. Both strategies suffer from some limitations: the former kind is computationally expensive and difficult to converge, whilst the latter kind optimizes the reconstruction error but ignores the discriminative power of channels. To overcome these drawbacks, we investigate a simple-yet-effective method, called discrimination-aware channel pruning, to choose those channels that really contribute to discriminative power. To this end…

  3. Close Another approach to the problem A method it uses
    Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
    Yang He, Guoliang Kang, Xuanyi Dong et al. · 2018 · Computer Vision and Pattern Recognition · cited 1,121 times · AI summary available

    Soft Filter Pruning updates pruned filters during training to accelerate CNN inference, while FBS dynamically skips channels at runtime.

    Abstract

    This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP enables the pruned filters to be updated when training the model after pruning. SFP has two advantages over previous works: (1) Larger model capacity. Updating previously pruned filters provides our approach with larger optimization space than fixing the filters to zero. Therefore, the network trained by our method has a larger model capacity to learn from the training data. (2) Less dependence on the pre-trained model. Large capacity enables SFP to train from scratch and prune the model simultaneously. In contrast, previous…

  4. Close Another approach to the problem A method it uses
    ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
    Jian-Hao Luo, Jian-Xin Wu, Wei-Yao Lin · 2017 · IEEE International Conference on Computer Vision · cited 1,911 times · Not summarized yet

    ThiNet prunes filters using statistics from the next layer to compress and accelerate CNNs, rather than selecting channels dynamically per input.

    Abstract

    We propose an efficient and unified framework, namely ThiNet, to simultaneously accelerate and compress CNN models in both training and inference stages. We focus on the filter level pruning, i.e., the whole filter would be discarded if it is less important. Our method does not change the original network structure, thus it can be perfectly supported by any off-the-shelf deep learning libraries. We formally establish filter pruning as an optimization problem, and reveal that we need to prune filters based on statistics information computed from its next layer, not the current layer, which differentiates ThiNet from existing methods. Experimental results demonstrate the effectiveness of this …

  5. Close Another approach to the problem A method it uses
    Learning to Prune Filters in Convolutional Neural Networks
    Qiangui Huang, Kevin Zhou, Su-Ya You et al. · 2018 · IEEE Workshop/Winter Conference on Applications of Computer Vision · cited 187 times · Not summarized yet

    Learning to Prune Filters trains agents to remove unnecessary CNN filters, rather than using FBS’s runtime channel selection.

    Abstract

    Many state-of-the-art computer vision algorithms use large scale convolutional neural networks (CNNs) as basic building blocks. These CNNs are known for their huge number of parameters, high redundancy in weights, and tremendous computing resource consumptions. This paper presents a learning algorithm to simplify and speed up these CNNs. Specifically, we introduce a “try-and-learn” algorithm to train pruning agents that remove unnecessary CNN filters in a data-driven way. With the help of a novel reward function, our agents removes a significant number of filters in CNNs while maintaining performance at a desired level. Moreover, this method provides an easy control of the tradeoff between n…

Papers that cite it

  1. Close* Builds on this paper Uses this paper in depth
    BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks
    Shangqian Gao, Yan-Fu Zhang, Feihu Huang et al. · 2024 · Computer Vision and Pattern Recognition · cited 29 times · Not summarized yet

    BilevelPruning unifies dynamic and static channel pruning, and cites FBS as related work.

  2. Close Builds on this paper Uses this paper in depth
    Multidimensional Dynamic Pruning: Exploring Spatial and Channel Fuzzy Sparsity
    Mingwen Shao, Jiandong Kuang, Chao Wang et al. · 2024 · IEEE transactions on fuzzy systems · cited 8 times · Not summarized yet

    Multidimensional Dynamic Pruning extends dynamic pruning across channel and spatial dimensions and cites FBS.

    Abstract

    Dynamic pruning is an effective model compression method to reduce the computational cost of networks. However, existing dynamic pruning methods are limited to pruning along a single dimension (channel, spatial, or depth), which cannot maximally excavate the redundancy of the network. Meanwhile, most of the current state-of-the-arts usually implement dynamic pruning via masked-out partial channels and pixels for training while failing to accelerate the inference speed. To tackle these limitations, we propose a novel fuzzy-based multidimensional dynamic pruning paradigm to dynamically compress neural networks along both the channel and spatial dimensions. Specifically, we design a multidimens…

  3. Close Builds on this paper Uses this paper in depth
    Self-Adaptive Network Pruning
    Jinting Chen, Zhaocheng Zhu, Chengwei Li et al. · 2019 · International Conference on Neural Information Processing · cited 24 times · Not summarized yet

    Self-Adaptive Network Pruning predicts channel saliency per layer and sample under a computation budget, citing FBS among related methods.

    Abstract

    Deep convolutional neural networks have been proved successful on a wide range of tasks, yet they are still hindered by their large computation cost in many industrial scenarios. In this paper, we propose to reduce such cost for CNNs through a self-adaptive network pruning method (SANP). Our method introduces a general Saliency-and-Pruning Module (SPM) for each convolutional layer, which learns to predict saliency scores and applies pruning for each channel. Given a total computation budget, SANP adaptively determines the pruning strategy with respect to each layer and each sample, such that the average computation cost meets the budget. This design allows SANP to be more efficient in comput…

  4. Close Builds on this paper Uses this paper in depth
    Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Prediction
    Sara Elkerdawy, Mostafa Elhoushi, Hong Zhang et al. · 2021 · Computer Vision and Pattern Recognition · cited 62 times · Not summarized yet

    Fire Together Wire Together predicts dynamic pruning masks with self-supervision and cites FBS as an earlier dynamic pruning method.

    Abstract

    Dynamic model pruning is a recent direction that allows for the inference of a different sub-network for each input sample during deployment. However, current dynamic methods rely on learning a continuous channel gating through regularization by inducing sparsity loss. This formulation introduces complexity in balancing different losses (e.g task loss, regularization loss). In addition, regularization based methods lack transparent tradeoff hyper- parameter selection to realize a computational budget. Our contribution is two-fold: 1) decoupled task and pruning losses. 2) Simple hyperparameter selection that enables FLOPs reduction estimation before training. Inspired by the Hebbian theory in…

  5. Close Builds on this paper Uses this paper in depth
    Progressive Channel-Shrinking Network
    Jianhong Pan, Si-Yuan Yang, Lin Geng Foo et al. · 2023 · IEEE transactions on multimedia · cited 5 times · Not summarized yet

    Progressive Channel-Shrinking addresses gating and indexing costs in dynamic channel pruning and cites FBS as prior work.

    Abstract

    Currently, salience-based channel pruning makes continuous breakthroughs in network compression. In the realization, the salience mechanism is used as a metric of channel salience to guide pruning. Therefore, salience-based channel pruning can dynamically adjust the channel width at run-time, which provides a flexible pruning scheme. However, there are two problems emerging: a gating function is often needed to truncate the specific salience entries to zero, which destabilizes the forward propagation; dynamic architecture brings more cost for indexing in inference which bottlenecks the inference speed. In this article, we propose a Progressive Channel-Shrinking (PCS) method to compress the s…

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 4 November 2026.

The illustrated guide to the papers around a paper

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.