# Locally Sparse Networks for Interpretable Predictions

**Authors:** Junchen Yang, Ofir Lindenbaum, Yuval Kluger  
**arXiv:** [2106.06468](https://arxiv.org/abs/2106.06468)  
**Date:** 2021-06-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2106.06468v1/

## Key points

- Neural networks have been successful in various fields but face challenges when applied to low-sample-size datasets.
- Locally Sparse Interpretable Networks (LSPIN) is a framework proposed by researchers to address these issues.
- LSPIN relies on local sparsity learned through a sample-specific gating mechanism that identifies the most relevant features for each measurement.
- The gating network predicts the sample-specific sparsity probabilities which are trained alongside the prediction network.
- LSPIN obtains an interpretable neural network that can handle LSS data and remove nuisance variables irrelevant to supervised learning tasks.
- Interpretability is essential in machine learning, particularly in medicine or biology where practitioners require explanations for predictions.
- Datasets in bioinformatics or medicine are often high-dimensional with low sample sizes (HDLSS), making analysis tasks like dimensionality reduction challenging.
- The LSPIN method presents a flexible predictive model that relies on local sparsity of input features for fitting predictive models to LSS data.
- Extensive synthetic simulations showed that LSPIN can learn the correct target function and identify informative variables while requiring few observations.

## Summary

The success of neural networks in various fields has been enormous. However, they still face challenges when applied to low-sample-size (LSS) datasets such as overfitting and difficulty in interpretation. To address these issues, a team of researchers proposed a framework for training locally sparse neural networks called Locally Sparse Interpretable Networks (LSPIN). The LSPIN model relies on local sparsity learned through a sample-specific gating mechanism that identifies the most relevant features for each measurement. The gating network predicts the sample-specific sparsity probabilities which are trained alongside the prediction network. By learning subsets and weights of a prediction model, LSPIN obtains an interpretable neural network that can handle LSS data and remove nuisance variables irrelevant to supervised learning tasks. Interpretability is essential in machine learning, particularly in medicine or biology where practitioners require explanations for predictions. Interpretation algorithms such as gradient evaluation or perturbations have been developed to identify essential input features for predictions. Alternatively, assessing the contribution of each training sample to the prediction could explain neural nets. Datasets in bioinformatics or medicine are often high-dimensional with low sample sizes (HDLSS), making analysis tasks like dimensionality reduction challenging. HDLSS also poses a challenge in supervised learning since prediction models tend to overfit when underdetermined. Regularization schemes have been proposed to sparsify input features and prevent overfitting in deep nets. The LSPIN method presents a flexible predictive model that relies on local sparsity of input features for fitting predictive models to LSS data. A gating network predicts probabilities of instance-wise gates being active whose parameters and model coefficients are learned together by minimizing classification or regression loss. This parametric construction leads to an interpretable model relying on subsets of input features for each instance. Extensive synthetic simulations showed that LSPIN can learn the correct target function and identify informative variables while requiring few observations.