# Active Learning for Deep Neural Networks on Edge Devices

**Authors:** Yuya Senzaki, Christian Hamelain  
**arXiv:** [2106.10836](https://arxiv.org/abs/2106.10836)  
**Date:** 2021-06-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2106.10836v2/

## Key points

- The paper focuses on updating deep neural network (DNN) models on edge devices.
- The authors propose a practical active learning framework for DNNs on edge devices.
- The framework reduces the problem to stream submodular maximization.
- The framework is designed to run efficiently with low computational resources while providing high-quality solutions.
- The approach outperforms other methods and runs at a practical speed on real devices.
- Active learning has been extensively studied as a way to train models with limited labeled samples.
- The active learning framework considers both informativeness and diversity when selecting data for training.
- In cold-start scenarios, random data collection or a model-agnostic method can be used to create an initial dataset.

## Summary

This paper focuses on the problem of updating deep neural network (DNN) models on edge devices. While it is ideal to update the model with real incoming data, limitations such as labeling and communication costs make it necessary to filter and select the data for training. The authors propose a practical active learning framework for DNNs on edge devices, which reduces the problem to stream submodular maximization. This framework is designed to run efficiently with low computational resources while providing solutions that are theoretically guaranteed to be of high quality. The authors evaluate their approach on classification and object detection tasks in a practical setting and demonstrate that their framework outperforms other methods while running at a practical speed on real devices. In terms of related work, active learning has been extensively studied as a way to train models with limited labeled samples. The standard active learning setup involves selecting data from a pool of unlabeled candidates and obtaining annotations from an oracle. The authors highlight the importance of active learning in the context of increasing amounts of available data. The active learning framework introduced in this paper considers both informativeness and diversity when selecting data for training. Informativeness quantifies how useful a sample is, while diversity measures how well a batch of samples represents different aspects of the data. By combining these criteria into an objective function, the authors aim to maximize value while considering dependencies between samples. The paper provides an overview of the workflow involved in using their framework: training a model with labeled data, deploying/updating the trained model on the device, capturing input samples, running predictions, deciding whether to store or discard each sample based on informativeness and diversity criteria, using prediction results, repeating this process multiple times, sending selected data to a server for annotation, and annotating received data. In cold-start scenarios where there is no existing dataset, the authors suggest collecting data randomly or using a model-agnostic method on the device to create an initial dataset. This collection method could be inspired by the approach proposed in this paper. Overall, the paper presents a practical and efficient framework for active learning on edge devices demonstrating its effectiveness in classification and object detection tasks as well as discussing potential research topics such as constructing an initial dataset in cold-start scenarios.