# FLBench: A Benchmark Suite for Federated Learning

**Authors:** Yuan Liang, Yange Guo, Yanxia Gong, Chunjie Luo, Jianfeng Zhan, Yunyou Huang  
**arXiv:** [2008.07257](https://arxiv.org/abs/2008.07257)  
**Date:** 2020-08-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2008.07257v3/

## Key points

- FLBench is a benchmark suite for evaluating federated learning systems and algorithms across various domains.
- It captures the characteristics of real-world isolated data islands in domains like medical, financial, and AIoT.
- The primary objective of federated learning is to construct machine learning models using data distributed on multiple devices while ensuring security and privacy.
- FLBench provides a more accurate assessment by simulating isolated data island scenarios instead of manually partitioning public datasets.
- Researchers and developers can explore novel algorithms tailored to specific use cases within medical, financial, or AIoT sectors using FLBench.
- It assesses aspects such as communication efficiency, scenario transformation capability, privacy-preserving techniques, data distribution heterogeneity management, and cooperation strategies within federated learning systems.
- FLBench is continuously evolving as an open-source platform to meet the demands of federated learning research.
- Packaged as an automated deployment tool, it provides an environment for developing cutting-edge federated learning algorithms.

## Summary

FLBench is a comprehensive benchmark suite designed for evaluating federated learning systems and algorithms across various domains. It offers a realistic evaluation by capturing the intrinsic characteristics of real-world isolated data islands in domains such as medical, financial, and AIoT. The primary objective of federated learning is to construct machine learning models using data distributed on multiple devices while ensuring security and privacy. Unlike existing benchmarks that manually partition public datasets, FLBench provides a more accurate assessment by simulating isolated data island scenarios. The suite enables researchers and developers to explore novel algorithms and solutions tailored to specific use cases within the medical, financial, or AIoT sectors. It assesses essential aspects such as communication efficiency, scenario transformation capability, privacy-preserving techniques, data distribution heterogeneity management, and cooperation strategies within federated learning systems. FLBench is continuously evolving as an open-source platform to meet the growing demands of federated learning research. Packaged as an automated deployment tool, this benchmark suite provides a promising environment for developing cutting-edge federated learning algorithms. It was developed by researchers Yuan Liang, Yange Guo, Yanxia Gong, Chunjie Luo, Jianfeng Zhan,and Yunyou Huang with the aim of advancing the field of federated learning. For more information about FLBench and its features visit https://www.benchcouncil.org/flbench.html.