# Practical Scheduling for Real-World Serverless Computing

**Authors:** Kostis Kaffes, Neeraja J. Yadwadkar, Christos Kozyrakis  
**arXiv:** [2111.07226](https://arxiv.org/abs/2111.07226)  
**Date:** 2021-11-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2111.07226v1/

## Key points

- Serverless computing is popular due to its ease-of-use and cost-efficiency
- Function scheduling within serverless systems is a critical aspect that has been overlooked
- The authors design a scheduler specifically tailored to serverless functions
- They create a taxonomy of scheduling policies and explore the policy space using simulation techniques
- Commonly used features like late binding and random load balancing are sub-optimal for typical execution time distributions and load ranges
- The authors propose Hermes, a novel scheduler with three key characteristics:
- Uses early binding and processor sharing to mitigate head-of-line blocking caused by high variability in function execution times
- Incorporates hybrid load balancing for consolidation during low loads and least-loaded balancing during high loads to optimize resource allocation based on workload demands
- Designed to be both load and locality-aware, reducing the number of cold starts and minimizing latency associated with starting new instances
- Hermes is implemented for Apache OpenWhisk and evaluated using real-world traces from Azure and other sources
- Results show significant improvements over existing policies, achieving up to 85% lower function slowdown and 60% higher throughput in various real-world scenarios

## Summary

In the rapidly growing field of serverless computing, the ease-of-use and cost-efficiency it provides have made it increasingly popular. However, one critical aspect that has been overlooked is function scheduling within serverless systems. In this paper, the authors address this gap by designing a scheduler specifically tailored to the unique characteristics of serverless functions as observed in real-world deployments. To begin, the authors create a taxonomy of scheduling policies along three dimensions. They then use simulation techniques to explore the scheduling policy space based on function characteristics derived from a 14-day trace of Azure functions. Through their analysis, they find that commonly used features such as late binding and random load balancing are sub-optimal for typical execution time distributions and load ranges. Using these insights, the authors propose Hermes, a novel scheduler for serverless functions with three key characteristics. First, to mitigate head-of-line blocking caused by high variability in function execution times, Hermes employs a combination of early binding and processor sharing for scheduling at individual worker machines. This approach ensures efficient utilization of resources. Secondly, Hermes incorporates a hybrid load balancing approach that enhances consolidation during low loads while employing least-loaded balancing during high loads to maintain high performance levels. This strategy optimizes resource allocation based on workload demands. Lastly, Hermes is designed to be both load and locality-aware, reducing the number of cold starts compared to pure load-based policies. By considering both workload distribution and proximity between functions and worker machines, Hermes minimizes latency associated with starting new instances. The authors implement Hermes for Apache OpenWhisk and evaluate its performance using real-world traces from Azure as well as other sources. The results demonstrate significant improvements over existing policies. Specifically, Hermes achieves up to 85% lower function slowdown and 60% higher throughput compared to current approaches when considering function patterns observed in various real-world scenarios. Overall, this paper presents an innovative approach to function scheduling in serverless computing environments. By addressing the unique characteristics of serverless functions and leveraging a combination of early binding, processor sharing, hybrid load balancing, and load locality awareness, Hermes offers substantial performance enhancements compared to existing policies.