# New Phenomena in Large-Scale Internet Traffic

**Authors:** Jeremy Kepner, Kenjiro Cho, KC Claffy, Vijay Gadepally, Sarah McGuire, Lauren Milechin, William Arcand, David Bestor, William Bergeron, Chansup Byun, Matthew Hubbell, Michael Houle, Michael Jones, Andrew Prout, Albert Reuther, Antonio Rosa, Siddharth Samsi, Charles Yee, Peter Michaleas  
**arXiv:** [2201.06096](https://arxiv.org/abs/2201.06096)  
**Date:** 2022-01-16  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2201.06096v1/

## Key points

- The Internet is undergoing a significant transformation, prompting the need for a deeper quantitative understanding of Internet traffic patterns.
- A team of researchers collected and curated extensive sets of publicly available Internet traffic data, analyzing 50 billion packets using 10,000 processors within the MIT SuperCloud infrastructure.
- Their groundbreaking discovery highlights the crucial role played by previously unnoticed leaf nodes and isolated links in shaping Internet traffic dynamics.
- Research findings point to the efficacy of a two-parameter modified Zipf-Mandelbrot distribution in accurately characterizing source/destination statistics across different data collections spanning various years and continents.
- The distribution model has shown promise in distinguishing between different network streams, with one specific parameter - the "leaf parameter" - exhibiting a strong correlation with traffic flowing through distinct network topologies.
- Insights from this study shed light on how hidden leaf nodes and isolated links contribute significantly to overall network behavior.
- The comprehensive study not only uncovers new phenomena within large-scale Internet traffic but also provides valuable insights for optimizing network performance and enhancing cybersecurity measures.

## Summary

The Internet is undergoing a significant transformation, prompting the need for a deeper quantitative understanding of Internet traffic patterns. A team of researchers has taken on the task of collecting and curating extensive sets of publicly available Internet traffic data to address this need. Through an analysis involving a staggering 50 billion packets and utilizing 10,000 processors within the MIT SuperCloud infrastructure, they have made a groundbreaking discovery. This discovery highlights the crucial role played by previously unnoticed leaf nodes and isolated links in shaping Internet traffic dynamics. Furthermore, their research findings point to the efficacy of a two-parameter modified Zipf-Mandelbrot distribution in accurately characterizing a diverse range of source/destination statistics. This distribution model has proven effective when applied to moving sample windows containing anywhere from 100,000 to 100,000,000 packets across data collections spanning various years and continents. Notably, the measured parameters of this model have shown promise in distinguishing between different network streams. Of particular significance is the revelation that one specific parameter within this model – referred to as the "leaf parameter" – exhibits a strong correlation with the proportion of traffic flowing through distinct underlying network topologies. This insight sheds light on how these hidden leaf nodes and isolated links contribute significantly to overall network behavior. The comprehensive study conducted by this multidisciplinary team not only uncovers new phenomena within large-scale Internet traffic but also provides valuable insights into how these intricate networks operate. The implications of these findings extend beyond theoretical understanding, offering practical applications for optimizing network performance and enhancing overall cybersecurity measures.