# Machine Learning needs its own Randomness Standard: Randomised Smoothing and PRNG-based attacks

**Authors:** Pranav Dahiya, Ilia Shumailov, Ross Anderson  
**arXiv:** [2306.14043](https://arxiv.org/abs/2306.14043)  
**Date:** 2023-06-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.14043v1/

## Key points

- Randomness plays a crucial role in machine learning (ML) in various aspects
- Poor randomness can be exploited by attackers to compromise ML systems
- Randomised Smoothing is a popular approach for training robust models and certifying input datapoints
- The paper introduces a novel attack against Randomised Smoothing by manipulating supplied randomness
- Attacks can falsely certify an overestimate or underestimate of robustness with minimal changes to randomness
- The attack can be hidden in the random number generator and is difficult to detect using NIST tests
- NIST guidelines on random number testing need updating for safety-critical and security-critical ML applications
- Importance of considering vulnerabilities associated with randomness in ML systems
- Need for improving random number generation techniques for better security in ML applications.

## Summary

Randomness plays a crucial role in various aspects of machine learning (ML), including optimization, data selection, privacy, and security. However, attackers have a history of exploiting poor randomness or even creating it to compromise ML systems. This paper focuses on Randomised Smoothing, a popular approach used to train robust models and certify specific input datapoints. Randomised Smoothing relies on sampling Gaussian noise to explore the volume around a data point and certify that a model is not vulnerable to adversarial examples. The authors propose an entirely novel attack against Randomised Smoothing where an attacker manipulates the supplied randomness to falsely certify either an overestimate or an underestimate of robustness. They demonstrate that such attacks are possible with minimal changes to randomness and can be challenging to detect. As an example, they hide the attack in the random number generator and show that the randomness tests suggested by NIST fail to detect it. The paper highlights the need for updating the NIST guidelines on random number testing to make them more suitable for safety-critical and security-critical machine learning applications. It emphasizes the importance of considering the potential vulnerabilities associated with randomness in ML systems and suggests improving random number generation techniques for better security. In conclusion, this research sheds light on the risks posed by compromised randomness in ML systems and presents a new attack method against Randomised Smoothing. It calls for stronger measures in assessing and ensuring the integrity of random number generators used in machine learning applications.