Results of the summarizing process for the arXiv paper: 2306.14043v1

Randomness plays a crucial role in various aspects of machine learning (ML), including optimization, data selection, privacy, and security. However, attackers have a history of exploiting poor randomness or even creating it to compromise ML systems. This paper focuses on Randomised Smoothing, a popular approach used to train robust models and certify specific input datapoints. Randomised Smoothing relies on sampling Gaussian noise to explore the volume around a data point and certify that a model is not vulnerable to adversarial examples. The authors propose an entirely novel attack against Randomised Smoothing where an attacker manipulates the supplied randomness to falsely certify either an overestimate or an underestimate of robustness. They demonstrate that such attacks are possible with minimal changes to randomness and can be challenging to detect. As an example, they hide the attack in the random number generator and show that the randomness tests suggested by NIST fail to detect it. The paper highlights the need for updating the NIST guidelines on random number testing to make them more suitable for safety-critical and security-critical machine learning applications. It emphasizes the importance of considering the potential vulnerabilities associated with randomness in ML systems and suggests improving random number generation techniques for better security. In conclusion, this research sheds light on the risks posed by compromised randomness in ML systems and presents a new attack method against Randomised Smoothing. It calls for stronger measures in assessing and ensuring the integrity of random number generators used in machine learning applications.

• - Randomness plays a crucial role in machine learning (ML) in various aspects
• - Poor randomness can be exploited by attackers to compromise ML systems
• - Randomised Smoothing is a popular approach for training robust models and certifying input datapoints
• - The paper introduces a novel attack against Randomised Smoothing by manipulating supplied randomness
• - Attacks can falsely certify an overestimate or underestimate of robustness with minimal changes to randomness
• - The attack can be hidden in the random number generator and is difficult to detect using NIST tests
• - NIST guidelines on random number testing need updating for safety-critical and security-critical ML applications
• - Importance of considering vulnerabilities associated with randomness in ML systems
• - Need for improving random number generation techniques for better security in ML applications.

Randomness is very important in machine learning. It helps us make decisions and learn new things. But if the randomness is not good, bad people can use it to harm our machine learning systems. Randomised Smoothing is a way to train our models better and make sure the information we put into them is correct. But there are some new attacks that try to manipulate the randomness and trick us into thinking our models are safe when they're not. These attacks are hard to detect and we need better ways to test the randomness for safety and security in machine learning." Definitions- Randomness: When something happens by chance or without a pattern. - Machine Learning (ML): A type of computer program that can learn from data and make decisions on its own. - Exploit: To take advantage of something for personal gain or harm. - Robust: Strong or resistant against problems or attacks. - Certify: To confirm or guarantee that something is true or accurate. - Manipulate: To change or control something in a dishonest way. - Random Number Generator: A tool used to create random numbers in computer programs. - NIST Tests: Tests created by the National Institute of Standards and Technology to check if random number generators are working properly. - Safety-critical: Something that could cause harm if it doesn't work correctly. - Security-critical: Something that could be attacked or hacked if it's not secure enough.

Randomness and Machine Learning: Exploring the Risks of Compromised Randomness

Randomness plays a crucial role in various aspects of machine learning (ML), from optimization to data selection, privacy, and security. Unfortunately, attackers have a history of exploiting poor randomness or even creating it to compromise ML systems. This paper focuses on Randomised Smoothing, a popular approach used to train robust models and certify specific input datapoints. It proposes an entirely novel attack against Randomised Smoothing where an attacker manipulates the supplied randomness to falsely certify either an overestimate or an underestimate of robustness.

What is Randomised Smoothing?

Randomised Smoothing is a technique used to train robust models by adding Gaussian noise around each data point in order to explore its volume. This helps identify potential adversarial examples that could be misclassified by the model. By sampling this noise multiple times and averaging out the results, one can certify that a given model is not vulnerable to such attacks with high confidence.

The Novel Attack Against Randomised Smoothing

The authors propose an attack against Randomised Smoothing where an attacker manipulates the supplied randomness in order to falsely certify either an overestimate or an underestimate of robustness for any given input datapoints. They demonstrate that such attacks are possible with minimal changes to randomness and can be challenging to detect - as they hide them within the random number generator itself - making them difficult for standard tests like those suggested by NIST (National Institute of Standards and Technology) guidelines on random number testing fail at detecting them.

Implications & Conclusion

This research sheds light on the risks posed by compromised randomness in ML systems and presents a new attack method against Randomised Smoothing which calls for stronger measures in assessing and ensuring integrity of random number generators used in machine learning applications. It emphasizes importance of considering potential vulnerabilities associated with random numbers when building ML systems as well as suggests improving techniques for better security when generating these numbers randomly