Supporting AI/ML Security Workers through an Adversarial Techniques, Tools, and Common Knowledge (AI/ML ATT&CK) Framework

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Same question Not relevant

   Supporting Artificial Intelligence/Machine Learning Security Workers Through an Adversarial Techniques, Tools, and Common Knowledge Framework

   Mohamad Fazelnia, Ahmet Okutan, Mehdi Mirakhorli · 2023 · IEEE Security and Privacy · cited 6 times · Not summarized yet

   The article presents the same AI/ML ATT&CK framework to help AI/ML security workers explore offensive and defensive tactics.

   Abstract

   This article focuses on supporting artificial intelligence (AI)/machine learning (ML) security workers. It presents AI/ML adversarial techniques, tools, and common knowledge (AI/ML ATT&CK) framework to enable AI/ML security workers to intuitively explore offensive and defensive tactics.

2. Very close Another approach to the problem Not relevant

   Attacks, Defenses, And Tools: A Framework To Facilitate Robust AI/ML Systems

   Mohamad Fazelnia, I. Khokhlov, Mehdi Mirakhorli · 2022 · arXiv.org · preprint, not peer reviewed · cited 9 times · Not summarized yet

   Attacks, Defenses, And Tools presents a framework for characterizing AI-system attacks and weaknesses and offering mitigations for software designers.

   Abstract

   Software systems are increasingly relying on Artificial Intelligence (AI) and Machine Learning (ML) components. The emerging popularity of AI techniques in various application domains attracts malicious actors and adversaries. Therefore, the developers of AI-enabled software systems need to take into account various novel cyber-attacks and vulnerabilities that these systems may be susceptible to. This paper presents a framework to characterize attacks and weaknesses associated with AI-enabled systems and provide mitigation techniques and defense strategies. This framework aims to support software designers in taking proactive measures in developing AI-enabled software, understanding the atta…

3. Close Another approach to the problem Not relevant

   Offensive security for AI systems: concepts, practices, and applications

   Joshua D. Harguess, Chris M. Ward · 2025 · Defense + Security · cited 7 times · Not summarized yet

   Offensive security for AI systems proposes threat simulation and adversarial testing across the AI lifecycle to uncover vulnerabilities.

   Abstract

   As artificial intelligence (AI) systems become increasingly adopted across sectors, the need for robust, proactive security strategies is paramount. Traditional defensive measures often fall short against the unique and evolving threats facing AI-driven technologies, making offensive security an essential approach for identifying and mitigating risks. This paper presents a comprehensive framework for offensive security in AI systems, emphasizing proactive threat simulation and adversarial testing to uncover vulnerabilities throughout the AI lifecycle. We examine key offensive security techniques, including weakness and vulnerability assessment, penetration testing, and red teaming, tailored …

4. Close Same question Not relevant

   Adversarial Attacks and Defenses for Deployed AI Models

   Kishor Datta Gupta, D. Dasgupta · 2022 · IT Professional · cited 9 times · Not summarized yet

   Adversarial Attacks and Defenses for Deployed AI Models examines AI/ML attacks, defenses, and the expertise needed to evaluate them.

   Abstract

   With the surge in the adoption of AI/ML techniques in industry, adversarial challenges are also on the rise and defense strategies need to be configured accordingly. While it is crucial to formulate new attack methods (similar to Fuzz testing) and devise novel defense strategies for coverage and robustness, it is also imperative to recognize who is responsible for implementing, validating, and justifying the necessity of AI/ML defenses. In particular, which components of the learning system are vulnerable to what type of adversarial attacks, and the expertise needed to realize the severity of such adversarial attacks. Also, how to evaluate and address the adversarial challenges to recommend …

5. Close Broader context Not relevant

   Securing AI Systems: A Guide to Known Attacks and Impacts

   Naoto Kiribuchi, Kengo Zenitani, Takayuki Semitsu · 2025 · arXiv.org · preprint, not peer reviewed · cited 4 times · Not summarized yet

   Securing AI Systems links eleven attacks on predictive and generative AI to impacts on confidentiality, integrity, and availability.

   Abstract

   Embedded into information systems, artificial intelligence (AI) faces security threats that exploit AI-specific vulnerabilities. This paper provides an accessible overview of adversarial attacks unique to predictive and generative AI systems. We identify eleven major attack types and explicitly link attack techniques to their impacts -- including information leakage, system compromise, and resource exhaustion -- mapped to the confidentiality, integrity, and availability (CIA) security triad. We aim to equip researchers, developers, security practitioners, and policymakers, even those without specialized AI security expertise, with foundational knowledge to recognize AI-specific risks and imp…

6. Close Broader context Not relevant

   AI for Security and Security for AI

   E. Bertino, Murat Kantarcioglu, C. Akcora et al. · 2021 · Conference on Data and Application Security and Privacy · cited 59 times · Not summarized yet

   AI for Security and Security for AI discusses both AI applications in security and adversarial methods that exploit AI vulnerabilities.

   Abstract

   On one side, the security industry has successfully adopted some AI-based techniques. Use varies from mitigating denial of service attacks, forensics, intrusion detection systems, homeland security, critical infrastructures protection, sensitive information leakage, access control, and malware detection. On the other side, we see the rise of Adversarial AI. Here the core idea is to subvert AI systems for fun and profit. The methods utilized for the production of AI systems are systematically vulnerable to a new class of vulnerabilities. Adversaries are exploiting these vulnerabilities to alter AI system behavior to serve a malicious end goal. This panel discusses some of these aspects.

7. Close Broader context Not relevant

   Poster CTI4AI: Threat Intelligence Generation and Sharing after Red Teaming AI Models

   Chuyen Nguyen, Caleb Morgan, Sudip Mittal · 2022 · Conference on Computer and Communications Security · cited 4 times · Not summarized yet

   CTI4AI prototypes a system for identifying AI/ML vulnerabilities through red teaming and sharing threat intelligence among stakeholders.

   Abstract

   As the practicality of Artificial Intelligence (AI) and Machine Learning (ML) based techniques grow, there is an ever increasing threat of adversarial attacks. There is a need to "red team' this ecosystem to identify system vulnerabilities, potential threats, characterize properties that will enhance system robustness, and encourage the creation of effective defenses. A secondary need is to share this AI security threat intelligence between different stakeholders like, model developers, users, and AI/ML security professionals. In this paper, we create and describe a prototype system CTI4AI, to overcome the need to methodically identify and share AI/ML specific vulnerabilities and threat inte…

8. Close Broader context Not relevant

   Adversarial Machine Learning-Industry Perspectives

   Ram Shankar Siva Kumar, Magnus Nyström, J. Lambert et al. · 2020 · 2020 IEEE Security and Privacy Workshops (SPW) · cited 291 times · Not summarized yet

   Adversarial Machine Learning-Industry Perspectives uses interviews to identify industry gaps and propose security-development research directions.

   Abstract

   Based on interviews with 28 organizations, we found that industry practitioners are not equipped with tactical and strategic tools to protect, detect and respond to attacks on their Machine Learning (ML) systems. We leverage the insights from the interviews and enumerate the gaps in securing machine learning systems when viewed in the context of traditional software security development. We write this paper from the perspective of two personas: developers/ML engineers and security incident responders. The goal of this paper is to layout the research agenda to amend the Security Development Lifecycle for industrial-grade software in the adversarial ML era.

Prior work it builds on

1. Close Groundwork for this paper A method it uses Not relevant

   A System-Driven Taxonomy of Attacks and Defenses in Adversarial Machine Learning

   Koosha Sadeghi, Ayan Banerjee, Sandeep K. S. Gupta · 2020 · IEEE Transactions on Emerging Topics in Computational Intelligence · cited 78 times · Not summarized yet

   A System-Driven Taxonomy organizes adversarial ML attacks and defenses, and the paper cites it.

   Abstract

   Machine Learning (ML) algorithms, specifically supervised learning, are widely used in modern real-world applications, which utilize Computational Intelligence (CI) as their core technology, such as autonomous vehicles, assistive robots, and biometric systems. Attacks that cause misclassifications or mispredictions can lead to erroneous decisions resulting in unreliable operations. Designing robust ML with the ability to provide reliable results in the presence of such attacks has become a top priority in the field of adversarial machine learning. An essential characteristic for rapid development of robust ML is an arms race between attack and defense strategists. However, an important prere…

2. Related Groundwork for this paper A method it uses Not relevant

   Adversarial machine learning

   Ling Huang, A. Joseph, B. Nelson et al. · 2019 · Security and Artificial Intelligence · cited 1,596 times · Not summarized yet

   Adversarial machine learning classifies attacks, adversary capabilities, vulnerabilities, and countermeasures, and the paper cites it.

   Abstract

   In this paper (expanded from an invited talk at AISEC 2010), we discuss an emerging field of study: adversarial machine learning---the study of effective machine learning techniques against an adversarial opponent. In this paper, we: give a taxonomy for classifying attacks against online machine learning algorithms; discuss application-specific factors that limit an adversary's capabilities; introduce two models for modeling an adversary's capabilities; explore the limits of an adversary's knowledge about the algorithm, feature space, training, and input data; explore vulnerabilities in machine learning algorithms; discuss countermeasures against attacks; introduce the evasion challenge; and…

3. Related Broader context Not relevant

   On managing vulnerabilities in AI/ML systems

   Jonathan M. Spring, April Galyardt, A. Householder et al. · 2020 · New Security Paradigms Workshop · cited 24 times · Not summarized yet

   On managing vulnerabilities in AI/ML systems considers how vulnerability-management practices could cover ML flaws, and the paper cites it.

   Abstract

   This paper explores how the current paradigm of vulnerability management might adapt to include machine learning systems through a thought experiment: what if flaws in machine learning (ML) were assigned Common Vulnerabilities and Exposures (CVE) identifiers (CVE-IDs)? We consider both ML algorithms and model objects. The hypothetical scenario is structured around exploring the changes to the six areas of vulnerability management: discovery, report intake, analysis, coordination, disclosure, and response. While algorithm flaws are well-known in academic research community, there is no apparent clear line of communication between this research community and the operational communities that de…

4. Related Groundwork for this paper A method it uses Not relevant

   A taxonomy and terminology of adversarial machine learning

   Elham Tabassi, K. J. Burns, Michael T. Hadjimichael et al. · 2019 · cited 127 times · Not summarized yet

   A taxonomy and terminology of adversarial machine learning provides terminology for the field, and the paper cites it.

   Abstract

   <jats:p />

5. Related Groundwork for this paper Not relevant

   The Limitations of Deep Learning in Adversarial Settings

   Nicolas Papernot, P. McDaniel, S. Jha et al. · 2015 · European Symposium on Security and Privacy · cited 4,455 times · Not summarized yet

   The Limitations of Deep Learning in Adversarial Settings formalizes adversaries and develops attacks on deep neural networks, and the paper cites it.

   Abstract

   Deep learning takes advantage of large datasets and computationally efficient training algorithms to outperform other approaches at various machine learning tasks. However, imperfections in the training phase of deep neural networks make them vulnerable to adversarial samples: inputs crafted by adversaries with the intent of causing deep neural networks to misclassify. In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs. In an application to computer vision, we show that our algorithms can reliably produc…

Papers that cite it

1. Related Review Not relevant

   Exploring Research and Tools in AI Security: A Systematic Mapping Study

   Sidhant Narula, Mohammad GhasemiGol, Javier Carnerero-Cano et al. · 2025 · IEEE Access · cited 11 times · Not summarized yet

   Exploring Research and Tools in AI Security maps and synthesizes AI security research, and it cites the paper.

   Abstract

   With the pervasive integration of artificial intelligence (AI) in various facets of modern technology, the importance of AI security has been thrust into the spotlight. The field is rapidly evolving, with new challenges and solutions emerging at a swift pace. However, the breadth and depth of AI security research have not been comprehensively mapped in recent times, presenting a crucial need for an extensive review and synthesis of existing literature. Given the increasing reliance on AI in critical domains such as healthcare, finance, and national security, ensuring the resilience and trustworthiness of these systems is imperative. This survey fulfills the pressing need for a structured and…