Beyond Labels: Empowering Human with Natural Language Explanations through a Novel Active-Learning Architecture

AI-generated keywords: Data Annotation Active Learning Natural Language Explanations Data Diversity-based AL Explainability

AI-generated Key Points

  • Data annotation is a costly task in machine learning
  • Existing Active-Learning (AL) techniques focus on labeling data points but overlook the importance of providing natural language explanations alongside the labels
  • Proposed AL architecture generates both labels and explanations to reduce the need for human annotations
  • Explanation-generation model assists humans in decision-making process in real-world applications
  • AL data selection strategy leverages explanation annotations and outperforms traditional strategies
  • Human evaluation shows preference for explanations generated by proposed system over state-of-the-art system
  • Lack of explainability in language models may lead to mistrust of predictions
  • Proposed framework has limitations including applicability to other NLP tasks beyond classification datasets and reliance on data diversity-based AL selector design
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Bingsheng Yao, Ishan Jindal, Lucian Popa, Yannis Katsis, Sayan Ghosh, Lihong He, Yuxuan Lu, Shashank Srivastava, James Hendler, Dakuo Wang

License: CC BY 4.0

Abstract: Data annotation is a costly task; thus, researchers have proposed low-scenario learning techniques like Active-Learning (AL) to support human annotators; Yet, existing AL works focus only on the label, but overlook the natural language explanation of a data point, despite that real-world humans (e.g., doctors) often need both the labels and the corresponding explanations at the same time. This work proposes a novel AL architecture to support and reduce human annotations of both labels and explanations in low-resource scenarios. Our AL architecture incorporates an explanation-generation model that can explicitly generate natural language explanations for the prediction model and for assisting humans' decision-making in real-world. For our AL framework, we design a data diversity-based AL data selection strategy that leverages the explanation annotations. The automated AL simulation evaluations demonstrate that our data selection strategy consistently outperforms traditional data diversity-based strategy; furthermore, human evaluation demonstrates that humans prefer our generated explanations to the SOTA explanation-generation system.

Submitted to arXiv on 22 May. 2023

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2305.12710v1

Data annotation is a costly task in machine learning, and researchers have proposed Active-Learning (AL) techniques to support human annotators. However, existing AL works only focus on labeling data points and overlook the importance of providing natural language explanations alongside the labels. In real-world scenarios, such as medical diagnoses, humans often require both labels and corresponding explanations simultaneously. To address this gap, this work proposes a novel AL architecture that reduces the need for human annotations by generating both labels and explanations. The proposed AL architecture incorporates an explanation-generation model that can explicitly generate natural language explanations for the prediction model. These explanations assist humans in their decision-making process in real-world applications. The framework also includes a data diversity-based AL data selection strategy that leverages the explanation annotations. To evaluate the effectiveness of the proposed approach, automated AL simulation evaluations were conducted. The results consistently showed that the data selection strategy outperformed traditional data diversity-based strategies. Additionally, a human evaluation was conducted to compare the generated explanations with those from a state-of-the-art (SOTA) explanation-generation system. The results revealed that humans preferred the explanations generated by the proposed system. The introduction highlights recent advancements in Natural Language Processing (NLP) and emphasizes the need for explainability within language models. While these models demonstrate impressive performance on various NLP tasks, their lack of faithful explainability may lead to mistrust of their predictions. Humans typically develop intermediate information as rationales to aid decision making which serves as faithful explanations. The paper also discusses limitations of the proposed framework including its applicability to other NLP tasks beyond classification datasets like e-SNLI and its reliance on a data diversity based AL selector design. Overall, this work presents an innovative approach to reduce human annotations by incorporating an explanation generation model into an AL architecture. The results demonstrate improved performance compared to traditional strategies and highlight human preference for generated explanations over existing systems.
Created on 21 Sep. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.