# Learning Credible Deep Neural Networks with Rationale Regularization

**Authors:** Mengnan Du, Ninghao Liu, Fan Yang, Xia Hu  
**arXiv:** [1908.05601](https://arxiv.org/abs/1908.05601)  
**Date:** 2019-08-13  
**Summary page:** https://www.summarizepaper.com/en/paper/1908.05601v1/

## Key points

- State-of-the-art deep neural networks (DNNs) often fail to consider the correct evidence when making decisions
- This affects their ability to generalize and undermines their trustworthiness among end-users
- CREX (Credible Deep Neural Networks with Rationale Regularization) is introduced as a method to address this issue and develop more credible DNNs
- CREX encourages DNN models to focus on relevant evidence and avoid overfitting to data-dependent biases and artifacts
- It achieves this by regularizing the training process of DNNs with rationales, which are a subset of features identified by domain experts as justifications for predictions
- By enforcing DNNs to generate local explanations that align with expert rationales, CREX enhances the credibility of these models
- Even in cases where rationales are not available, CREX remains useful by requiring the generated explanations to be sparse
- Experimental results on two text classification datasets demonstrate that DNNs trained with CREX exhibit increased credibility
- Comprehensive analysis reveals that while CREX does not always improve prediction accuracy on held-out test sets, it significantly enhances DNN accuracy on new and previously unseen data beyond the test set
- The authors propose an effective approach for developing more trustworthy DNN models through CREX which encourages them to prioritize relevant evidence and avoid overfitting
- The experimental results support its effectiveness in enhancing model credibility and improving performance on unseen data

## Summary

In recent studies on explainability, it has been observed that state-of-the-art deep neural networks (DNNs) often fail to consider the correct evidence when making decisions. This not only affects their ability to generalize but also undermines their trustworthiness among end-users. To address this issue and develop more credible DNNs, this paper introduces a method called CREX (Credible Deep Neural Networks with Rationale Regularization). CREX aims to encourage DNN models to focus on the relevant evidence for a given task and avoid overfitting to data-dependent biases and artifacts. It achieves this by regularizing the training process of DNNs with rationales, which are a subset of features identified by domain experts as justifications for predictions. By enforcing DNNs to generate local explanations that align with expert rationales, CREX enhances the credibility of these models. Even in cases where rationales are not available, CREX remains useful by requiring the generated explanations to be sparse. Experimental results on two text classification datasets demonstrate that DNNs trained with CREX exhibit increased credibility. Furthermore, comprehensive analysis reveals that while CREX does not always improve prediction accuracy on held-out test sets, it significantly enhances DNN accuracy on new and previously unseen data beyond the test set. This highlights the advantage of increased credibility in real-world scenarios. The authors of this paper propose an effective approach for developing more trustworthy DNN models through CREX which encourages them to prioritize relevant evidence and avoid overfitting. The experimental results support its effectiveness in enhancing model credibility and improving performance on unseen data. Overall, this research contributes significantly to advancing the field of explainable AI and addressing important concerns regarding trustworthiness in deep learning systems.