# Knowledge Graph Based Agent for Complex, Knowledge-Intensive QA in Medicine

**Authors:** Xiaorui Su, Yibo Wang, Shanghua Gao, Xiaolong Liu, Valentina Giunchiglia, Djork-Arné Clevert, Marinka Zitnik  
**arXiv:** [2410.04660](https://arxiv.org/abs/2410.04660)  
**Date:** 2024-10-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.04660v1/

## Key points

- Biomedical knowledge is complex and requires unique reasoning strategies compared to other scientific disciplines like physics or chemistry.
- Biomedical scientists use rule-based, prototype-based, and case-based reasoning to navigate this complexity.
- A new KG-based agent called KGARevion has been introduced to address these challenges by leveraging LLM's knowledge base and verifying generated triplets against a grounded KG for accuracy.
- KGARevion significantly improves accuracy in handling complex medical questions, with over 5.2% improvement compared to other models.
- KGARevion achieved a remarkable 10.4% improvement in accuracy on three new medical QA datasets with varying levels of semantic complexity.
- Existing language model-based methods face challenges in handling complex medical reasoning, leading researchers to explore alternative approaches such as retrieval-augmented generation (RAG).
- KGARevion represents a novel approach that combines non-codified knowledge from language models with structured codified knowledge from KGs for complex medical QA tasks.
- By dynamically adjusting its reasoning strategy based on the input question and incorporating pre-trained structural embeddings of triplets, KGARevion offers a nuanced and contextualized approach to answering both multi-choice and open-ended questions in the medical domain.

## Summary

, , , , 


Biomedical knowledge is known for its complexity and structured nature, requiring unique reasoning strategies compared to other scientific disciplines like physics or chemistry. Biomedical scientists utilize various reasoning approaches, including rule-based, prototype-based, and case-based reasoning, to navigate the intricate web of information within the field. To address these challenges, a new knowledge graph (KG) based agent called KGARevion has been introduced. This agent leverages the knowledge base of LLM to generate relevant triplets in response to queries and then verifies these triplets against a grounded KG to ensure accuracy and relevance in the final answer. is complex and requires unique reasoning strategies compared to other scientific disciplines like physics or chemistry. Biomedical scientists use rule-based, prototype-based, and case-based reasoning to navigate this complexity. A new KG-based agent called KGARevion has been introduced to address these challenges by leveraging LLM's knowledge base and verifying generated triplets against a grounded KG for accuracy. Unlike traditional models that may struggle with nuanced medical concepts due to their reliance on external databases with potentially incomplete or incorrect information, KGARevion's multi-step process ensures robust reasoning while adapting to different models of medical reasoning. Evaluations on gold-standard medical QA datasets have shown that KGARevion significantly improves accuracy by over 5.2% when compared to 15 other models in handling complex medical questions. Additionally, three new medical QA datasets with varying levels of semantic complexity were curated to test KGARevion's capabilities, where it achieved a remarkable 10.4% improvement in accuracy. The challenges faced by existing language model-based methods in handling complex medical reasoning have led researchers to explore alternative approaches such as retrieval-augmented generation (RAG). However, these methods are still limited by their dependence on external sources of information and lack post-retrieval verification mechanisms, making them susceptible to errors. have emerged as a promising solution by providing precise and specialized in-domain knowledge for medical QA models. Still, existing KGs may be incomplete and fail to capture complex associations between multiple medical concepts simultaneously. In response to these challenges, KGARevion represents a novel approach that combines non-codified knowledge from language models with structured codified knowledge from KGs for complex medical QA tasks. By dynamically adjusting its reasoning strategy based on the input question and incorporating pre-trained structural embeddings of triplets for fine-tuning purposes, KGARevion offers a more nuanced and contextualized approach to answering both multi-choice and open-ended questions in the medical domain. Overall, KGARevion's adaptive reasoning capabilities have demonstrated significant improvements in accuracy across various datasets and hold promise for advancing knowledge-intensive medical QA systems through enhanced integration of diverse sources of information and rigorous verification processes.