# Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines

**Authors:** David Oniani, Xizhi Wu, Shyam Visweswaran, Sumit Kapoor, Shravan Kooragayalu, Katelyn Polanska, Yanshan Wang  
**arXiv:** [2401.11120](https://arxiv.org/abs/2401.11120)  
**Date:** 2024-01-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.11120v1/

## Key points

- Study explores integration of Clinical Practice Guidelines (CPGs) into Large Language Models (LLMs) to enhance Clinical Decision Support (CDS)
- Three novel methods introduced for incorporating CPGs into LLMs: Binary Decision Tree (BDT), Program-Aided Graph Construction (PAGC), and Chain-of-Thought-Few-Shot Prompting (CoT-FSP)
- Research evaluates responses of four LLMs—GPT-4, GPT-3.5 Turbo, LLaMA, and PaLM 2—augmented with CPGs compared to Zero-Shot Prompting (ZSP) baseline
- BDT method outperforms CoT-FSP and PAGC in automatic evaluation metrics
- All proposed methods demonstrate high performance levels in enhancing recommendations for COVID-19 outpatient treatment
- Enriching LLMs with CPGs improves accuracy in providing treatment recommendations compared to plain LLMs with ZSP
- Study contributes valuable insights on effective integration of CPGs into LLMs for enhancing CDS capabilities
- Developed methods hold promise for improving decision-making processes in clinical settings and advancing AI technologies for healthcare applications

## Summary

In the study titled "Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines," authors David Oniani, Xizhi Wu, Shyam Visweswaran, Sumit Kapoor, Shravan Kooragayalu, Katelyn Polanska, and Yanshan Wang explore the integration of Clinical Practice Guidelines (CPGs) into Large Language Models (LLMs) to improve Clinical Decision Support (CDS). The researchers introduce three novel methods for incorporating CPGs into LLMs: Binary Decision Tree (BDT), Program-Aided Graph Construction (PAGC), and Chain-of-Thought-Few-Shot Prompting (CoT-FSP). To assess the efficacy of these methods, the team generates synthetic patient descriptions and evaluates the responses generated by four LLMs—GPT-4, GPT-3.5 Turbo, LLaMA, and PaLM 2—using Zero-Shot Prompting (ZSP) as a baseline. <br>
 show that all four LLMs exhibit enhanced performance when augmented with CPGs compared to ZSP. In particular, BDT outperforms CoT-FSP and PAGC in automatic evaluation metrics. Furthermore, indicate that all proposed methods achieve high performance levels. The findings suggest that LLMs enriched with CPGs offer superior accuracy in providing recommendations for COVID-19 outpatient treatment compared to plain LLMs with ZSP. This research highlights the potential for broader applications beyond the specific case study of COVID-19 outpatient treatment.<br>
Overall, contributes valuable insights into the effective integration of CPGs into LLMs for enhancing CDS capabilities. The developed methods show promise in improving decision-making processes in clinical settings and pave the way for further advancements in leveraging AI technologies for healthcare applications.