# Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations

**Authors:** Yiming Li, Xueqing Peng, Jianfu Li, Xu Zuo, Suyuan Peng, Donghong Pei, Cui Tao, Hua Xu, Na Hong  
**arXiv:** [2404.05415](https://arxiv.org/abs/2404.05415)  
**Date:** 2024-04-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.05415v2/

## Key points

- Precise identification of acupoints is crucial for effective acupuncture therapy
- Large language models (LLMs) like GPT offer an opportunity to extract valuable information on acupoint locations from textual knowledge sources
- Study compares performance of GPT with traditional deep learning models in extracting location-related relations pertaining to acupoints
- Research methodology involves using the WHO Standard Acupuncture Point Locations corpus, containing detailed descriptions of 361 acupoints
- Five distinct types of relations were identified through manual annotation: 'direction_of,' 'distance_of,' 'part_of,' 'near_acupoint,' and 'located_near'
- Fine-tuned GPT-3.5 consistently outperformed other models with a micro-average F1 score of 0.92 across all relation types studied
- Superior performance of fine-tuned LLMs like GPT in accurately extracting complex relations related to acupoint locations
- Practical applications in enhancing understanding and modeling of acupuncture knowledge, promoting standard implementation practices in acupuncture training and clinical settings
- Potential for large language models like GPT to revolutionize natural language processing tasks in specialized fields such as acupuncture therapy

## Summary

The precise identification of acupoints is crucial for effective acupuncture therapy. With the advanced language understanding capabilities of large language models (LLMs) like Generative Pre-trained Transformers (GPT), there is an opportunity to extract valuable information on acupoint locations from textual knowledge sources. This study compares the performance of GPT with traditional deep learning models such as Long Short-Term Memory (LSTM) and Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT) in extracting location-related relations pertaining to acupoints. The focus is on evaluating how pretraining and fine-tuning impact GPT's ability to accurately extract these relations. The research methodology involves using the World Health Organization Standard Acupuncture Point Locations in the Western Pacific Region (WHO Standard) as the corpus, which contains detailed descriptions of 361 acupoints. Five distinct types of relations ('direction_of,' 'distance_of,' 'part_of,' 'near_acupoint,' and 'located_near') were identified through manual annotation, totaling 3,174 annotated instances. Five different models were tested: BioBERT, LSTM, pre-trained GPT-3.5, fine-tuned GPT-3.5, and pre-trained GPT-4. Performance evaluation metrics included micro-average exact match precision, recall rates, and F1 scores across all relation types. Results showed that fine-tuned GPT-3.5 consistently outperformed other models with a micro-average F1 score of 0.92 across all relation types studied. This highlights the superior performance of fine-tuned LLMs like GPT in accurately extracting complex relations related to acupoint locations. These findings have practical applications in enhancing our understanding and modeling of acupuncture knowledge and promoting standard implementation practices in acupuncture training and clinical settings. This study also underscores the potential for large language models like GPT to revolutionize natural language processing tasks in specialized fields such as acupuncture therapy, offering promising avenues for further exploration and utilization in diverse healthcare contexts where accurate information extraction is crucial for informed decision-making processes.