# Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering

**Authors:** Maciej P. Polak, Dane Morgan  
**arXiv:** [2303.05352](https://arxiv.org/abs/2303.05352)  
**Date:** 2023-03-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.05352v2/

## Key points

- Growing interest in automating data extraction from research papers using natural language processing and language models
- Proposed method called ChatExtract that leverages conversational language models for accurate data extraction with minimal effort and background knowledge
- Utilizes engineered prompts to identify relevant data sentences and extract information
- Includes follow-up questions to ensure correctness of extracted information
- Overcomes challenges associated with large language models providing inaccurate responses
- Achieves high-quality data extraction results with precision and recall rates close to 90%
- Information retention capabilities of conversational models combined with purposeful redundancy and introducing uncertainty through follow-up prompts contribute to exceptional performance
- Similar approaches likely to become powerful tools for automated data extraction in the future due to simplicity, transferability, and accuracy
- Application example: databases developed for critical cooling rates of metallic glasses and yield strengths of high entropy alloys using ChatExtract
- Innovative method for automating accurate data extraction from research papers using conversational language models and prompt engineering has potential as an effective tool for extracting valuable information from scientific literature in various domains.

## Summary

In recent years, there has been a growing interest in automating the extraction of data from research papers using natural language processing and language models. To address this issue, the authors propose a new method called ChatExtract that leverages advanced conversational language models to automate accurate data extraction with minimal initial effort and background knowledge. ChatExtract utilizes a set of engineered prompts applied to a conversational language model. These prompts help identify sentences containing relevant data and extract that data. Additionally, the method includes a series of follow-up questions to ensure the correctness of the extracted information. This approach overcomes known challenges associated with large language models providing inaccurate responses. The authors demonstrate that ChatExtract can be applied with any conversational language model and achieves high-quality data extraction results with precision and recall rates close to 90%. The exceptional performance is attributed to the information retention capabilities of conversational models combined with purposeful redundancy and introducing uncertainty through follow-up prompts. The study suggests that approaches similar to ChatExtract are likely to become powerful tools for automated data extraction in the future due to their simplicity, transferability, and accuracy. As an application example, databases for critical cooling rates of metallic glasses and yield strengths of high entropy alloys are developed using ChatExtract. Overall, this research presents an innovative method for automating accurate data extraction from research papers using conversational language models and prompt engineering which highlights its potential as an effective tool for extracting valuable information from scientific literature in various domains.