# Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering -- Example of ChatGPT

**Authors:** Maciej P. Polak, Dane Morgan  
**arXiv:** [2303.05352](https://arxiv.org/abs/2303.05352)  
**Date:** 2023-03-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.05352v1/

## Key points

- Authors Maciej P. Polak and Dane Morgan discuss automating data extraction from research papers using NLP, LMs, and LLMs.
- Introduction of ChatExtract method to address challenges like initial investment in effort, expertise, and coding.
- ChatExtract leverages conversational LLMs with engineered prompts for accurate data extraction with minimal upfront effort.
- Sentences containing relevant data are identified, extracted, and verified for accuracy through follow-up questions.
- ChatExtract consistently delivers high-quality data extraction results with precision and recall rates exceeding 90% in materials data tests.
- Exceptional performance attributed to information retention capabilities of conversational models, purposeful redundancy, and introduction of uncertainty through follow-up prompts.
- Approaches like ChatExtract are expected to replace traditional methods due to simplicity, transferability, and high accuracy levels.
- Automated techniques like ChatExtract anticipated to play a significant role in streamlining data extraction processes in the future.

## Summary

In their paper titled "Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering -- Example of ChatGPT," authors Maciej P. Polak and Dane Morgan discuss the growing trend towards automating data extraction from research papers using natural language processing (NLP), language models (LMs), and large language models (LLMs). The authors introduce the ChatExtract method as a solution to the challenges associated with these methods, such as requiring a substantial initial investment in terms of effort, expertise, and coding. This approach leverages advanced conversational LLMs to automate accurate data extraction with minimal upfront effort or background knowledge. By utilizing engineered prompts applied to a conversational LLM, sentences containing relevant data can be identified, extracted, and verified for accuracy through follow-up questions. This effectively addresses the issue of inaccurate responses often seen in LLMs. The results demonstrate that ChatExtract consistently delivers high-quality data extraction results when implemented with various conversational LLMs. In tests focusing on materials data, precision and recall rates exceeding 90% were achieved with the best conversational LLMs, potentially surpassing human accuracy levels in many cases. The exceptional performance of ChatExtract is attributed to the information retention capabilities of conversational models, purposeful redundancy, and introduction of uncertainty through follow-up prompts. Overall, this suggests that approaches like ChatExtract are poised to replace traditional methods of data extraction due to their simplicity, transferability, and high accuracy levels. The authors anticipate that automated techniques will play a significant role in streamlining data extraction processes in the near future.