# Knowing When to Ask -- Bridging Large Language Models and Data

**Authors:** Prashanth Radhakrishnan, Jennifer Chen, Bo Xu, Prem Ramaswami, Hannah Pho, Adriana Olmos, James Manyika, R. V. Guha  
**arXiv:** [2409.13741](https://arxiv.org/abs/2409.13741)  
**Date:** 2024-09-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2409.13741v1/

## Key points

- Integration of Large Language Models (LLMs) with Data Commons
- Two primary methods explored: Retrieval Interleaved Generation (RIG) and Retrieval Augmented Generation (RAG)
- RIG trains LLMs to generate natural language queries for data retrieval
- RAG fetches relevant data tables to augment the LLM's prompt
- Addressing challenges of diverse schemas and formats in public statistical data
- Emphasis on context in interpreting information
- Utilization of existing techniques like Toolformer for self-supervised learning
- Application of RIG and RAG in enhancing LLMs
- RIG trains LLMs to ask and retrieve statistics in natural language without structured questions
- RAG grants access to external knowledge sources for more comprehensive outputs
- Combining Data Commons' Knowledge Graphs with RAG for accurate responses
- Achieving compelling results in generating informative responses
- Functionality of Data Commons as a collection of interoperable Knowledge Graphs with standardized data and schema
- Enables seamless exploration of diverse datasets using a Natural Language interface
- Reference to LIMA approach for better alignment with user preferences

## Summary

This paper focuses on improving the accuracy of Large Language Models (LLMs) by integrating them with Data Commons. The authors explore two primary methods: Retrieval Interleaved Generation (RIG) and Retrieval Augmented Generation (RAG). RIG trains LLMs to generate natural language queries for retrieving data from Data Commons, while RAG fetches relevant data tables to augment the LLM's prompt. The study addresses challenges posed by diverse schemas and formats of public statistical data and emphasizes the importance of context in interpreting such information. It builds upon existing techniques like Toolformer, which enables LLMs to leverage external tools through self-supervised learning. RIG is an application of Toolformer that trains LLMs to ask and retrieve statistics in natural language without structured questions. On the other hand, RAG enhances language models by granting access to external knowledge sources, leading to more comprehensive outputs. By combining Data Commons' Knowledge Graphs with RAG, the study achieves compelling results in generating accurate and informative responses. Furthermore, the paper discusses how Data Commons functions as a collection of interoperable Knowledge Graphs with standardized data and schema, enabling seamless exploration of diverse datasets using a Natural Language interface. The study also references LIMA, an approach that uses limited examples for better alignment with user preferences. Overall, this work represents a significant step towards developing trustworthy and reliable LLMs grounded in verifiable statistical data for complex factual reasoning tasks.