# PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs

**Authors:** Jiuzhou Han, Nigel Collier, Wray Buntine, Ehsan Shareghi  
**arXiv:** [2305.12392](https://arxiv.org/abs/2305.12392)  
**Date:** 2023-05-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.12392v1/

## Key points

- The paper introduces a framework called Prompting with Iterative Verification (PiVe) to enhance the graph-based generative capability of large language models (LLMs).
- PiVe addresses the limitation of LLMs not being well-equipped for tasks involving structured data generation.
- PiVe trains a small language model as a verifier module for the output of an LLM, enabling more accurate text-to-graph generation and iterative corrections offline.
- Experimental results on three graph-based datasets consistently show improvement achieved through PiVe.
- The proposed verifier module serves as a valuable data augmentation tool to enhance the quality of automatically generated parallel text-graph datasets.
- Challenges associated with creating high-quality parallel text and graph pairs or pre-training protocols due to limited data availability are discussed.
- GenWiki is highlighted as an automatically constructed dataset containing non-parallel text and graphs with shared content, which can only be used for unsupervised training.
- The verifier module introduced in PiVe naturally serves as a data augmentation tool to improve overlap between text and graph in automatically constructed datasets.
- In conclusion, PiVe offers a promising framework to enhance the graph-based generative capability of LLMs by incorporating iterative verification. It improves performance and provides a cost-effective solution for text-to-graph generation tasks while serving as a valuable data augmentation tool for enhancing automatically generated parallel text-graph datasets.

## Summary

The paper introduces a framework called Prompting with Iterative Verification (PiVe) to enhance the graph-based generative capability of large language models (LLMs). PiVe addresses the limitation of LLMs not being well-equipped for tasks involving structured data generation by training a small language model as a verifier module for the output of an LLM. This approach enables more accurate text-to-graph generation and can apply iterative corrections offline. Experimental results on three graph-based datasets consistently show improvement achieved through PiVe. Furthermore, the proposed verifier module serves as a valuable data augmentation tool to enhance the quality of automatically generated parallel text-graph datasets. The authors also discuss the challenges associated with creating high-quality parallel text and graph pairs or pre-training protocols due to limited data availability. They highlight GenWiki, an automatically constructed dataset containing non-parallel text and graphs with shared content which can only be used for unsupervised training. The verifier module introduced in PiVe naturally serves as a data augmentation tool to improve overlap between text and graph in automatically constructed datasets. In conclusion, PiVe offers a promising framework to enhance the graph-based generative capability of LLMs by incorporating iterative verification. It not only improves performance but also provides a cost-effective solution for text-to-graph generation tasks while serving as a valuable data augmentation tool for enhancing automatically generated parallel text-graph datasets.