# WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences

**Authors:** Xiao Liu, Hanyu Lai, Hao Yu, Yifan Xu, Aohan Zeng, Zhengxiao Du, Peng Zhang, Yuxiao Dong, Jie Tang  
**arXiv:** [2306.07906](https://arxiv.org/abs/2306.07906)  
**Date:** 2023-06-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.07906v1/

## Key points

- WebGLM is a web-enhanced question-answering system that aims to augment pre-trained large language models (LLMs) with web search and retrieval capabilities.
- The system is based on the General Language Model (GLM) and is designed with strategies for the LLM-augmented retriever, bootstrapped generator, and human preference-aware scorer.
- The goal of WebGLM is to provide an effective and cost-effective solution for constructing long-formed QA datasets with open-world references, which are difficult to create due to the need for expert-level annotations.
- Most Open QA datasets and models are limited to answering short answer phrases, while people usually prefer more informative long-formed answers with references.
- WebGLM was developed with accuracy, efficiency, and cost-effectiveness advantages over existing systems like WebGPT.
- Results suggest that WebGLM designs outperform existing systems in terms of accuracy.
- Despite the bottleneck in fetching each page from different sources during retrieval, WebGLM's retrieval efficiency was found to be far better than that of WebGPT.
- Overall; WebGLM has demonstrated comparable answer quality performance compared to similar sized WebGPT models (13B & 175B) in human evaluation; while being more efficient & cost effective for real world deployments.

## Summary

WebGLM is a web-enhanced question-answering system that aims to augment pre-trained large language models (LLMs) with web search and retrieval capabilities, while being efficient for real-world deployments. The system is based on the General Language Model (GLM) and is designed with strategies for the LLM-augmented retriever, bootstrapped generator, and human preference-aware scorer. The goal of WebGLM is to provide an effective and cost-effective solution for the challenge of constructing long-formed QA datasets with open-world references, which are difficult to create due to the need for expert-level annotations. Most Open QA datasets and models are limited to answering short answer phrases, while people usually prefer more informative long-formed answers with references. Recent attempts at creating such datasets include ELI5 and WebGPT. However, these methods require considerable expenses, time, and training as they rely on abundant expert-level annotations of browsing trajectories, well-written answers, and answer preference labeling. In addition, the behavior cloning method used in WebGPT requires its base model GPT-3 to emulate human experts by instructing the system to interact with a web browser. To address these limitations, WebGLM was developed with accuracy, efficiency, and cost-effectiveness advantages over existing systems like WebGPT. The proposed systematic criteria for evaluating web-enhanced QA systems were evaluated through multi-dimensional human evaluation and quantitative ablation studies. Results suggest that WebGLM designs outperform existing systems in terms of accuracy. The speed analysis conducted on WebGLM's LLM-augmented retriever revealed that retrieval is the most time consuming part of any web scale QA system. However despite this bottleneck in fetching each page from different sources during retrieval; WebGLM's retrieval efficiency was found to be far better than that of WebGPT. Overall; WebGLM has demonstrated comparable answer quality performance compared to similar sized WebGPT models (13B & 175B) in human evaluation; while being more efficient & cost effective for real world deployments.