# TnT-LLM: Text Mining at Scale with Large Language Models

**Authors:** Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W White, Longqi Yang, Reid Andersen, Georg Buscher, Dhruv Joshi, Nagu Rangan  
**arXiv:** [2403.12173](https://arxiv.org/abs/2403.12173)  
**Date:** 2024-03-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.12173v1/

## Key points

- The paper focuses on utilizing Large Language Models (LLMs) to transform unstructured text into structured and meaningful forms.
- The TnT-LLM framework consists of two phases: a zero-shot multi-stage reasoning approach for producing and refining label taxonomies using LLMs, and utilizing LLMs as data labelers to generate training samples for building lightweight supervised classifiers.
- Application of TnT-LLM in analyzing user intent and conversational domain for Bing Copilot demonstrates superior performance in generating relevant label taxonomies compared to state-of-the-art baselines.
- The framework strikes a favorable balance between accuracy and efficiency in classification at scale.
- Practical experiences and insights regarding challenges and opportunities associated with using LLMs for large-scale text mining in real-world applications are shared by the authors.

## Summary

, , , , 

The paper "TnT-LLM: Text Mining at Scale with Large Language Models" by Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W White, Longqi Yang, Reid Andersen, Georg Buscher, Dhruv Joshi and Nagu Rangan focuses on utilizing Large Language Models (LLMs) to transform unstructured text into structured and meaningful forms. This approach aims to automate the process of generating label taxonomies and assigning labels in a more efficient manner without heavy reliance on domain expertise and manual curation. The proposed TnT-LLM framework consists of two phases - a zero-shot multi-stage reasoning approach for producing and refining label taxonomies using LLMs and utilizing LLMs as data labelers to generate training samples for building lightweight supervised classifiers. The application of TnT-LLM is demonstrated in analyzing user intent and conversational domain for Bing Copilot (formerly Bing Chat), an open-domain chat-based search engine. Through extensive experiments using both human evaluation metrics and automatic evaluation metrics, it is shown that TnT-LLM outperforms state-of-the-art baselines in accurately generating relevant label taxonomies. The framework strikes a favorable balance between accuracy and efficiency in classification at scale. Additionally, the authors share practical experiences and insights regarding the challenges and opportunities associated with using LLMs for large-scale text mining in real-world applications. Overall, this paper contributes towards advancing text mining techniques by leveraging LLMs to efficiently structure unstructured text data into meaningful categories.