# MarkLLM: An Open-Source Toolkit for LLM Watermarking

**Authors:** Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, Philip S. Yu  
**arXiv:** [2405.10051](https://arxiv.org/abs/2405.10051)  
**Date:** 2024-05-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.10051v4/

## Key points

- Large Language Models (LLMs) like ChatGPT, GPT-4, and LLaMA have revolutionized tasks such as information retrieval, content comprehension, and creative writing.
- Concerns have been raised about impersonation, academic paper ghostwriting, and the spread of fake news due to the proficiency of LLMs in generating high-quality text.
- Watermarking has emerged as a promising solution to ensure the authenticity of digital communication.
- An open-source toolkit called MarkLLM has been introduced to address these issues and support researchers in implementing LLM watermarking algorithms from families like KGW and Christ.
- MarkLLM provides user-friendly interfaces for loading algorithms, generating watermarked text from LLMs, conducting detection processes, and visualizing algorithm mechanisms.
- The toolkit includes 12 evaluation tools addressing detectability, robustness, and impact on text quality perspectives.
- The KGW method within MarkLLM involves partitioning vocabulary sets into green and red lists based on preceding tokens to bias logits towards green list tokens during text generation.
- Various modifications have been proposed to enhance performance through list partitioning or logit manipulation.
- MarkLLM aims to support researchers in understanding LLM watermarking technology while involving the general public in advancements in research and application.

## Summary

In the digital era, Large Language Models (LLMs) like ChatGPT, GPT-4, and LLaMA have revolutionized various tasks such as information retrieval, content comprehension, and creative writing. However, the proficiency of LLMs in generating high-quality text has raised concerns about impersonation, academic paper ghostwriting, and the spread of fake news. To address these issues and ensure the authenticity of digital communication,  has emerged as a promising solution. The complexity of watermarking algorithms, evaluation procedures, and perspectives poses challenges for researchers and the community to easily experiment with and assess advancements in this field. In response to this need for consensus and support within and outside the field, an open-source toolkit called  has been introduced. This toolkit offers a unified framework for implementing LLM watermarking algorithms from families like KGW and Christ. From a functional perspective, provides user-friendly interfaces for loading algorithms, generating watermarked text from LLMs, conducting detection processes,and visualizing algorithm mechanisms. It also includes 12 evaluation tools addressing detectability,< kd >robustness,< /kd >and impact on text quality perspectives. Additionally,< kd >MarkLLM </ kd >features automated evaluation pipelines that allow customization of datasets,
models,
metrics,
and attacks
for flexible assessments. The KGW method within < kd >MarkLLM </ kd >
involves partitioning vocabulary sets into green
and red lists based on preceding tokens
to bias logits towards green list tokens during text generation. Various modifications have been proposed to enhance performance through list partitioning or logit manipulation. The toolkit also supports visualization solutions for both major watermarking algorithm families. Overall, < kd >MarkLLM</kd> aims to support researchers in understanding LLM watermarking technology while involving the general public in advancements in research and application. By providing a comprehensive suite of tools for implementation framework development visualization solutions evaluation modules across different perspectives,< kd >MarkLLM</kd> seeks to foster consensus drive further progress in this critical area of study.