# The Go Transformer: Natural Language Modeling for Game Play

**Authors:** Matthew Ciolino, David Noever, Josh Kalin  
**arXiv:** [2007.03500](https://arxiv.org/abs/2007.03500)  
**Date:** 2020-07-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2007.03500v3/

## Key points

- Authors Matthew Ciolino, David Noever, and Josh Kalin explore natural language modeling for generating strategic moves in the game of Go
- They use Generative Pretrained Transformer (GPT-2) to emulate the style of Go champions from Smart Game Format (SGF)
- GPT-2 maintains punctuation and spacing in generated text, allowing seamless integration with game visualization tools
- Sabaki project's game engine leverages auto-replays based on raw output from the text generator
- Language modeling effectively captures sequencing format and strategic formations in championship-level Go games
- Fine-tuning GPT-2 reveals more efficient opening move sequences that prioritize corner play over less advantageous center and side plays
- Potential for applying similar approaches to over 40 other board games using historical text annotations as training data
- "The Go Transformer" demonstrates how natural language modeling can enhance gameplay strategies in traditional board games like Go and beyond

## Summary

In their paper titled "The Go Transformer: Natural Language Modeling for Game Play," authors Matthew Ciolino, David Noever, and Josh Kalin explore the use of natural language modeling in generating strategic moves for the ancient game of Go. They utilize the Generative Pretrained Transformer (GPT-2) to emulate the style of Go champions as documented in Smart Game Format (SGF). This provides textual descriptions of move sequences which are used to train the model and produce valid yet previously unseen strategies for playing Go. One key advantage of using GPT-2 is its ability to maintain punctuation and spacing in generated text. This allows for seamless integration with game visualization tools and creative patterns. For example, the Sabaki project's game engine leverages auto-replays based on the raw output from the text generator. The results of their study demonstrate that language modeling can effectively capture both the sequencing format found in championship-level Go games and the strategic formations employed by skilled players. Additionally, compared to randomly generated game boards, fine-tuning GPT-2 reveals more efficient opening move sequences that prioritize corner play over less advantageous center and side plays. This suggests that leveraging language modeling techniques can lead to improved gameplay strategies in Go. Furthermore, this research opens up new possibilities for applying similar approaches to over 40 other board games where historical text annotations serve as valuable training data. Games like Amazons and Connect 4/6 could potentially benefit from this novel method of game generation through language modeling. Overall, "The Go Transformer" presents a compelling case for how natural language modeling can enhance gameplay strategies in traditional board games like Go while also paving the way for innovative applications across a broader spectrum of gaming experiences.