# Jigsaw: Large Language Models meet Program Synthesis

**Authors:** Naman Jain, Skanda Vaidyanath, Arun Iyer, Nagarajan Natarajan, Suresh Parthasarathy, Sriram Rajamani, Rahul Sharma  
**arXiv:** [2112.02969](https://arxiv.org/abs/2112.02969)  
**Date:** 2021-12-06  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2112.02969v1/

## Key points

- Authors discuss the intersection of large pre-trained language models (PTLMs) and program synthesis
- Potential benefits and risks associated with PTLMs in code generation
- Proposal to augment PTLMs with post-processing steps based on program analysis and synthesis techniques
- Importance of incorporating user feedback to enhance accuracy of code synthesis systems
- Introduction of a tool called Jigsaw for synthesizing code using Python Pandas API through multi-modal inputs
- Jigsaw functions as a multi-modal interactive code synthesis platform with user-friendly interface for clarifying intent specifications and providing feedback
- Challenges encountered when using general-purpose PTLMs for specific domains
- Emphasis on user engagement and refinement processes in bridging the gap between PTLM capabilities and domain-specific requirements

## Summary

In their paper titled "Jigsaw: Large Language Models meet Program Synthesis," authors Naman Jain, Skanda Vaidyanath, Arun Iyer, Nagarajan Natarajan, Suresh Parthasarathy, Sriram Rajamani, and Rahul Sharma discuss the intersection of large pre-trained language models (PTLMs) such as GPT-3, Codex, and Google's language model with program synthesis. They acknowledge both the potential benefits and risks associated with these developments. While PTLMs have the capacity to enhance productivity by serving as automated AI pair programmers for developers worldwide, there is a concern regarding the quality of code generated since these models lack an understanding of program semantics. To address this issue, the authors propose augmenting PTLMs with post-processing steps based on program analysis and synthesis techniques that comprehend both syntax and semantics. They demonstrate how incorporating user feedback can further enhance the accuracy of code synthesis systems. The team presents their experiences in building and evaluating a tool called Jigsaw , which focuses on synthesizing code for utilizing Python Pandas API through multi-modal inputs. Through their experiments, they emphasize the crucial role Jigsaw plays in refining PTLM-based code synthesis systems. Jigsaw functions as a multi-modal interactive code synthesis platform where users can specify intent using natural language descriptions and test cases (input-output examples). The system features a user-friendly interface integrated within programming environments to facilitate seamless interaction. This interactive component enables developers to clarify ambiguous intent specifications while providing valuable feedback for system improvement. The authors highlight challenges encountered when employing general-purpose PTLMs for specific domains through example queries  By emphasizing user engagement and refinement processes within their design principles, Jigsaw aims to bridge the gap between PTLM capabilities and domain-specific requirements effectively.