# Beyond Code Generation: LLM-supported Exploration of the Program Design Space

**Authors:** J. D. Zamfirescu-Pereira, Eunice Jun, Michael Terry, Qian Yang, Björn Hartmann  
**arXiv:** [2503.06911](https://arxiv.org/abs/2503.06911)  
**Date:** 2025-03-10  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2503.06911v1/

## Key points

- Authors explore explicit Large Language Model (LLM)-powered support for iterative program design
- Program design involves navigating space of alternative problem formulations and solutions iteratively
- Introduction of Integrated Development Environment (IDE) to facilitate program design by generating new problem framing approaches and alternative solutions
- IDE helps track design decisions and identify implicit choices made by programmers or LLMs
- User study shows users combining and parallelizing design phases to explore wider design space with IDE
- Challenges include keeping up with changes from LLMs and managing information overload
- Fundamental challenge highlighted for future IDEs: effectively managing attention and determining when agents should surface information to designers
- Research contributes insights into enhancing efficiency and effectiveness of program design processes through innovative IDE tools tailored for collaboration between human designers and AI-powered systems

## Summary

In their work titled "Beyond Code Generation: LLM-supported Exploration of the Program Design Space," authors J. D. Zamfirescu-Pereira, Eunice Jun, Michael Terry, Qian Yang, and Björn Hartmann delve into the realm of explicit Large Language Model (LLM)-powered support for the iterative design of computer programs. They highlight how program design involves navigating a space of alternative problem formulations and solutions in an iterative manner. The authors introduce an Integrated Development Environment (IDE) that facilitates program design by generating and presenting new problem framing approaches alongside alternative solutions. This IDE also helps track design decisions and identify implicit choices made by programmers or LLMs. Through a user study, they observed that users utilizing their IDE were able to combine and parallelize design phases to explore a wider design space. However, they also encountered challenges in keeping up with changes originating from LLMs and managing information overload. These findings underscore a fundamental challenge for future IDEs aiming to support program design through higher-level instructions provided to LLM-based agents: effectively managing attention and determining when agents should surface information to designers. The research contributes valuable insights into enhancing the efficiency and effectiveness of program design processes through innovative IDE tools tailored for collaboration between human designers and AI-powered systems.