# Creative Problem Solving in Large Language and Vision Models -- What Would it Take?

**Authors:** Lakshmi Nair, Evana Gizzi, Jivko Sinapov  
**arXiv:** [2405.01453](https://arxiv.org/abs/2405.01453)  
**Date:** 2024-05-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.01453v1/

## Key points

- Authors Lakshmi Nair, Evana Gizzi, and Jivko Sinapov explore integrating Computational Creativity (CC) with large language and vision models (LLVMs) for creative problem solving.
- Preliminary experiments show applying CC principles through augmented prompting can enhance LLVMs' capabilities in creative problem solving.
- Task re-representation through improved prompting is a key concept that can enhance LLVMs' creative problem-solving abilities.
- Multi-modal prompting capabilities, including diverse data types like images or spectral data, are highlighted as beneficial for creative problem solving in LLVMs.
- Insights from Computational Creativity can aid in developing meaningful representations across different modalities to improve task performance.
- Understanding the significance of object material or shape for specific tasks can guide model development towards more effective solutions.
- The need for further exploration into integrating creativity into machine learning models to enhance task performance and general intelligence is emphasized.

## Summary

In their paper titled "Creative Problem Solving in Large Language and Vision Models -- What Would it Take? ", authors Lakshmi Nair, Evana Gizzi, and Jivko Sinapov explore the integration of Computational Creativity (CC) with research in large language and vision models (LLVMs) to address a key limitation of these models: creative problem solving. Through preliminary experiments showcasing how CC principles can be applied to overcome this limitation via augmented prompting, the authors aim to stimulate discussions on Computational Creativity within the realm of machine learning algorithms for creative problem solving in LLVMs. The experiments conducted by Nair et al. demonstrate promising results that suggest extending principles of Computational Creativity could effectively enhance LLVMs' capabilities in creative problem solving. Particularly noteworthy is the concept of task re-representation through improved prompting, which warrants further exploration regarding the automatic generation of prompts based on the nature of the creative task at hand. Moreover, the authors emphasize the potential benefits of multi-modal prompting capabilities in achieving creative problem solving within LLVMs. They highlight the challenges associated with describing affordances solely through words and propose leveraging diverse data types such as images or spectral data for material properties to enrich prompt generation. This necessitates the utilization of multi-modal LLVMs capable of processing various data modalities effectively. Furthermore, Nair and colleagues underscore that insights from Computational Creativity can inform meaningful representations across different modalities, aiding in achieving creative problem-solving tasks. They posit that understanding whether object material or shape holds more significance for specific tasks can guide model development towards more effective solutions. Lastly, it is important to note that the examples of creative problem solving presented in their experiments are human-centric, underscoring the need for further exploration into how creativity can be integrated into machine learning models to enhance both task performance and general intelligence. Through their innovative approach, Nair et al. pave the way for future research endeavors aimed at bridging computational creativity with large language and vision models for enhanced problem-solving capabilities.