Can ChatGPT really be considered creative?

Language plays an important role in our lives and that is why NLP(Natural Language Processing) has been a very important branch of AI. NLP involves using computational methods to analyze and understand human language. This includes tasks such as text classification, sentiment analysis, machine translation, speech recognition, and chatbot development. AI has made significant achievements in NLP, improving our ability to understand, analyze, and generate natural language. Since the invention of transformers, we have witnessed a very fast acceleration of the pace of development in the past decade. And now with the evolution of transformers from BERT to GPT the possibility of exploiting large-scale data sets has led to the definition of the so-called foundation models, which are able to achieve state-of-the-art performance in a variety of tasks.

LLMs have captivated the imagination of millions of people, also thanks to a series of entertaining demonstrations and open tools released to the public. The examples are many from poetry or storytelling to culinary recipes and the results are often remarkable.

Notwithstanding, it is not obvious whether these “machines” are truly creative, at least in the sense originally discussed by Ada Lovelace (Menabrea and Lovelace, 1843). LLMs have already been analyzed (and sometimes criticized) from different perspectives, e.g., fairness (Bender et al., 2021), concept understanding (Bender and Koller, 2020), societal impact (Tamkin et al., 2021), and anthropomorphism (Shanahan, 2022) just to name a few. However, a critical question has not been considered yet: can LLMs be considered creative?’’

Margaret Boden defines creativity as “the ability to come up with ideas or artifacts that are new, surprising and valuable” (Boden, 2003). In other words, Boden implicitly derives criteria that can be used to identify a creative product. They suggest that creativity is about novelty, surprise and value. We will refer to them as Boden’s three criteria. We will analyze to what extent state-of-the-art LLMs satisfy them and we will question if LLMs can be really considered creative.

Value refers to utility, performance, and attractiveness (Maher, 2010). It is also related to both the quality of the output, and its acceptance by the society. Due to the large impact LLMs are already having (Bommasani et al., 2021) and the quality of outputs of the systems based on them (Stevenson et al., 2022), it is possible to argue that the artifacts produced by them are indeed valuable.

Novelty refers to the dissimilarity between the produced artifact and other examples in its class (Ritchie, 2007). However, it can also be seen as the property of not being in existence before. This is considered in reference to either the person who comes up with it or the entire human history. The former is referred to as psychological creativity (shortened as P-creativity), whereas the latter as historical creativity (shortened as H-creativity) (Boden, 2003). While the difference appears negligible, it is substantial when discussing LLMs in general. Considering these definitions, a model writing a text that is not in its training set would be considered as P-novel, but possibly also H-novel, since LLMs are commonly trained on all available data. Their stochastic nature and the variety of prompts that are usually provided commonly lead to novel outcomes (McCoy et al., 2021); LLMs may therefore be capable of generating artifacts that are also new. However, one should remember how such models learn and generate. Even if prompted with the sentence “I wrote a new poem this morning:”, they would complete it with what is most likely to follow such words, e.g., something close to what others have written in the past (Shanahan, 2022). It is a probabilistic process after all. The degree of dissimilarity would therefore be small by design. High values of novelty would be caused either by accidental, out-of-distribution productions, or by a careful prompting, i.e., one that would place the LLM in a completely unusual or unexpected (i.e., novel) situation.

Surprise instead refers to how much a stimulus disagrees with expectation (Berlyne, 1971). It is possible to identify three kinds of surprise, which correspond to three different forms of creativity. Combinatorial creativity involves making unfamiliar combinations of familiar ideas. Exploratory creativity requires finding new, unexplored solutions inside the current style of thinking. Transformational creativity is related to changing the current style of thinking (Boden, 2003). These three different forms of creativity involve surprise at increasing levels of abstraction: combining existing elements, exploring for new elements coherent with the current state of the field, and transforming the state of the field so as to introduce other elements. The autoregressive nature of classic LLMs make them unlikely to generate surprising products (Bunescu and Uduehi, 2019), since they are essentially trained to follow the current data distribution (Shanahan, 2022). By relying only on given distributions and being trained on them, LLMs might at most express combinatorial creativity. Of course, specific different solutions may be generated by means of prompting or conditioning. For instance, recent LLMs are able to write poems about mathematical theories, a skill that requires the application of a certain existing style to a given topic, yet leading to new and unexplored solutions. However, the result would hardly be unexpected for whom has prompted the text. For an external reader, the surprise would probably come by the idea of mathematical theories in verses, which is due to the user (or by the initial astonishment of a machine capable of it (Waite, 2019)). Transformational creativity is not achievable by means of the current LLM training solutions.

In conclusion, while LLMs are capable of producing artifacts that are valuable, achieving novelty and surprise appears to be more challenging as their inner autoregressive nature seems to prevent them from reaching transformational creativity.

Related posts:

Are you looking to demonstrate your expertise in designing, developing, and managing secure, scalable, and reliable cloud solutions using Google Cloud Platform (GCP)? If so, the Google Cloud…

For all you job seekers out there wondering if cold applications still get jobs, the answer is yes. Although networking fills 70% of executive positions, cold applications still fill 30% of these…

Of course there are many things that require taking care of in the corporate world, however, you can’t deny the importance of corporate apparel and uniforms. Corporate wear can help your brand stand…