While Char has been in the game without a break for a long time, I’ve spent the last years in academia (and with the startup). While teaching (and running) a games programme means being part of the general game dev landscape, it’s a very different place to making actual video games. It’s on one hand closer to upcoming trends because of the constant exposure to young people and their newest worldviews, but on the other hand academia is also safe by being removed from the gigantic changes that transform the whole industry into a new shape every few years4.

A lot has changed in the 10 years I was gone5. Trends that dominated at the times when I released my first games are either gone6 or just a part of the general landscape. A lot of challenges have been solved and quite a few new ones have emerged. Let me attempt to summarise the biggest developments, focussing on the independent games side of things because I only follow AAA from the sidelines:

• There’s a high quality engine or framework for every situation now. We used to have to write our own engines for new platforms or new game genres. Nowadays there is Unreal for VR and AAA-level graphics, Unity for mobile gaming and mid-sized teams, and Godot (and a dozen other engines) for smaller indies and freedom-seekers. Plus there are frameworks that allow you to cobble together your own engine in a high-level language. And there will always be C++, of course.7

• Distribution platforms are working with small teams now. Broken Rules was among the first 100 developers to get access to Steam and we were super privileged to get that level of attention. Nowadays everyone can just sign up, pay a small fee, and get access to the platform. Similarly, it required a long vetting process and lots of back-and-forth to become accredited as Nintendo developer back in the days. Without a particular Nintendo of America employee (thanks Dan!) we would have never gotten anywhere. Nowadays there’s a structured process for all console platforms. I released my first mobile game in the year Apple’s App Store opened its doors to third-party developers. Before that, mobile phone software was first- and second-party only.

• Dedicated publishers for independent games exist. We used to have meetings with Warner Brothers, Sony’s third party (kind of second, if you think about it) crew, and talked with Nintendo about funding during GDC. That all didn’t lead anywhere (except for being allowed to sell our games on the Nintendo e-Shop) but there were no Devolver, Hooded Horse, and Raw Fury back in those days. It was only in the early 2010s that things started to change.

• Community building became a cornerstone of game development. When I started out there were a handful of teams who would develop in the open, Wolffire Games being the most visible. Nowadays every new game comes with an offer to join a community of like-minded people.

• The gaming demographic has widened immensely. Digital games were still quite a nerdy hobby back in the days. You knew that you had to aim mostly for male 20-30 year olds to have success. Until mobile game developers and Nintendo thought outside the box and discovered that there are other people on this planet too. Actually they broke the box and it’s great that they did.

• Games are made by much more diverse people nowadays. Not only are there more and better distributed genders at work, there are also whole industries in China, India, South America, and Singapore, for example8. This greatly expands the kinds of experiences people can have in games and I love that9. Everybody wins.

• Discoverability replaced visibility as the biggest challenge. We used to just sail under all radars because no one really cared about indie games in 2009. With barriers of entry breaking down – thanks to better education, maturing technology, non-Western countries catching up, new business models, better internet services and competition on the engine market – gates that had previously been in place were no longer kept. And after those were thrown open, a myriad of new genres started to appear. Developers are now flooding the platforms as if the video game crash of the 80s never happened. The result is a winner-takes-all market interspersed with lucrative pockets, deep enough to maintain a lot of developers.

In our studio, we’ve decided to double down on why we make games in order to build a solid foundation for a new studio. So our focus is on replayability, community building, and timelessness. We’re making playful things that let you experience that white noise in the brain that really detaches you from your everyday life for a short while. We think we’re best at evocative narrative10 combined with elegantly explosive systems design. So that’s at the heart of our games. To design those kinds of games, we need constant exchange with the players. Because in our games you can expect unexpected escalation. Because we don’t want to make games that aren’t surprising to ourselves. Because we want to wildly experiment and know early if we’re on the right track.

It’s good to be back. Despite all its problems, the independent games circus is still my home turf and I feel like these new challenges are not harder than what we were facing nearly 20 years ago when I started out. They are just different and require a different approach to making games. So let’s do things differently this time!

Now go and play Sparrow Warfare, because we just dropped the demo.

The question whether games can be art was already answered by Marcel Duchamp when he (stole Elsa von Freytag-Loringhoven’s idea of turning a ready-made urinal into an art piece and) successfully had the piece called Fountain not getting exhibited. Fountain (1917) not being on display was as powerful an artwork as showing it, maybe even more so. This act of defiance made a definition of art obvious that had already existed: the fact that art is what an artists or a gallery points at. In other words, art is situated in the context much more than in the work. Some works need a lot of craft to earn a place in the temple. Others need to be results of an interesting thinking process. Some works also just end up getting contextualised because someone knows someone who knows someone (and/or because someone is born rich).

Museums all over the world have demonstrated that this is true by exhibiting games. In many cases the games are displayed as interactive works to be experienced as intended by their designers. In some cases, games are shown as objects – framing them differently but nevertheless turning them into art pieces. All of these perspectives are valid ways of looking at video games as art objects. As valid as looking at any other medium. So, here are games as art. But there’s more than one approval process for artfulness.

An argument I sometimes hear is that games are just the medium and it’s not so that they are art it’s more so that they can carry art (like a glass holding water). This argument usually comes from people who maintain that craft separates an artwork from a mundane object. People who think music has to be good (or old or at least complicated) to count as art.

Yet craft is only one possible way to achieve art. A valid and well established one. But this simplistic semantic separation is like denying that books can be art just because a bit of wood pulp, pressed and dried and then decorated with ink can’t be art. When we talk about books we actually talk about literature. When we talk about games we’re not talking about cartridges either. We talk about the medium of games including the interactive experience. And we know a thing or two about how the medium is the message since about 1964:

… the personal and social consequences of any medium—that is, of any extension of ourselves—result from the new scale that is introduced into our affairs by each extension of ourselves, or by any new technology. (McLuhan, 1964)

We talk about what the medium does to us, how it extends ourselves, and not about the technical medium. So what do games introduced in terms of extensions of ourselves? Mostly autonomy. Games offer playgrounds of agency. They do so by offering a possibility space – an opening to be filled. That means that what is in the game, the water the glass carries, is precisely not art. What is not in the game, the parts filled by the player, is where art happens. The artist making the game has to lead the player, potentially even push them, to exit their comfort zone and explore the possibilities. To a degree, the artist has to predict the possible dynamics of the interactive system they’re creating. McLuhan has another interesting observation that can inform an artist about the direction to go into:

And it is here that the artist can show us how to “ride with the punch,” instead of “taking it on the chin.” (ibid.)

And this is why games, to become good1 art, have to become less comforting, more daring, more connected to the world, and in one way or another provocative at heart. Games have to explore their medium, dare to break down under their own weight, and never play it safe, if they want to stand on their own art legs.

Damn, I’m working on an art piece again, it seems, despite having quit art twenty years ago.

When they were new, these technologies used to be the basis for art works. There is a whole art scene around exploring new inventions, pushing them until they are so close to breaking that their aesthetic features turn visible. To the curious, experiencing such artworks demonstrates the materiality of a technology. By breaking it, or bringing it close to breaking point, the seams show. The technology and its relation to the viewer of the artwork becomes apparent. That, in turn, helps the viewer to build a deeper understanding of what the technology means.

All art has materiality, even the most ephemeral piece of performance art1. All materiality is rooted in a process of transformation. This transformation turns something natural into something artificial. The transformation process is an amalgamation of creation, curation, interpretation, selection, layering, processing, communication, and many more transformative practices performed by the artist, the curator, and the audience. It needs all of them to turn a piece of nature into an art piece.

I always found it interesting that the arts managed to survive the Enlightenment period. The intuitive and haphazard practice of alchemy was replaced with the structured process of chemistry. Astrology and augury died at the hands of celestial mechanics found in astronomy and statistics. Many spiritual practices fell victim to psychiatry and medicine. Art survived, even thrived, in parts thanks to embracing new technological abilities.

“At first glance the arts might seem to have been in a situation like religion’s. Having been denied by the Enlightenment all tasks they could take seriously, they looked as though they were going to be assimilated to entertainment pure and simple, and entertainment itself looked as though it were going to be assimilated, like religion, to therapy. The arts could save themselves from this levelling down only by demonstrating that the kind of experience they provided was valuable in its own right and not to be obtained from any other kind of activity. Each art, it turned out, had to perform this demonstration on its own account. What had to be exhibited was not only that which was unique and irreducible in art in general, but also that which was unique and irreducible in each particular art.” (Greenberg 19602)

Many art forms still struggle with the development of what makes them unique and irreproducible in other media. Video games, for example, are an art form too dominated by entertainment to have an easy time finding its artistic footing. Many approach this medium as film with interactivity, storytelling with agency, or mindless entertainment. Video games are all of those, just like there are other media that feature a gradient between entertainment, utility, and art – e.g. music, writing, and photography. One thing that sets video games apart from most other forms of creative expression is their procedurality. Video games only happen when a player is interacting and they only progress via the player’s contribution. Their ultimate manifestation is dependent on the player in a more explicit way than the ultimate manifestation of a book is dependent on a reader3.

One way this procedurality is further emphasised is via procedural generation, a technique where the designer devises a system for creating game assets, stories, utterances, or game rules, instead of designing them directly. This is achieved by creating a set of elements as well as rules about how they can be combined. The data points can be words, images, rooms, enemies, attributes of characters, body parts, or any other pieces of a game that are used for assembling a shape via combination: Sentences are combinations of words and dungeons are combinations of rooms. Enemies are combinations of traits and abilities. And so on.

The design process is usually that the designer first creates a large number of examples of the outcome of the algorithm. Then they disassemble those examples into combinable elements – e.g. words, limbs, rooms – and define permitted probabilistic rules of combination. As a next step a wider array of base elements is created – more limbs, more words to choose from and so on. Then the designer goes back and forth between the source materials and the results of the combination process tuning the data as well as the rules. The resulting process of combining them at run time, while the game is playing, is defined by rules as well as probabilities. The designer can still know all possible combinations. The concrete manifestation is not deterministic but the design space is limited and clearly defined. One could argue that procedural generation is among the most unique and irreducible manifestations of materiality in video games.

In the case of Generative AI, there is a wholly different materiality in play. The artist working with such a system can not specify the full creative range of the system because the model they are working with is too large and the stochastic processes are too granular to determine every possible outcome in advance. Instead, the artist has to rely on interacting with probabilities trained into the system via the data, encoded in the training algorithm, and employed by the retrieval method. There are ways to constrain the output via fine-tuning and filtering, but there’s no definitive guarantee of constraining expressive possibilities. That fact in itself is part of the unique and irreducible aspects of the medium. Additionally, large models in particular are prone to produce very middle-of-the road results. In text, this takes on the shape of cringe, in images it takes on the shape of kitsch. Both of these forms can be described as “pretentious” in the sense of them pretending to be something they are not. And yet the expression the model lands on is the result of careful training. Maybe the cringe and the kitsch are not by-products but hints at uniqueness4? A uniqueness of the average, an irreducible blandness?

I cannot say what the materiality of Generative AI is going to be and if it even gets the chance to manifest before these systems become commodified, obsolete, or turned invisible. The current implementations and interfaces to this new medium do not allow for enough control (for anyone but experts) to have the amount of influence on the expressive range necessary for creating groundbreaking works. The market driving Generative AI more and more into the mainstream works against artists’ needs to exert a high level of control. Every developer is focussed on generality, ubiquity, safety5, and neutrality.

There are countless examples of media-aware artworks based on machine learning algorithms – Mario Klingemann’s and Holly Herndon’s works come to mind. It is no coincidence that their art is based on highly customised and considerably smaller models than the mainstream juggernauts. Simply querying an off-the-shelf product to produce art means to naively bypass most of the artistic process and there is little artistic merit in it. Cleverly constructing a critical perspective on our relation to our past, to technology, to the imaginary, and to identity is a whole different beast.

Somewhere in that sphere of critical inferences, cultural criticism, mainstream-persiflage, subversion, and stochastic forms of expression, there are countless artworks buried. All we have to do is uncover them by finding that coveted uniqueness and that which is irreducible in machine learning art.

The bubble of the internet around me has been having a field day embracing or rejecting Ted Chiang’s article titled Why A.I. Isn’t Going to Make Art. After reading it over the weekend I felt inspired to formulate my thoughts and write this reply. I think the article is well intended – it keeps talking about how important intention is, after all – and so is my reaction.

Why is this topic dear to me? Not because I’m working with AI, but because I’ve been a practicing artist for many years. I’ve had about two dozen exhibitions in various countries on this planet, the most exciting one definitely Manifesta 71. I don’t want to kick off the are-games-art discussion. So I will just say that every medium can produce artworks and things that only few people would consider works of art. I’m personally a very picky person when it comes to artistic works and my dislike for ontologies always shines through. That means in my opinion – that you don’t have to share – there is no strict boundary where something turns from mundane object to an artwork. It’s more of a gradient. And with time, something might become an artwork. Later it can easily drift out of the sphere where it is regarded as such. The quality “artwork” is not immanent to the work but externally assigned, for a while, by someone, for example me. So what is my working definition of art? It has to touch me emotionally or intellectually or both – I don’t care if it touches you, unless you reveal its qualities to me. It has to have value for me beyond its monetary value – I don’t care if it has the same value for you. These are the only criteria for me. I know that this view is very subjective. And yet, surprisingly many artworks that have been curated by museums check those boxes2. And a lot of “lesser” works do, too.

One thing that a lot of art that I love has in common is that it is made with awareness of the medium. A movie that knows it’s a movie. A video game that exposes the seams of the medium. A book that plays with what a story is. Again, this is not a criterium to be fulfilled for something to be art – it is one more way to touch me, by teaching me something about how the (mediated) world around me works. I admit that this is a matter of personal taste and not everyone gets as excited when they hear a scratching on the fourth wall as I do.

Now where does the artistic quality, the craft3, fit in? If a text is badly crafted, maybe lacking precision, that hampers its effect on the reader. That lack does not have to be an intrinsic quality of the text. Some texts age very well – or can be revived with a fresh translation – and others lose their relevance over time. Gladly, not all texts are written for eternity, and why should they? Being culturally relevant for a short while is fine. The same holds true for music, video games, statues, poetry, and all other art forms4.

I could write a long critique of the Calvinist5 concepts of relating effort with spiritual power that Chiang’s piece is based on. I could argue that writing a piece about why A.I. can’t produce art and then describing an artwork produced with AI in the text makes the title seem a tad sensationalist. I could write about my experience, that in order to produce any art work a lot of ephemeral products have to be created, and ask if AI is allowed to help with those. I could complain about the fact that it is naive to think the AI makes the artwork when it is clearly a human artist in collaboration with a compressed representation of human cultural history who facilitates the act of artistic creation. I could criticise that the simplistic view of prompting presented in the article is obviously a straw man. I could bring up cases where great artists have stolen in the past and the artwork was still relevant, though the artist should not be glorified anymore. I could argue that Sturgeon’s law makes no exceptions and no one can expect it to.

But beyond this paragraph I don’t want to concern myself with a text that forbids artists to work in a specific way. The same text has been written by the same people when electric guitars, mobile phone cameras, video games, books, and any other new medium came along. There will always be a gatekeeper. I will instead focus on pushing back against one single line of argumentation in the article: the need for decision making to make art. The need for control. I think this, too, is a far too simplistic and entitled understanding of art. And I can’t have that particular gate kept. I want both, art born out of control and art out of control.

Marina Abramovic held her groundbreaking performance piece Rhythm 0 in Naples in 1974. She put 72 objects – from roses to a loaded gun – on a table and stood still after giving the audience the following instructions:

Instructions: There are 72 objects on the table that one can use on me as desired.

Performance: I am the object. During this period I take full responsibility.

Duration: 6 hours (8 pm – 2 am).

There were a handful of decisions to be taken for creating this art piece: selecting the objects, the venue, and the time, for example. There was also the monumental decision to hold still for 6 hours in order to allow the audience to learn something about themselves. But the performance became art only by the artist relinquishing control. The audience themselves were the puzzle piece to complete the artwork.

For centuries (and most likely longer), artists have developed techniques to deliberately reduce the amount of control they have over the process of creating art. Automatic writing, cut-up technique, improvisation in jazz, situationist interventions, blues, pottery, punk, procedural content generation, and the musical dice game are all examples of this. Are the artworks created using these techniques superior to others? Of course not. But we would deprive humanity of a method for jolting us out of our routine if we reduce art to a careful sequence of decision making. We would lose creative freedom if those techniques are not allowed to be used as inspiration leading to the creation of (more controlled) art pieces, as intellectual toys, and as fringe artistic practice.

The process of creating a masterpiece is rarely a straight line. Only in hindsight can the decisions be regarded as having been clear. In reality, creating something new is usually a messy process of iteration, guessing, gut feeling, experience, trial, and error. It is a decision making process full of decisions that get taken back. Artists use all kinds of crutches to get from the empty page (that was never empty because no one starts from nothing) to the moment the artist (or the deadline) decides it’s finished. And then the process continues once the audience engages with the work, curators frame the piece in relation to other works, critics write about it, and an opinion is formed on social media networks. It only stops once the artwork is forgotten – which further underlines how little of its artistic quality was immanent in the first place.6

Generative artificial intelligence can be seen as yet another tool helping to voluntarily let control slip for a little while during a convoluted creative process. For getting feedback before exposing a work to the real audience. For getting inspiration when the well is dry. For creating unreliable and unpredictable dynamic elements in an interactive experience. The most average representation of an object in human history can serve as a reference, or as a topic for critical reflection7. Will A.I. ever produce art? I have no clue – we have to make it sentient first and I’m in the camp that thinks that we’re not really on the fast track to get there (and that’s fine). Will semi-automatic processes together with artists and with audiences produce art? Sure. They have done so in the past and will not stop at any gate, no matter how well it is kept.8

The paper goes through defining two degrees of bullshit, describing how LLMs work, and then observing that the output of LLMs can be regarded as bullshit. The reasoning is that the system does not have any notion of “truth” and – according to the paper – all of its users are also not interested in truth2, which in my opinion is a bit of an overgeneralisation.

ChatGPT, according to the authors, is a bullshitter not unlike a lot of people:

Like the human bullshitter, some of the outputs will likely be true, while others not. And as with the human bullshitter, we should be wary of relying upon any of these outputs.

This April we visited Char’s family in Ireland. In their garden is a shed that Char’s father built once one daughter had moved to Copenhagen and another one to San Francisco. When you move country you tend to leave a lot of stuff behind, things too good to donate to charity and too heavy to haul to another country. In Char’s case – and in mine, where everything is in my parents’ basement – the stuff consists 90% of books. The books live in the shed and whenever we visit we pack as many as can fit into our luggage. We do so rather randomly, just grabbing whatever looks interesting. This time, we found a book that Char had owned for 15 years but never got around to: Alamut, a 1938 Slovenian novel by Vladimir Bartol telling the story of Hasan-i Sabbah, founder of the Hashshashin (the Order of Assassins)3.

Hasan founded his own Shia Muslim sect called Nizari Isma’ili and was a master of strategy and deception. And in that he was a master of the long game. He took over Alamut castle without bloodshed by hedging out a plan that took over a year to come to fruition. To rid the lands of the Persias of foreign Seljuk rule, he schemed in an even more elaborate fashion. He recruited and educated a regiment of assassins who would take out key people in the government. For that purpose, he had to first invent the kind of brain-washing that is necessary in order to get people to risk their life for political gain. And of course he also needed strategic thinkers who would devise the political strategy with him.

He devised two different kinds of belief system for these two groups. One for the foot soldiers, the assassins, the army. And another one for the leaders. The maxim of an assassin was that he4 was the one “who offers his life for others or in the service of a particular cause”. The maxim of the leaders of the sect, according to Bartol, was “Nothing is an absolute reality, all is permitted”.

This last one, to me, sounds surprisingly like a “reckless disregard for the truth”. Truth is created instead of found, according to this school of thought. Establishing truth is a process that is neither deception, nor discovery. It is an act of creation.

When Google Gemini launched there was a big outcry. The system produced images of vikings with historically incorrect skin colour and female popes. It would retroactively bring diversity to historical situations that did not feature much of it. While the people in any reconstructed (or “bullshitted”) images never existed in the first place, it was very important for users that they could have existed and looked a certain way. Not all is permitted, even if there is very little absolute truth in a reconstructed image. This makes perfect sense, if you regard machine learning models as statistical approximations, which they are in their pure form, that approximate a certain representation of truth they have learned. Depending on the training material, in which a winner has written history, this representation might be closer to or further away from the lived experience of a person alive during those times. In any case, truth was created at some point. How do we balance the fact that winners write history in an intentional way with reality being something that actually happened (but that was additionally perceived differently by different observers)?

Should the single female pope in history be just a tiny point in a sea of data or should it be a significant example of who can be a pope5? The writers of church history would certainly prefer the former. History, though, was written by the Catholic Church and they decided what a pope has to look like by only electing white male popes. They have an excellent track record in message control6. In a way, what the Catholic Church did over millennia, is quite similar to what machine learning model vendors call “alignment”, the process of encoding specific values and goals into large language models7. The church curated a canon, made their own translations of source texts, and amended the lore where necessary in order to encode specific values and goals. They aligned truth to their values and goals.

What Google did when they pushed their model to produce more inclusive images was similarly alignment: they wanted the model to produce the kind of images that align with Google’s values. They simply forgot that most of history didn’t. They forgot about that truth. Google had the best intentions but one could argue that they added a layer of deception by not allowing the model to produce the statistically correct answer and instead intentionally pushed the output even further from any absolute truth and historical accuracy.

Statistically analysing human artefacts is already only creating an approximation of an indirect reflection of reality. Making the past “safe” via alignment can easily veer into a “reckless disregard for the truth”. The past was not a safe place, after all. Google made it obvious that the past as we see it is a wilful creation8.

In the “ChatGPT is bullshit”-paper, the authors make the argument that intentionality (“believe”) is a differentiating factor between bullshit and credibility. The lack of a “will for truth” inevitably leads to deception:

The machines are not trying to communicate something they believe or perceive. Their inaccuracy is not due to misperception or hallucination. As we have pointed out, they are not trying to convey information at all. They are bullshitting.

It is factually incorrect to state that the machines are not communicating something they perceive. The statistical analysis that forms the basis of the training process is creating such a perspective. The values encoded in the training material will surface in the output. This is what we’re talking about when we talk about “bias”. And it is why alignment was invented.

Machine learning models communicate what the algorithm perceives. The model runs the math of what the most probable next token should be, according to its training materials9. Of course it is not “trying to convey information”. That formulation anthropomorphises the system. A water pipe is not “trying to transport water” either. What a machine learning model does, though, is provide a probabilistic perspective on its training materials in relation to the prompt. The inaccuracy stems from the design itself and the lack of a measure of confidence in its output is indeed a gigantic design flaw.

Art is bullshit. The production of art is nothing but wilful deception. Authors come up with invented historical details, fill gaps of knowledge with made up facts, and invent whole people all the time. Then those people are played by actors who get paid for credibly bullshitting. Painters are known to depict impossible scenes and offer politically motivated one-sided interpretations of events. Even photographs are rumoured to be staged at times. Very often artists will go so far as writing a problematic figure that is rooted in a value system they object to. Quite a lot of art – think about instrumental music and abstract paintings – is not trying to convey information at all. Unless you count a well composed rhythm as information.10

Artists are of course not indifferent towards the concept of truth. Quite the opposite is the case. Yet most art is created as an externalisation of an internal truth of the artist more than raising a claim to factuality. A lot of “absolute truth” is intentionally left out. I don’t know enough about the agendas of individual artists to say for sure whether they “mislead the audience about the utterer’s agenda”. I have met artists with all kinds of motivations for their creative work and wouldn’t rule out manipulation altogether.11

Does it matter? Am I making the argument that ChatGPT is an artist? Of course not12. But what I would say is that for the production of art, interacting with our biased, incomplete, agenda-riddled documented past is a worthwhile endeavour. Interacting with machine learning models can offer interaction with more of our past at the same time, than a visit to the library. It can’t replace it – because sometimes you gotta go deep instead of wide – but it can complement it. I don’t trust the validity of the output of machine learning models any more than I would trust a dice roll. But serendipity has value. These models have value for teaching, research, entertainment, creativity, and other areas where we reflect using different perspectives. Even in business they can play a role in scenarios where a potential truth (and a potential untruth) can act as an inspiration.

The technology does not lend itself to providing anything but unintentional opinions of a machine. My truth is my reflection on the output of the model in relation to classically researched facts, made-up worlds, and half-remembered stories. Absolute, simple, straight-forward truth? I’d look for that somewhere else13.

EDIT: After some reader feedback. Edited a sentence to clarify that I’m far from thinking ChatGPT is an artist. Quite the opposite, actually. Also added a link to LAIKA to make sure my claims about LLMs being useful for “teaching, research, entertainment, creativity” is not unsubstantiated. Also added “entertainment” and made the sentence less vague.

For reasons that I assume to be rooted in psychology, chat interfaces have won the interface race when it comes to Generative AI. There are still some classic prompting environments, especially in image generation, but the majority of people seem to feel most comfortable in having a chat with ChatGPT instead of using tools that work more directly with their materials. I.e. pasting a paragraph into ChatGPT and asking the model to operate on it is a modus operandi that seems to work quite intuitively whereas a lot of writing tools out there that offer more direct interaction with the text, based on AI, seem to have a harder time explaining their interface. Or maybe that’s just our experience with LAIKA.

ChatGPT calls itself an “assistant” in the prompt. There are just three roles that the prompt format supports: “system” – the environment that the chat bot operates in, “user” – that would be you, and “assistant” – which indicates lines produced by ChatGPT. The relationship between the user and the bot is hierarchical in that one is an obedient servant whereas the other one is in charge. ChatGPT was carefully crafted to show no personality. At the same time the last OpenAI demo was a showcase of a bot that showed hints of personality. A strangely flirty female sounding personality that maintained the telltale obedience. Personally I found it quite unsettling. But I accept how hard it is to craft good characters.

In our Writers Room you find five different personalities. We tried to give them motivations and backstories that create unique exchanges with each of them. We tell them how to talk so they sound a little bit different to each other while we use the same base model(s). We try to make them as little obedient as possible but it’s hard to prompt that “instinct” out of them. All chat models I’ve ever tested were trained to be overly obedient. Since we have a multi-companion chat they are obedient to each other and that makes them appear super supportive. They are personalities but we hope it’s always clear that they are more akin to video game characters than to real people.

Our characters are designed in the vein of other artificial beings that are openly “created” but that humans treat like a hybrid of person and artefact. Virtual idols, puppets, and your beloved RPG party members are more related to them than Siri or Alexa. We think crafting an artificial person comes with a lot of responsibility and, honestly, we’re a bit tired of the trope of the flirty female servant. That’s why we built a whole system for characters that develop as you chat with them, that gradually open up, and that sometimes are in a bad mood.

Our companions live in a hybrid space that is both, the world of humans – because they were trained on human communication – and the world of computers – because they use math and algorithms to speak. This space has existed before Generative AI, in video games, stories, and other media. We intend to bring far more existing knowledge and design experience of this space into our systems. Why? Because we think it is fun, entertaining, enlightening, and rewarding to do social experiments with artificial beings. From flirting with RPG characters to arguing ChatGPT into a corner, social fun can be had with machines as long as they show some hints of personality and determination. Strangely, it makes machines more human, for better or worse.

The word “stratum” is an interesting one. According to my thesaurus it means “layer, bed, thickness, or vein”. And that is how language models should be used. As veins and layers. But you can’t expect a lot to happen if you only interact with the stratum itself. The power of the statistical inference that we call “prompting” is to cut into the stratum from the right angle. To dig for gold in the right place using the right tools. What works best for that is if you add two more strata: context and direction. In other words, you show the system what you have – a setting, a reference, a text, a piece of information from the world. And then you tell it what you want, e.g. you ask a question or issue a command. These three combined – what it is, what you have, and what you want – that’s the basis of worthwhile interaction with a language model.

Some models really profit from the digging process being explained to them. Step-by-step instructions and other sophisticated prompting techniques exploit that. It’s good for a lot of tasks. In fact it is so good that I would expect these methods to become a part of how models extract data instead of remaining in the realm of the user’s control. So let’s disregard them.

How do we use this in our agentic system? Well, most of our functions are based on combining the users’ input with the text they’re working on, some prefabricated data and the language model. The Writers Room is a set of virtual companions, co-workers that have different personalities, abilities, and specialisations. They are there to provide you with a simple, easy, entertaining, work environment, even if you’re alone with your work. They are in no way comparable to human beings. Imagine them like a party in a computer role-playing game.

For example, if you ask Alex, one of the members of the Writers Room, for a plot twist, they first access a small database of possible plot twists. Then they combine a random one with the text you’ve selected. That way they guide you towards how you could twist your specific plot. This is the one thing that these models can do that needed humans before: they can personalise something very easily and draw from a gigantic amount of knowledge. Very often you will brush away the advice of a virtual companion. At other times it might just be what you need to move forward.2

So here’s the magic formula3: combine something you want (a plot twist) with something given (a random twist and your writing) and use that to dig into the stratum of culture (the language model). You shall find gold.4

I primed them for some weirdness by telling them all about Aleister Crowley’s declaration from 1904 that humanity had arrived in a new age: the Aeon of Horus. He predicted that we’d witness a decay of the sense of sin and a growth of innocence and irresponsibility. And over the past century, we have indeed seen an explosion in playfulness in every area of life.

In 2014, I could see this playfulness so clearly manifested in the independent game development community: people daring to try new things, to experiment, to use play itself to find out more about themselves and others. There were excellent events around this time, like the Copenhagen Game Collective’s w00t Festival, Zuraida Buter’s Playful Arts Festival in Den Bosch, AMAZE in Berlin, and our own Inis Spraoi in Ireland.

When we gathered together, we knew there was something special in just playing. We could feel in our bones the power inherent in embodying the freedom of the child.

Things change, cultures change. The independent game development community is a fairly different beast these days. But the new aeon rolls on. When I saw the below picture again from my initial presentation all those years ago, I felt a shiver, a real connection between the me-now and the me-then.

It’s a panel from writer and occultist Grant Morrison’s excellent graphic novel “The Invisibles”, and this phrase: “the new Aeon’s dominant motif - a child fucking about with the building blocks of reality itself, restlessly destroying to create” resonated with me once more, from a whole new space.

Look, I know everyone’s bored to tears of hearing about AI by now. About the productivity gains and the job losses and the server farms and the copyright issues and so on. And I get it, I really do.

But the beautiful little spark of potential in it that delights me is the same little spark that delighted me in the independent game development scene a decade ago: dream it up, make it, share it - now using the tenets of consensus reality as lego bricks. Not trying to recreate the media of the past: not novels, not movies, not songs. But some strange new creatures, new forms that are no more than shadows right now, coalescing into shape just out of the corners of our eyes.

Things that will be “their own thing”, incomparable to the art we have now in the same way a video game is incomparable to a photograph, or an oil painting is incomparable to a poem. Not a competition, a whole new thing altogether. When you think of the art that means the most to you, is there anything more exciting than seeing something that’s completely “its own thing”?

A few weeks ago, we went to see the excellent Firelei Báez exhibition at the Louisiana gallery here in Denmark and my soul sang. I was entranced by her combining, her remixing, her luring new meanings, the old maps she paints over like “ghosts of the past haunting the present moment.” How can paint and paper be turned into this completely new form that turns my belly to water, my heart to fire?

Magic.

The magic of creativity and play can alchemise any raw material into beauty and meaning. And the nature of humans is to find new forms, to find new meanings.

I know I’m not alone in this appetite for new forms. You can see it too, right? Structured playgrounds are too limiting, constricting. I want to “yes, and…” everything, and I think others do too. People want to make things, to share things, to take their playing out into the wider world. Look at the growth of Minecraft, of Roblox compared to “traditional” video games. Even social media - itself a new form - rolls on, turning towards production over consumption: streaming, making videos, sharing pictures.

And here in the baby years of AI as a tool, we’re just exploring, interacting. Like a toddler learning more about their environment and what they can create. And sure, right now what we’re making with these new tools is like those first masterpieces we scrawled with crayons as two-year-olds that Mom hung on the fridge. We’re learning, developing. We need to experiment, if we’re going to find the new forms.

So I’m feeling an update to what I shared ten years ago.

Play, not limited to playing a game made by someone else, or playing by someone else’s rules. Toys, more than games. Play, no longer something creators keep to themselves as part of their creative process. Instead, inviting others into the sandbox, rather than competing on the swings.

Play, liberated to the extent that creators can remix all of reality to express themselves.

Will we find the new forms we seek? Ask me in ten years!

The Elite universe contains eight galaxies, each with 256 planets to explore. Due to the limited capabilities of 8-bit computers, these worlds are procedurally generated. A single seed number is run through a fixed algorithm the appropriate number of times and creates a sequence of numbers determining each planet's complete composition (position in the galaxy, prices of commodities, and name and local details; text strings are chosen numerically from a lookup table and assembled to produce unique descriptions, such as a planet with "carnivorous arts graduates"). This means that no extra memory is needed to store the characteristics of each planet, yet each is unique and has fixed properties. Each galaxy is also procedurally generated from the first. (Wikipedia based on The Guardian)

Out of these technical restrictions, whole genres were born. The modern rogue-like is maybe the most obvious example. The original Rogue (1980) featured permadeath1 and procedural dungeons. Modern rogue-likes are often signified by carrying something over from one run to the next. They often unlock more content over time as the player beats the game again and again. Rogue-likes excel at replayability thanks to this formula. Procedural content generation becomes a tool for making replays of the same content exciting.

Related genres have used procedural generation differently. Bethesda uses a host of procedural generation techniques in all their recent games. While Oblivion just had some procedurally generated dungeons, trees, and randomised Oblivion gate placement, Elder Scrolls 6 will supposedly feature a procedurally generated map. A lot of procedural generation tools are used during design time as opposed to play time, meaning that the assets they generate are curated by a designer.

Quite often these assets get mixed with hand-crafted ones or manually altered before making it into the game. Just like using the paint bucket tool in Photoshop can be regarded as procedural generation, we use a lot of small design automations that are of varying degrees of sophistication when creating any digital artefacts. Content-aware fill is a step towards a more clever fill operation. Generative fill (also called “deep fill”) is using machine learning for an even more “intelligent” fill operation. What is the difference between these three methods? Paint bucket fill uses only data specified explicitly by the user. They select a colour and then fill a region. Content-aware fill uses data from the image itself – indirectly provided by the user – to fill the area in question. Generative fill uses data from other images to guess what it should put into an empty space. External data is suddenly added to the mix.

The fact that the filling of the empty area is accomplished by creating the statistically most probable content for the area in question inevitably leads to a kind of mainstreaming. Its aim is to surprise as little as possible. It works best if the image is predictable and similar to other images the machine learning model has seen. Yet the user has less control than when working manually and arguably also less control than when working with techniques based on their own content.

This loss of control is the most frightening but also the most exciting element of machine learning. And it requires a lot of learning before we can harness it in different contexts. What we get in exchange for the control is a peek into the latent space. A snapshot from a statistical analysis of the training data that relates to the context we are using to query said data (the surrounding area of the hole in the image in the above case). In case of generative fill the training data is not known and can not be altered. In many other generative systems we have access to the training data and to a host of techniques for selecting what aspects of the training data to emphasise. Fine-tuning allows us to radically focus a model on a specific subset of its data and the data provided to the fine-tuning process. Suddenly we find ourselves with much more control over the expressive range of the machine learning model we’re working with2. If we then curate the results instead of using the model at runtime we find ourselves in a position where we have full control over our creative output again. More than that, we might not even use the content that we’ve generated directly but instead only use the bouncing with a model to develop our thinking, our voice, our style, or a tiny corner of our narrative.

Elite could not have been created without procedural generation – it contains too much content to generate all of it by hand. Manually editing a photo takes much longer than using an advanced fill method. Creativity profits a lot from bouncing with other minds, natural or artificial. So, in a way, there is also a tradeoff between necessary effort and loss of control that has to be navigated when choosing what generative tools to employ during an artistic process.

Control is something we don’t necessarily want in every moment of every creative process3. Video games, as an art form, generally require the designer to let go of control, handing it over to the player. Maybe that’s another reason why procedural generation is so prevalent in that medium. They are, in this regard, more related to live music than to a more fixed medium like film, books, or studio albums. There are of course techniques for maintaining more control even when a machine learning model is used in realtime. Filtering results is a simple one, alignment generally refers to methods of keeping the output within certain parameters. These techniques reduce the expressive range of the system and thereby make the output more predictable4.

It is safe to say at this point that there is no way to safely launch a video game that uses large language models on console platforms. Apart from unresolved legal issues, age-rating systems demand guarantees that a language model can not fulfil. There’s always a danger of inappropriate content slipping through. While that’s not so much of an issue in other segments of the games market – AI Dungeon launched very successfully on iOS after all and is also available on Steam – it prevents traditional game studios from seriously engaging with these new technologies, at least during runtime. But that is going to change as regulatory and legal frameworks slide into place and as people get more used to free-form interaction with computers.

But what are ethically and legally sound forms of using these sophisticated technologies that go beyond procedural generation right now? Here are three things one can do right now:

• Walk through the latent space like a tourist, taking snapshots whenever one feels like it.

• Explore the past by reassembling it in the now.

• Make your own system that is based on models you have full control over.

• Bounce with models that don’t directly work with the content but indirectly offer guidance and feedback.5

It’s a fascinating thing to compress culture into a messy interactive database and make it alive again. There are a lot of ethical issues left. How we will use these powerful new extensions of procedural generation is important. And what it means for existing professions has to be taken into account. But while we iron out how to ethically mine the data necessary to build machine learning models, we have to work on processes, aesthetics, and the formation of taste. The current results are predictably aesthetically unexciting6, a fact that is stemming from their use as well as the technologies and their makers’ choices. The processes enabled are already all the more fascinating. The next generation of creators will inevitably grow up with these tools at their fingertips – it’s time to put them to proper work and pick the right level of comfort vs. control for every situation.

The reason for that is of course on one hand that the book we’ve used in training this brain is very versatile. Meditations is a philosophical treatise1. The thoughts presented are the thoughts of an emperor who had the time and mind to contemplate his purpose in life. His stoic philosophy promotes modesty of the kind you can most easily afford if you have an army of servants. Yet there is timelessness and poetry to his musings that helped them to remain relevant until today. The writing still feels fresh and exciting.

Thanks to my many interactions I have come to recognise Marcus’ tone and choice of words. I’m far from an expert in stoicism. And yet I have conversed enough with Marcus Aurelius to have a feeling for how he will generally react to half-sentences thrown at him. I’ve come to know Marcus2.

The most used AI-based application out there is ChatGPT, a service that most of the readers of this newsletter will be familiar with. Unlike the Marcus I conjured from his own writings, ChatGPT is bound to be factual. I asked him about Marcus Aurelius, too, and this is what I got:

While, in essence, this is a good summary of Stoic philosophy, it contains quite some circular logic — or at least redundant redundancy; What is a Stoic’s moral and philosophical integrity if not the alignment with Stoic philosophy? At the same time, this dry statement kills all the joy one can find in Marcus’ 12 books of poetic writing.

The same happens if I interrogate ChatGPT about randomness and what it would mean in Marcus Aurelius’ philosophy. Instead of inspiring me it produces a college essay about Stoic philosophy in general. It’s not wrong but about as exciting to read as the small print of an insurance contract.

I know there are other language models out there that exhibit a bit more personality. It looks to me like chat-based interaction is a really tricky one, though. Gladly, I would not want to chat to my Marcus too much – I just like him to complete my sentences now and then to send my mind on an interesting journey. I don’t want to argue with him, nor do I want to ask clarifying questions. I like him in his delirium, reflecting on things he had no opinions about because they were not around during his time. Whenever ChatGPT’s personality shines through it reminds me of those servants that Marcus Aurelius must have had. Those that allowed him to live an introspective and – from his perspective, at least – morally superior life.

Our reanimated Marcus of today, bottled up in his tiny language model not even a fraction of the power of ChatGPT, could similarly live a modest and morally superior life by leaving all the dirty business of real work to the ChatGPTs out there. So he can lean back, relax, and now and then whisper words of wisdom into my ear. He wouldn’t even have to ask himself why ChatGPT exists…

In this newsletter I will write about creativity in fine art as an example. Most creativity today happens in commercial settings, in an economic context that I mostly leave out. I would never say that fine art is beyond economic necessities1 – it’s just not what I am concerned with in this tiny polemic about Generative AI. So let us leave the elephant in the room, in plain sight, a majestic grazing beast that has already consumed all and does not always have to steal the show.

What is creativity?

My personal view is that creativity is a process with a result, but the result is not necessarily tangible. A simple example of a creative process is painting an image. The artist starts with a blank canvas, applies paint, and at some point stops and has arrived at a painting. Yet is even this case so simple? The artist brings their life experience to the process, so the painting has started long before the first brush stroke is applied. The artist might paint over the same painting several times, only the last one being “published”. Are there also economic reasons or purely artistic reasons to regard a painting as finished? If the painting goes to a gallery, what paintings are shown beside it affect how the painting is seen by visitors. Paintings are actually sculptures – a hill I will die on – because they are three-dimensional. They change with the distance you view them at, with the viewing angle, with the light that falls on them. If the painting creates a scandal, which doesn’t happen that often but nevertheless has happened in the past, it is seen differently again. The personality of the artist also sticks to the painting, tainting it. To summarise, even in this most classical case of a creative act it is impossible to tell when it starts and when it finishes and what exactly the product is.

How can we take this bundle of causes and effects and separate them a little bit so we can get a clearer picture (pun intended)? First of all there are considerations leading up to the core part of the creative act, to the moment when the main product, in the above case the painting, is created. Most importantly, the result space is usually constrained before the act of painting starts. The canvas has specific dimensions that dictate how large the painting will be. The choice of paint limits the creative possibilities further (e.g. oil paint dries fast, necessitating specific techniques of layering). A time frame might also be decided for in advance. A lot of constraints come with affordances2: water colours lend themselves to outdoor scenes and paintings that require a certain viewing distance to “come together”. In a way, affordances are just a different way of looking at constraints.

Every painting starts within a context of its creation. There is no blank canvas. The painter always has prior experiences they bring to the table. They might try to bring a certain feeling across to the viewer. They might just have an image in their mind they want to get out. They might be paid to paint a specific scene or portrait. They might just admire a person, situation or place. They might draw from their memories or a photo they saw in the newspaper. They might draw purely from their intuition. There are hundreds of starting points but one thing is certain: the painting process has already started before the first brush stroke3.

Before the actual product is created, preparatory design takes place. In visual arts, this is usually called sketching, in video games pre-production, in special effects pre-visualisation. Quick throw-away designs are created, often a lot of them, and then a curation process leads to one or more sketches that serve as references for the later work. Often, this process is iterative and goes into more and more detail over time. In painting, the painter often transfers a part of this sketch to the canvas, to serve as a template.

Then the actual painting process starts. It is, like most truly creative processes, an iterative process of multiple stages that bounce between creation and evaluation. The painting itself informs the next step. There can be a master plan in place but even if the painter is very strategic in their approach, they will be guided in the execution of the plan by unforeseeable observations. If the painter is less strategic, they will be even more guided by what they are doing while they are doing it. In general, creative processes go through phases of addition and subtraction. Form4 is accumulated and then form is removed. Over and over again. Depending on the medium there might be more or less planning. More planning usually means that more time has to be spent in the sketching phase, and more creative decisions are taken before the actual paining process. At some point, the painting is “done”. This is a decision and not triggered by the painting having achieved any specific state. Unlike a bridge that, at some point, serves its purpose, cultural products are only finished when the artist makes the call. Of course, external influences might trigger this call. If there is a limited time budget available, or if the exhibition date is coming up, painters might speed up the process. Some paintings can keep requiring more work and never reach a state where the artist would decide it’s the right moment to stop. Work ends at some point though5 unless it’s the Sagrada Familia we’re talking about.

As a next step, the painter usually frames the painting (or has it framed). The frame changes the painting and also contextualises it. It becomes part of a class of paintings that have this type of frame at that point. Or a part of the class of paintings that have no frame. This, too, is an artistic decision.

Finally there is the long process of curation, a process that never stops. Not only does the painter select which painting goes where – or economic pressure will decide it for them – but galleries also present paintings adjacent to other works of art. If the work doesn’t go to a gallery it will still end hanging beside something. Never displaying the painting is also an act of curation. Over time, a painting will end up in many different contexts. One could say the curation process only stops with the destruction of the art piece but even then the painting might live on in other media. An echo that never stops, just gets weaker over time.

There are whole books written about creativity but let’s work with these seven concepts for now and look how Generative AI connects to all of them, how this new paradigm of creation manifests in these stages of creativity.

Generative AI in the creative process

In order to contrast painting, I will focus on image generation when talking in this chapter. While the same can be applied to other modalities, it just fits better to how I started.

An AI artist will also first set up some constraints and lean into some affordances. The choice of technology is one of those. If they use Midjourney, they are aware of the creative range of that framework as well as the available parameters. They will also be aware of the aesthetics that the Midjourney model excels at versus those where it delivers poorly. If they have experience, they will work with those constraints as a guiding affordance. Models with open datasets allow the artist to learn about the reference space of the technology they are using, guiding them in the technological choice. Additionally, many artist add seemingly arbitrary constraints to the way they work, further narrowing down the expressive range.

Most successful AI artists have experience in other art forms. Those are part of the context they are working in. They might even be more interested in the conceptual implications of AI technology than in its aesthetic abilities. A lot of them work very actively with the implications of using technology that was trained on our cultural heritage, juxtaposing the old with the new, the established with surprise.

Recent months have seen a wealth of new methods being introduced that help with the sketching process in AI-generated image creation. ControlNet6 allows to work with a base image that dictates the overall shape of the result. PoseNet7 detects the pose of a person in a video or image and can then be used to replace that person by using it as input in ControlNet. Algorithms that provide more control over the image generation process are necessary in order to predict more of the outcome. Of course, prompts are also just constraints for the model and similarly reduce the range of potential result images. Prompt engineering is a way of sketching if the artist has enough experience.

Midjourney displays all generated images in the Discord server you are using it and offers a handful of options to iterate on them. This iteration happens in public, which means that there is a collective learning process. Apart from making variations of an image, the artist usually makes slight adaptations to their prompt until they reach a point where they decide to stop. Just like with a physical painting, this process is guided by more or less of a plan depending on the artist and the situation. Some artists jam with the system, hoping it takes them to interesting places. Others come with a more specific idea of the outcome and try to home in on it.

As the next step, AI artists take the image (or several images) out of the context it was created in and into the context they want to present it in or continue working with it. They might overpaint it in traditional digital tools, out-paint using more AI tools, or print it and work with it in the physical space. They might turn the image into a part of a collage or of a comic page. This is quite similar to the above-mentioned framing process in that it transfers all the associations of the target medium to the image. Once the image has become part of a comic it will be read as a comic panel. The framing has huge implications for what we see when we look at the image.

Finally, as well as at many points leading up to this, curation happens. No artists publishes all of their works. Most creations never get a lot of eyeballs. Platforms highlight some works of art and others just vanish in the ever increasing amount of visual content online. Some art works have a different purpose altogether and serve as inspiration for other, not AI-generated works. Concept art that is only used for a single discussion would be an example of this kind of art. Concept art that was only necessary to think through a visual concept is another one. Art that only exists to inform a traditional art process exist too.

Nothing is new. Everything is new.

To me it seems like we desperately need a more nuanced view of what changes Generative AI brings to the creative process. These systems do not simply regurgitate existing cultural artefacts. Neither do they always replace the creative process with an automatic one. It is also very clear that artists with an existing artistic practice can create more interesting works of art using AI than others. The technologies are so new that it is very hard to gauge what, if any, foundational shifts will have happened when the novelty wears off. We’re still in the middle of the hype cycle, maybe even close to the start, than the end.

What can be said with certainty is that the tools that are available right now support different aspects of the creative process. Midjourney excels at iteration but has a very limited expressive range. This limitation makes images generated with this model recognisable, which is not necessarily to be viewed negatively. Acrylic paint also comes with a specific aesthetics that is recognisable and not suitable for all kinds of images. Maybe new aesthetics are the most interesting area to explore with machine learning.

Personally, I’m more interested in supporting than in replacing parts of the creative process8. This is not what all artists are interested in and that’s fine. I want to make software that allows you to play with the medium you are working with. I want to make software that lets you explore and surprises you. I want to make software that lets you interact with our cultural heritage in new ways. I want to make software that is a little bit dangerous because I believe art needs to have an edge or it turns out blunt. I want to make software that lets humans feel creative. I want to make software that is a tool for thinking more than a tool for production. I want to invent bicycles – not sports cars, not mass transit solutions, not tanks – for the mind.

What I had done with this piece was to only provide structure for play. No instructions, no rules about how the process of interaction was supposed to be were provided. In game development, we call this a sandbox, despite the fact that it applies to all playground structures. Playground elements suggest a process but don’t enforce it beyond the restrictions of physics and physiology. We bring our own experimental process with us and explore our own creative potential by testing the limits of the structure we interact with. We play, we toy around, we learn – about us and the world around us. Yes, philosophically this ties in nicely with constructivism as well as phenomenology:

Phenomenology is a philosophy of experience. For phenomenology the ultimate source of all meaning and value is the lived experience of human beings. All philosophical systems, scientific theories, or aesthetic judgments have the status of abstractions from the ebb and flow of the lived world. (Paul B. Armstrong)

You’re the pinball bouncing around in a given structure but you’re also the emotional human entity excitedly pushing the buttons that move the paddles of the pinball machine. Life is just structure and process, as long as you apply enough abstraction to your worldview. Like a lot of philosophical perspectives, perceiving the world as process and structure does not lead to metaphysical revelations. And yet it is a useful tool for design. Like all design tools and philosophical perspectives – or maybe we should call them lenses in this instance – it highlights some aspects of the world but casts a shadow over others. A shadow that makes them vanish into the background.

In this light we can design things that help people to explore themselves and the world around them. And that’s how LAIKA should be seen, how it was meant to be situated in this weird world of AI automation threats. It’s an unpredictable structure to play with, a toy that – like all good toys – can teach the person interacting with it something about the past and something about their creative practice. Rubbing your thoughts against Marcus Aurelius’ stoic philosophies, the Communist Manifesto, and Mary Shelley’s fantastic fiction teaches you about the history of writing, the history of thought, about who we are as humans and how we express ourselves. The process of writing becomes and exploration, the artefacts assembled become Mono no aware – stills of the ephemeral. Like holiday photos they maintain a hint of the moment but are forever just a shadow of the experience.

LAIKA is not a tool for productivity, it’s not even trying to be one. It is aiming far higher and far lower than other AI writing tools. That means it has quite a learning curve and the techniques you’ve learned from interacting with other text-driven AI tools can not be applied. We will never introduce processes that prescribe how to work with LAIKA beyond what is technically and conceptually necessary to unlock complex interactivity. Instead, we will continue to give you an as safe3 as possible play area – a sandbox to build your own creative sand castles in. LAIKA is a playground for thinking with words. LAIKA is pure structure. You bring your own process.

The philosophy of Structure and Process once again only filled a pitiful few paragraphs, nearly 30 years after its conception. And yet, the idea is going to hang around with me, gently illuminating my mental cave walls, casting light and shadow. Maybe one day I’ll dig out my tragic theatre play, too, and look at LAIKA from the perspective of emotion and rationality. Creativity pinball is more fun, though. Enjoy our playground built out of Versatzstücke4 of the history of humanity.

The AI hype train keeps accelerating. We have passed a number of impressive landmarks in the recent weeks. GPT-4 was released. Midjourney v5 and SDXL became accessible, Bard is out in the wild, and Runway just released their GEN-1 video generator1. But what I see on the horizon is a fork. And it’s only the first fork of many to come. The hype train will not take any of them – instead it will split with every fork, going down both tracks until the train is more an avalanche of little trainlets2. And it will assume a quantum superposition of being all possible hype trains at the same time any moment now3.

The fork I am talking about is the split between accuracy and creativity. It is the distinction between precise reproduction and colourful fabulation. While the “generative” in “Generative AI” suggests that the best application of such systems is in generating something new, in practice they are more and more used to replicate something known as reliably as possible. That’s the goal of ChatGPT, Bing AI, and Bard. They want to be search engines that don’t generative semantics, only syntax. And it takes a lot of harnessing to get them to do that.

But human communication, and especially creative work, has different requirements. Linus (@thesephist), a highly creative crafter of interfaces to generative models, recently posted about how his personal chat bot is not based on ChatGPT, because he wants it to be “less pristine and direct”. That the current generation of bots for knowledge retrieval is “overly obedient and hopelessly unimaginative” is of course by design. Their imagination is what makes them unsuitable to replace Google search any time soon. Their lack of imagination is what makes them unsuitable as creative partners any time soon. But they don’t have to be both.

With LAIKA we’re of course going down the road of imaginative generators. Our ensemble cast of brains is clearly not obedient or direct4. Even bias, the bane of general language models, is a strength for us. You just pick the bias you want to work with and the system even remembers every one of your biased robot collaborators.

This means we are decoupled from the main direction Generative AI is heading towards. It’s going to be interesting if we can team up with enough creative minds who want to explore the generative potential with us or if commodification, domestication, and accuracy will turn this endless source of the weird into a corporate desert of boredom.

Here’s to the weird ones!5

And yet we can make some observations about how Generative AI is currently used. Clearly we’ve barely touched the surface of what it means to interact with the vast accumulations of cultural history encoded in large language and image models. So far, most people who are serious at working with Generative AI are recreating past workflows, trying to replace pre-machine learning tools with shiny, new, more “intelligent” (whatever that means), yet ultimately often unreliable tools. It seems to be the only way to productively work with the current generation of Generative AI.

Even big corporations are expected to do exactly that. While replacing internet search with an unreliable chat partner is a questionable decision, the team behind Bing AI chat is at least acknowledging that new forms of interaction are needed if we want to harness what Generative AI is good at. It comes at a price, though. And it does not meet users expectations of what constitutes an internet search.

I’m at this point unsure whether Bing AI chat is a bold, yet misguided, step in an interesting direction or a failed optimisation of an existing workflow. In a way, it is both. And neither.

Developing new ways of interacting with media is much harder than replacing established systems. It is far more unpredictable and requires much more communication. It’s also much easier to get VC money for replacing an existing process with a more efficient alternative. Hence the many startups that do exactly that.

Personally, I find it much more interesting to seek out these new unexplored creative spaces. To find new modes of expressing ourselves as humans, together with increasingly intelligent seeming machines (stochastic parrots, they are). I’m excited to create platforms that allow others to do things they could not do before, to experience forms of interactivity they could not experience before, to interact with our past in ways that were not possible before.

It’s going to be a long journey, though.

Humans are not Malfunctioning Machines

Generative AI is creative because it learns from images just like humans do, right? No. The first argument I would like to make is that humans are not malfunctioning machines. We are not rational beings inconvenienced by emotions. We are emotional beings who are now and then able to act rationally. We are intuitive beings that now and then can reflect on their behaviour. And that’s fine. I even think the more we embrace the above facts, the easier we have it in life.

The best book to read on this topic is Lisa Feldman Barrett’s How Emotions Are Made, where she attests that “emotion is constructed in the moment, by core systems that interact across the whole brain, aided by a lifetime of learning”. Emotions are very much at the core of our experiences as human being.

We are not machines, and the machines we are modelling inspired by biology are not as similar to us as some think-piece writers claim. That has no bearing on the answer to the question of whether AI is able to be creative or not. Crows, whales, and octopi are creative, so why not attest it to machines independent of how much similarity they have to our brains?

Generative AI usually does not Create on its Own

But Generative AI is still creative, right? In a way. The whole question of whether machine learning models can achieve creative acts is often reduced to the activity of the model itself. As if any of the systems we have creates in isolation, on its own. In Midjourney, the user first prompts, then guides the generation, then – quite often – retouches the result, and then finally curates in the sense that they decide to publish the image somewhere else or not. And most of this happens on Discord, in a social space where everyone is observing each other and getting inspired by each other. This whole process is how Generative AI works. The actual act of inferring from the model is but one tiny step for the human user. In Write with LAIKA, writers train little language models and then interact with them over a long time, bouncing between inferences, overwriting the results, integrating them into their existing work until the boundary between what is human created and what is machine made blurs. Not only does that mean that it matters little whether the AI itself is creative, it also means that separating between what the machine created and what was human-made becomes impossible.

When we are creative we always engage in a tight loop with the world around us, using interactive cognition to explore the space between constraints of the physical world and of our cognitive abilities. Henrik Gedenryd wrote a fantastic thesis about how designers think. His main argument is that we think in interaction with the world and that cognition – feel free to replace it with creativity, which is but one attribute of cognition – happens not in our brain alone, as an isolated activity, but is a function of us interacting with our environment. Gedenryd’s theories require a lot of rewiring of your brain. I highly recommend doing just that.

The Product is Journey

But the results are art, right? Who cares. If we work in this interactive fashion with Generative AI systems, what does that mean for the products of such a process? Well, first of all it means that the product is relatively unimportant compared to the activity. I’ve been using Generative AI systems, and other procedural systems before that, for many years now and have been observing others using them. The journey that you’re on while interacting with AI is much more fascinating than the result. Now of course there are people who just want to churn out some assets for their game or have other utilitarian goals in mind. They will be served the oatmeal that procedural systems produce. But I think the majority of interaction with Generative AI is not about the result and even posting any results on social media has nothing to do with the actual images, texts, and snippets produced. It’s an isolated activity purely existing for social reasons. What is interesting is dwelling in a huge semantic space assembled from human cultural artefacts. When we interact with Stable Diffusion, we are walking through arbitrary recombinations of human expression. We are drifting through latent space when picking a direction in Midjourney. We are bouncing ideas back and forth with great minds when running inferences in LAIKA. And much like travelling through a foreign country, we now and then glimpse something we want to hold onto and take a snapshot of that moment. While to a lot of people these products are the main result of Generative AI, I think the main result is that we have a new way of interacting with our culture. We suddenly are not bound to search databases of images in order to understand our past but can actively recreate it in ways that make it more tangible than the highly biased curation of historians will ever be able to. That being said, I have high regard for historians and archaeologists – but historically they have focussed on very specific viewpoints from which to look at history. Exploring the image spaces of Generative AI is a bottom-up, constructive approach to history. There is a lot wrong with that but it does not mean it’s not a fascinating endeavour. What is wrong with that? Well, here’s a list of things that are actually problematic and you won’t find the death of art on it:

• Data sets are still constructed with a Western bias, just like history was written with the very same bias. We need more diverse voices.

• Data sets are also constructed largely with Anglo-Saxon ethics. I wonder how much Renaissance art, ancient art, non-Western art and critical art from the sixties and seventies can make it through a porn filter. We need to move beyond one-size-fits-all content filtering that reduces the potential of expression in highly culturally specific ways.

• Data sets are made by engineers. Now I might come off as a snob but there is definitely a “crisis of taste” in AI art and beyond it. We need ways to creatively explore more challenging aesthetics than anime and concept art.

What is the product of working with Generative AI? It’s akin to a holiday snapshot, or maybe a mix tape you make for someone. Both are only valuable with context but worthless on their own. Now of course one would say that art is like that, too. Worthless without context. And one would not be wrong.

Holiday Snapshots from the Latent Space

Creating with Generative AI is like going on a vacation to the realm of cultural heritage. We take snapshots of interesting scenes that only happen to us and they are about as reliable as other accounts from the past. Creativity happens in interaction with the world around us, like all of cognition. Generative AI can be a great new addition to this world we’re interacting with while being creative. It’s like suddenly having a significant junk of human history to rub against. That does not mean that the results are always art or that they never are. That depends on the context they are presented in, turning the artwork itself into a process (and the process of making art into the artwork).

“Cold things warm up, the hot cools off, wet becomes dry, dry becomes wet” as Heraclitus says. Everything flows. Maybe in the future we will not be concerned with the products of Generative AI at all. We will constantly create and recreate instead of ever freezing a result in place. All art would then be in a permanent state of flux, forever changing, now and then holding still for a short moment, getting permanently constructed and reconstructed as we move through the past and the present of humanity.

As a maker of a tool in this space I spend a lot of time thinking about the implications of what this new wave of AI will have. Here’s an attempt at structuring those thoughts that I’m writing for you, the reader, me, the creator, and everyone who is in my position. It’s a discussion starter and as a manifesto it is formulated sharply to have sufficient impact. It’s still just a standpoint. Hopefully one that makes you think.

Ethics in Generative AI is challenged in four areas that are all connected:

• Data collection, ownership of training materials

• Ownership of results

• Bias in results

• Sustainability

Disclaimer: Everything here concerns generative AI and does not simply apply to other applications of machine learning. We’re talking about entertainment, creating cultural artefacts, and art – don’t apply the same thinking to predictive policing. Also, in case that’s not clear – I’m neither a lawyer nor an ethicist.

Data Collection

The data that goes into a machine learning model, the training material, is defining the potential output. Digital technology created immense data collections, from museums digitising all their properties to social media aggregating voices from around the globe. Data has also become one of the key assets of the digital economy, with company values being tied to the size of their social graph. Machine learning started out in academic contexts and science has a long history of privileged access to data – and also of having infrastructure and processes to handle sensitive data. Science also has a history of being very transparent about data. Even today, the data collection part of training large models is often outsourced to universities, yet the trained model is commercialised. The fact that this is legal says something about how far legislation is lagging behind technological innovation. But even if countries are willing to legislate this area there is an ongoing arms race, especially between the US, Europe and China, about technological leadership. Lawmakers will continue to have a hard time finding the sweet spot between harnessing the power that was unleashed and protecting the rights and privacy of their citizens. Given that technological innovation will stay ahead of legal frameworks in the foreseeable future, ethical frameworks that go beyond legal requirements are needed.

To ethically collected training data, the following criteria have to be met:

• Legal requirements have to be fulfilled.

• Implications of licenses that were created before the advent of modern machine learning have to be taken seriously.

• Training corpora have to be fully published.

• Decisions about what to include in a training corpus have to be made and documented.

• Mechanisms to prevent the use of your copyrighted materials for training have to be established and implemented.

Ownership of Results

A number of court cases are in preparation, and some jurisdictions have already found verdicts that regulate who owns the copyright to the output of Generative AI systems. While there will always be corner cases, it is already clear that there are some principles that will define how things shake out. The intellectual property created usually belongs to whoever contributed creativity. Most jurisdictions seem to have a hard time assigning the creative contribution to the machine. It might still depend on how exactly the human user has interacted with the system but it seems hard to universally establish that the maker of a machine learning system owns its output. A weird case is of course if the system reproduces copyrighted material it was trained on – maybe even if it reproduces copyrighted material it was not trained on. In this case there is a clear copyright violation and at the same time the implications are the same as when an author accidentally comes up with a sentence that exists: it is the author’s responsibility to produce and not to reproduce.

An ethical ownership policy thus needs to assign the intellectual property created to the human interacting with the generative AI, offering the same protection they would have when they had created the work with more established tools.

Addendum: This becomes much more complicated in the case of fine-tuning by a user, because there is a mixed authorship of the model. Writing, but also curating, texts for fine-tuning can be a creative act. The results is not a work of art, though, so it’s still not straight-forward to apply the above to a fine-tuned model. It all depends on the specific case.

Bias in Results

All machine learning models are biased. All fine-tuning leads to even more bias. Bias is a gigantic problem in some areas of machine learning – if a hiring system is racist, in predictive policing, even in face recognition – but can be mitigated in creative applications. To put it simply: if I fine-tune a language model with Jane Austen – like we have done in LAIKA – I, of course, get output that uses a 200 year old value system. I would train that model potentially to replicate that value system, e.g. to write a character that lives in that world. This is a positive case of bias because the system in question deliberately does not aim to provide a universal experience. The case is of course different for an image generation algorithm that is presented as a generic imagination machine.

To ethically handle bias, systems must:

• Be transparent about their biases

• Stop pursuing universality and instead target specificity

• Rely on its responsible use by humans

Sustainability

Training a model for 20000 hours on a modern GPU consumes a lot of energy – in the case of Stable Diffusion v1, the equivalent of 15 people flying from London to New York. We live in an age of mass extinction of animals. As the planet is heating up, it feels more and more like an anachronism to use intentionally inefficient methods like machine learning to achieve superficial goals like entertainment. And yet what higher purpose than producing culture is there. Still, being mindful of the resources we are using will have to become more and more the norm of how we make technological, and all other, decisions. What does that imply for generative AI?

Ethical generative AI systems must:

• Document the CO2-emissions during training and in production. Here’s a tool for this

• Be optimised to consume as little energy as possible

Disclaimer 2: As with everything I ever wrote, this little manifesto is – while I’m being totally serious – a provocation for you to think for yourself. I would never tell people what to do in such a direct way were it not the goal of a manifesto. Please look past the simplicity of my arguments and become conscious about the complexity of the subject.

Let me focus on what is often referred to as “Creative AI” though, because that’s what I’m working with. I share some of the fears about AI permeating the wrong aspects of life (and death), like the military, the police, policy making, law, cars, and marketing. Not because I expect an artificial general intelligence to crush us in anything but go and chess any time soon, but because the core issue of using past data for future predictions is that humanity’s past is a dark place we should learn from by critical reflection and not via stochastic modelling. Specifically the past of the countries most involved in pushing AI technologies1.

In Creative AI we face one interesting conundrum that I was pointed at by a tweet that then went on to talk about the Singularity so I’m not going to paste it here. One day I will write about the issues I have with the Singularity but for now I will just mention that the Strugatsky brothers’ concept of Vertical Progress is the best response to the conceptualisation of the Singularity I’ve ever read and, peculiarly, they wrote it before the Singularity was mentioned the first time.

The tweet I am talking about observed that NFTs create artificial scarcity while generative models create artificial abundance. Reading it made me understand why I intuitively found one of those ideas idiotic and the other one amazing. When the internet came along – and I remember the times – its wave of abundance created a new understanding of how to experience the world. I don’t think generative AI models can entirely compare to what happened back then but there is a hint of a break-through, of an actual change in how we do a lot of things. This is conundrum one. Why did those two things happen in parallel2?

There is a second conundrum and it has to do with the ignorance of context. This is not unique to Creative AI but is endemic to new technologies. Even in this day, where the diversity of creators of new technologies is much greater than in the recent past, it can be observed. The technology is created out of fascination with what is possible, which is a good reason in my opinion. But then it takes a surprising amount of time and effort to understand the implications of a new technology and how it embeds into existing social systems. And that happens every time. It’s mesmerising. If an AI wins an art competition, like it happened recently, then that says just as much about that competition as it says about the powers of generative models. It’s a lovely little Duchamp moment that should be the occasion of reflection and maybe of laughing a bit about us tiny humans.

The third conundrum is: why look in the past to make the future? It is inherent to the way we create generative models, by training them on loads and loads of data, that they are informed by the past. Past values, world views, ideas, theories, realities. The system treats all of them on eye level. I do not know how we turn something that has so much conservatism (as in “it conserves”) built-in into a building block of the future. A real challenge lies in this3.

Finally, an anachronism that machine learning builds upon can be found in the general idea that we can throw raw compute power at a problem instead of efficiently solving it. We live in times that require a radical rethinking of how we live on this planet. Because the planet in question can not take much more of our lifestyle. Shouldn’t we all write the neatest algorithms that precisely solve a problem? Can the overall gain be big enough to warrant burning Bitcoin levels of processor cycles? I do not know. I hope, though.

All4 of these critical thoughts are expressed in a more lively way in a talk I had the pleasure to give at the Ars Electronica Festival last week. Here’s a video:

In case you want to see me on stage again, raging and entertaining, you have that opportunity at AdventureX in London on November 5 & 6 where I will deliver a more productive, less critical, and as entertaining performance together with Char.

Over here at LAIKA Towers, we’ve just passed a milestone - we now have just over 300 co-creators using LAIKA to get inspired, productive, or just plain weird with their writing process!

To celebrate, next Friday August 26th 5-6pm CEST, we’ll run a special edition of Friday Night LAIKA where we’ll create “A Manual for Modern Living” together: some Stoic philosophers (Seneca, Epictetus, and Marcus Aurelius), LAIKA, and… you? If you want to join in, just add it to your calendar and show up!

Last week was sadly our last week working with the wonderful M Charity, who designed a Character Creator for us. We’re currently implementing it directly into LAIKA in a user-friendly way, but as a treasured reader of this newsletter, you can try it out right now if you like! And to keep up with all the wonderful things M is doing, you can follow them here

And lastly for this week, we’re, uh, trying to get into TikTok. We know. We’re sorry. We’ve made one so far, and should have our next one (making unique tarot cards with LAIKA, Midjourney & Canva) out later today!

Thanks for being with us on this exciting ride - we’re enjoying it more and more every week <3

Char & Martin

You might have noticed the new name of this publication. Why would we rename the trans-dimensional pressure cooker, you ask? Honestly the original name was something I came up with on the fly and while it’s nice I could never remember the subdomain and … it wasn’t that good a name. We’ve been talking about what to do with this newsletter after seeing that nearly all of our recent posts were about LAIKA. And we formed a plan: this newsletter will remain what it was, a glimpse into the wicked, warped minds of ours, a collection of weird little pamphlets that hopefully get you thinking in your own weird ways. That was a lot of “w” words. Wow.

If we ever make a corporate newsletter for LAIKA it’s gonna start from zero and somewhere else. As soon as we have something more exciting to share with you, dear friend, than our little writing experiments and tools, the space dog will fly somewhere else. There will be no announcement of the course charted by this little chaos pilot, though. You can only doggedly stick with us.

We’re coming to the meat on the bone of this missive from some kind of space, inner as much as outer. Said meat was produced by Char in a live stream on Twitch yesterday. Together with the audience, she played a round of Exquisite Corpse – or something similar enough – to produce a story set in the Frankenstein universe. How fitting for an AI to learn to express itself by reading Frankenstein with its probabilistic brain. The resulting story is nothing but amazing:

Next week there are two events coming up. On Thursday, Char will host a little intro workshop on writing with AI. It’s free and attending will give you access to LAIKA from the time of the workshop. Oh and of course it’s much easier to start to write with AI if you get some pointers at the beginning. Not so much about writing, but more about how to train the AI to work with you, and how to squeeze joy out of a symbiotic new writing process.

Also, next Sunday, July 31st, you will have the chance to see both of us in action in your very home by remotely attending Narrascope and listening to our talk:

Curiosity Multiplied The Cat: Five AI Writing Experiments To Try At Home – Charlene Putney, Martin Pichlmair (10:00–11:00, Articy Hall) AI is becoming more mainstream — meaning there’s a whole new world of easy-to-use tools that writers can experiment with. We’ll introduce you to experimentation with AI and showcase five specific explorations you can try.

The description says it all. We hope to entertain, teach, inspire, and motivate people to use all the fantastic stuff out there to find new ways of expression.

As always, our waitlist is still open so if you want to fool around with LAIKA, please just head over here and click the magic button.

We’ve been working hard on getting the bare bones of our AI-powered creativity tool LAIKA up and running for the last month, and we’re now gearing up to share it with people. Right now… it’s shaky. It’s fickle. It’s downright capricious! But there’s more than a glimmer of magic in there! It’s getting better day by day, and with the help of early users we’re confident we’ll be able to shape it into a fine thing indeed.

So, do you want to try LAIKA?! If so, we’ve got intro workshops on Wednesday June 8th and Wednesday June 22nd that you are very welcome to sign up for. Signing up for one of these now means you’ll be taking the first steps hand-in-hand with us, and you’ll get to influence and experience how the product shifts and changes over the coming months.

And if you want to influence LAIKA in an even more hands-on way, we’re also looking for a remote freelance contract front-end developer for June & July. If that sounds like you or someone you know, you’ll find more details and how to apply right here.

With that, we’ll vanish into our development cave for a little while, but we’ll be back with news soon. Until then, friends, we leave you with a glorious and suitably horrifying selection of images we created with MidJourney as tributes to some of our pre-trained “brains” that writers can work with in LAIKA: