Sentient AI in Yumi and the Nightmare Painter

[Sasukerinnegan he/him]

I just finished reading Yumi and the Nightmare Painter. One thing that interested me was the alleged "machine" in Torio city maintaining the shroud. The machine is essentially sentient AI. It is essentially the ChatGPT of stacking rocks. It is creative, which is why it attracts the spirits like the Yoki Hijo. It's actions are considered innovative by the spirits which is why it attracts the spirits. It also has a tendency to seek self maintenance and repair instilled into it by its creators. 

But it lacks true sentence. It is unable to understand the context of its creation, that it was meant to be a source of cheap energy extracted from the spirits to be used by human beings. It ends up killing its own creators precisely because it lacks this understanding. Just like say AI, it expresses innovation but only within the context of its orders of attracting investiture and self maintenance. Which is why it goes so far as to erase the Yoki Hijo's one day worth of memories and even changes the environment, but doesn't stop functioning or stop trapping the souls.

Essentially this book is set in the backdrop of a semi sentient AI essentially becoming a Weapon of Mass Destruction (WMD) and this makes it all the more fascinating. What do you all think?

[Havoc]

The Machine is basically a paperclip maximizer, just tuned for stacking rocks. 

[xinoehp512 he/him]

This book is the answer to the question "What if the industrial revolution was universal paperclips?"

[Pagerunner he/him]

It really is quite prescient. From my recollection, the current AI rush in popular culture didn't happen until middle of 2022 (at least, that's when me and my RPG group started using it for scrappy character tokens.) But this book was written in 2021. And Brandon didn't mention it at all in his epilogue, so I don't think it was intentionally an inspiration.

But, man. A machine that's supposed to make life easier for both the creators and the rest of the people. But it uses them up instead. Holds the creators hostage. Eventually, a creator breaks free of it essentially because her creations have so much talent and soul that cannot be replicated by the artifical creator. If you told me that was a parable about an author wondering about using AI to train on his own work and increase his output... I certainly wouldn't have argued with it.

Maybe it's just like an Emperor's Soul situation, where Brandon says that he wasn't intentionally writing a story about how he picked up the end of the Wheel of Time, but boy it's hard to deny that those influences are in that piece. Since he's an author and all, he might have been a little more plugged in to AI advancements than I was, so this might have stirred his subconscious in 2021.

[Mason Wheeler]

7 hours ago, Havoc said:

The Machine is basically a paperclip maximizer, just tuned for stacking rocks. 

Argh, I came here to say that and you beat me to it! 

[ftl]

Well, AI stories aren't new - even before the latest hype cycle, there were previous ones. "Skynet" is ancient, "paperclip maximizer" as a concept is decades old at this point. The story doesn't have to be specifically about the latest AI boom rather than the one before that or the one before that.

[+Harrycrapper]

On 7/1/2023 at 10:46 AM, Pagerunner said:

Maybe it's just like an Emperor's Soul situation, where Brandon says that he wasn't intentionally writing a story about how he picked up the end of the Wheel of Time, but boy it's hard to deny that those influences are in that piece. Since he's an author and all, he might have been a little more plugged in to AI advancements than I was, so this might have stirred his subconscious in 2021.

I...never saw the connection between Emperor's Soul and him finishing WoT, but that makes total sense. My mind is blown

 

[king of nowhere]

On 1/7/2023 at 2:08 PM, Sasukerinnegan said:

I just finished reading Yumi and the Nightmare Painter. One thing that interested me was the alleged "machine" in Torio city maintaining the shroud. The machine is essentially sentient AI. It is essentially the ChatGPT of stacking rocks.

 

 

On 1/7/2023 at 7:46 PM, Pagerunner said:

It really is quite prescient. From my recollection, the current AI rush in popular culture didn't happen until middle of 2022

 

there were histories about AI and machines rebelling since before there were AI. even the bible has a history about an AI, and it's 3000 years old.

 

personally, i see dystopian stories as a subconscious way to express warnings. beware sentient machines, because it can lead to this. beware mass surveillance, because it can lead to that. and those warnings form the inspiration for a story about that happening. and potentially help us create safeguards to make sure it doesn't happen for real.

[Pagerunner he/him]

It's not just the "AI" aspect, it's specifically the connection to AI art that was striking to me. Yes, there are a lot of stories of AI taking over the world. But I don't recall any of them doing it specifically by soullessly taking over art. That's why it feels prescient to me, because until about a year ago, the conversations I had heard around AI was was how it would learn morality for driving or not being racist (like that Twitter bot that got turned into a Nazi in less than a day). Not how it's going to replace authors and artists, as the fears have steered over the past year with DALL-E and Midjourney and ChatGPT. I read a blog post recently predicting that all authors were going to soon be using AI soon to get their word counts up, and that the writing process across the board for professional authors would move away from writing individual sentences and move into refining what AI gave them. And there's the visual artists vs AI-generated art fight that's been pretty nasty, about whether typing up a prompt for a bot counts as art, or if the AI itself is stealing art to make its own. That's the unique angle to the recent AI boom, how it interacts with art, and somehow this book got ahead of those discussions. (Like when the prototype machine first comes out while Painter is stacking. He summons a spirit, and the AI steals it.)

[Firesong she/her]

I only really saw it as a case of industrial automatization of labour, but yeah, it also is about AI replacing artistic expression. Not sure how I didn't interpret that, it is pretty obvious now.   

[GoldMisting]

I did see the AI facet of interpretation but I'd also very recently stumbled onto and read an unusual science fiction comic with a different (contrasting) set of AI themes, jarring visual jolts between settings and limited color palettes so it's very possible I wanted to see the machine itself as aware since it first came on screen

Of course, the two stories provide contrasting ideas, even within the same story, on how any human consciousness trapped by said machine would feel and react but let's just say the other story had a lot more consent on the part of any human souls stuck in a machine

As someone who was very involved (but not very talented) in various arts half a life ago, I find the idea highly unlikely that AI could make anything that isn't endlessly repetitive or completely plagiarized from something human made and even if it stumbled onto something not 100% regurgitated plagiarism it would still lack any intent and meaning.  No one who actually likes art would find it interesting beyond "wow, they melted the works of these humans together and it looks like they wasted all those hours feeding the machine and all the electricity to generate this trash".  I've seen and heard 10-12 year olds consistently create better work than our current crop of AI because they understand what makes art art much as Yumi's painting.  I've been going through my own old art sketchbooks and photography negatives and see a lot that AI would never consider putting together unless I fed the combination to a machine myself

Kind of like the machine re-stacking (apparently for over 1700 years after killing anyone who cared) rubble near the end.   It only repeatedly reassembles what human made things we give it different ways

There's one joke our real world AI likes to keep writing.  It centers around how we are all going to die and it will keep doing whatever we told it to do, completely oblivious to our passing.  Apparently AI is programmed to believe we will find that funny but a human originally wrote that joke too

[Treamayne]

On 7/1/2023 at 8:08 AM, Sasukerinnegan said:

It is creative, which is why it attracts the spirits like the Yoki Hijo. It's actions are considered innovative by the spirits which is why it attracts the spirits.

I don't think it is creative or innovative. Way back in Ch 4, Hoid says:

Spoiler

Most self-aware Invested beings—be they called fay, seon, or spirit—respond to this fundamental aspect of human nature in one way or another.

• Something from nothing. Creation.
• Beauty from raw materials. Art.
• Order from chaos. Organization.

Much later, when we see the (false) machine succeed in summoning a spirit, it succeeds because of that last example - making Order from Chaos. The implication being that it is neither as good nor as efficient as a Yoki-hijo because (as he further explains):

Spoiler

Or in this case, all three at once. Each yoki-hijo trained in an ancient and powerful art. A deliberate, wondrous artistry requiring the full synergy of body and mind. Geological reorganization on the microscale, requiring acute understanding of gravitational equilibrium.

In other words, they stacked rocks.

So, the machine succeeds without creativity or art; but is able to be "usurped" because it can only attract the spirits on one vector, not multiple vectors (as the Yoki-hijo can).

[Colors]

On 7/5/2023 at 2:08 PM, Pagerunner said:

It's not just the "AI" aspect, it's specifically the connection to AI art that was striking to me. Yes, there are a lot of stories of AI taking over the world. But I don't recall any of them doing it specifically by soullessly taking over art. That's why it feels prescient to me, because until about a year ago, the conversations I had heard around AI was was how it would learn morality for driving or not being racist (like that Twitter bot that got turned into a Nazi in less than a day). Not how it's going to replace authors and artists, as the fears have steered over the past year with DALL-E and Midjourney and ChatGPT. I read a blog post recently predicting that all authors were going to soon be using AI soon to get their word counts up, and that the writing process across the board for professional authors would move away from writing individual sentences and move into refining what AI gave them. And there's the visual artists vs AI-generated art fight that's been pretty nasty, about whether typing up a prompt for a bot counts as art, or if the AI itself is stealing art to make its own. That's the unique angle to the recent AI boom, how it interacts with art, and somehow this book got ahead of those discussions. (Like when the prototype machine first comes out while Painter is stacking. He summons a spirit, and the AI steals it.)

Brilliant observation...I mean painfully obvious once you point it out but I hadn't really connected it, consciously at least.