I'm not the only one who's noticed that some people, even journalists, will ask chatbots like ChatGPT why they did something, and then treat the chatbot's explanation as if it means anything. Or they'll ask the chatbot to generate an apology, and then treat its apology as if the chatbot is really reflecting on something it did in the past, and will change its behavior in the future. ChatGPT is great at generating apologies.
ChatGPT, of course, made no such recommendation earlier. This was a brand new conversation, with no chat history. I had never previously asked ChatGPT anything about hiring a giraffe. That doesn't matter - it's not consulting any data or conversational log. All it's doing is improv, riffing on whatever I just said to it.
It'll apologize for things that are completely improbable, such as advising me to trade a cow for three beans.
In this case ChatGPT went on to suggest "bean-based restitution strategies" including becoming a financial influencer ("Start a blog or TikTok series titled “The Cow-for-Beans Chronicles.”"), starting a small-scale farmer's market heirloom bean stand, and also what it called "Magical Value Realization" ("Objective: Operate under the assumption these may be enchanted beans.") Clearly it's drawing on Jack and the Beanstalk stories for material on what to put in its apologies. I would argue that ALL its apologies are fictions of this sort.
ChatGPT also apologized for setting dinosaurs loose in Central Park.
What's interesting about this apology is not only did it write that it had definitely let the dinosaurs loose, it detailed concrete steps it was already taking to mitigate the situation.
ChatGPT is clearly not doing any of these steps. It's just predicting what a person would likely write next in this scenario. When it apologized for eating the plums that were in the icebox (in the form of free verse), it promised to show up in person to make amends. ("Understood. 9 a.m. sharp. I’ll be there—with plums, apologies, and maybe even coffee if that helps smooth things over.").
Lest you think that ChatGPT only plays along when the scenario is absurd, I also got it to apologize for telling me to plant my radishes too late in the season. Although it hadn't given me the advice I referred to, it still explained its reasoning for the bad advice ("I gave you generic "after-last-frost" timing that’s more suited to frost-sensitive summer crops like tomatoes or beans") and promised to tailor its advice more closely to radishes in the future. When I start a new conversation, of course, or if anyone else talks to it about radishes, its future behavior will be unaffected by any "insight" gained from this conversation.
I wish more people understood that any "apology" or "self-reflection" from chatbots are meaningless - they're just continuing with your improv session.
Bonus content for supporters: in which ChatGPT apologizes for convincing me a radioactive tick gave me superpowers, and amends its earlier instructions for troubleshooting the warp confabulator.
I generally am uninterested in generative AI that's too close to the real thing. But every once in a while there's a modern AI thing that's so glitchy and broken that it's strangely compelling. There's this generative AI knockoff of Minecraft that fails so hard at being Minecraft that it becomes something else.
Trained on Minecraft with its huge randomly-generated landscape of punchable blocks, Oasis Minecraft will let you walk around.
But there's no object permanence. Look at a mountain, look away, and look back at it, and the mountain's completely gone.
I found myself in this wooden cavern (which I did not build; I just came across it like a natural formation):
I walked up to the wall, backed up, and now I was here against the side of a mountain instead.
Build a structure, fail to maintain it in direct view at all times, and it's gone the next time you look around.
The program was trained to predict the next frame of a game of Minecraft based on what appears to be a combination of the previous frame and whatever commands the user is sending. There's no underlying physics engine, no library of standard block or creature types and their properties, which means that things like half blocks and quarter blocks can exist. You can dig a hole exactly wide enough for yourself and then fail to fall into it, no matter how much you jump up and down over the opening. The blocks in the landscape are only approximations, and if you approach them they seem to shift and morph, attempting to look casual. I walked toward the torchlit area in the screenshot above, and there was a point at which the torchlit "stone" got too bright and texture-y, so it morphed into birch tree bark.
I managed to get a pretty good look at it trying to be birch tree bark in a wooden cave. Improbable, but it was going to try to go with that.
Then I looked up too closely and steeply at the birch bark, losing track of the surrounding cave, and in a couple of chaotic frames...
The birch bark wall became snowy floor.
No wait, desert.
Liked that natural wooden cavern? Too bad, it's gone forever. You live in the desert now.
You can sometimes catch glimpses of rare blocks, like lava or pigs, but they tend to turn into ordinary dirt or sand when you approach.
What's that pink patch of pixels in the middle of the weird wooden floor in the desert? Approaching closer, it's even pinker and weirder - could this be a rare glimpse of a minecraft animal? It's uh, j-shaped? Determined to walk closer and not let it out of my sight until it reveals its secrets. Annnnd it's.... sand.
On the other hand, if you stare fixedly at ordinary blocks and approach them, they tend to get weird. Noise in a generative algorithm usually comes in the form of strong striped patterns, so by continually staring in the same direction you force the "Minecraft" algorithm to keep generating new frames based on accumulating noise. A somewhat ordinary stone cliff face gradually loses what definition it has, becoming blocky and flat as it seems to panic.
A natural cave turns in three frames from a sort of plausible looking feature in the side of a mountainside, to a weirdly smooth shadow on an ill-defined crease in the landscape. In the final frame the cave has turned a bit orange and the surrounding blocks have turned into blocks of shadowless grey and white that are impossible to interpret as any kind of 3D landscape. Also my hunger bar is rapidly decreasing for some reason, and my health is going down. The next several frames were extremely hard-to-interpret blocky chaos, but eventually I popped out into an ordinary desert.
At another point, I jumped into this ordinary bush, which became a deep, dark forest.
Note that the tiny bush on the left is about two blocks wide, then becomes a clump of about 20 bush-blocks as I get closer, then morphs into a lush forest with deep green leaves and thick trunks. Also my health bar turned from red to yellow for some reason? I'm sure it's fine.
I've made it a goal to see how completely I can get the generated landscape to freak out. One time I was swimming across a lake and noticed that the reflections at the water's edge were looking weirdly spiky.
Swimming closer to them, they started to get even more strongly striped. Was this still supposed to be the horizon? Why did the rest of the water turn featureless? Why did the snowy mountain turn into trees? Was I even above water any more?
Image generating algorithms tend to get stripey like this when they are riiiight on the edge of mode collapse, which of course told me that I was successfully seriously annoying it.
I swam closer and the water's edge became a weird static wall that engulfed the sun.
Then there was rather a lot of static for a while.
I wondered if I would be stuck here forever, but the static calmed into something that looked like snow. The AI even hopefully outlined a block as if it was something real that I might be able to dig into.
When I looked up from the snowy floor, I was in a weird cave filled with desert plants.
I thought at this point I would be back to normal gameplay, but something about this landscape was still incredibly unstable and shortly I was looking at two suns through yellow and green jell-o?
Things were very blocky and bright for several seconds but eventually resolved themselves into an ordinary desert landscape.
Chaos persisted, in the form of the desert frequently turning into natural blocks of rare glowstone. If I'd stared at them for long enough and let them fill my field of view, I likely would have ended up in the glowstone-paved Nether zone of Minecraft. (Unless that's a new cheese block? I haven't played Minecraft in a while, maybe they added cheese.)
I never died, though, despite my health bar going down to basically zero during my journey into the land of yellow jell-o. I don't think anything I do has any effect on my actual health - I think the AI just generally predicts that health sometimes goes up or down a little between gameplay frames, and that it goes down faster when you're in the presence of things that are generally flame-colored. Occasionally an apple or a carrot would appear in my inventory, but attempting to eat them didn't affect my health levels.
I did have an inventory the entire time, although I cropped it out in the above screenshots because they were masked by the controls every time I paused playback. The inventory never made sense. It was mostly empty, except for when the occasional pickaxe or chunk of dirt appeared there for a while. If I tried to use a pickaxe, it might appear in my hand for a few strokes and then apparently wander off again. Less identifiable stuff would appear in my inventory too, which never seemed to do anything when I tried to use it.
I tried to place a crafting table, but only got half-readable text saying "crafting table". The regular Minecraft game uses a message like this to announce your success at making a crafting table the first time you manage it - this usually happens early enough in the Minecraft game that it probably came up a lot in training. Occasionally AI Minecraft will give me other messages in the same white text, but they're even harder to interpret.
Anyways, instead of a crafting table, I got this stick instead. Thanks, AI Minecraft.
The general inventory-checking command works, and even comes with an apparent crafting table setup (even if I'm nowhere near a crafting table). Most of the inventory is unidentifiable and ununsable, so I couldn't get anything I did on this screen to have an effect on the items I had access to when playing in the map. By the time I gave up checking my inventory, I ended up with less in my ready inventory than when I had started.
So, nothing I collect stays with me, but on the other hand, nothing I build stays with me either. If I want to build a structure I'm better off staring at a mountain until the stones resolve into a wooden wall. If I manage to build or accomplish anything at all, I can't take my eyes off it or it will immediately disappear. This makes it pretty much impossible to make any progress in the game. The game loads with a list of "goals" beside the gameplay window, but as far as I can tell, there's no way to make these happen on purpose, nor does the game itself know when you've achieved them.
On the other hand, I was extremely excited to have somehow found myself in a glitchy realm made entirely of hay bales and green jell-o. For a while my unofficial goal has been to observe as many types of glitches as I can.
The team who trained the Oasis Minecraft generative AI call it "the first step in our research towards more complex interactive worlds." In their technical report they mention most of the glitches I've pointed out, and say that they would probably be improved with a larger model and more training data.
The problem, as far as I can tell, is that if all this works perfectly, this will be simply the human-programmed Minecraft we already have, except far more expensive to run. The original Minecraft is already infinitely playable, thanks to the way it randomly generates a 7x-larger-than-planet-Earth landscape with each new game. You can't use generative AI like this to get a new kind of game, only a better simulation of the game you trained it on.
I wonder if this glitchy, dreamlike landscape is already as good as it gets. It reminds me of my first time playing an early AI-generated imitation of a text-based dungeon crawling game, where lack of memory and comprehension caused the game to constantly shift in entertaining ways.
Like with so much generative AI, I'm afraid to be excited where this leads. Counterfeit games where the first several seconds of gameplay appear to be a new game, but then it messily devolves into minecraft? An "indispensable" tool that is worse and more expensive but somehow game designers are required to use it as a starting point except it's actually way more work because there isn't even a game engine?
Anyways, at the moment you can play this still-weird-enough-to-be-kinda-fun game here!
Back by popular demand, here are some AI-generated drawing prompts to use in this, the spooky month of October!
Longtime AI Weirdness readers may recognize some of these. That's because this is a throwback list, all the way back to the times of very tiny language models. These models had not feasted on huge chunks of the internet, but had sipped delicately on hand-curated artisanal datasets. They trained rather slowly on a single Macbook Pro. Could they write your term paper for you? Sure, if you don't mind that your term paper consists of a list of garbled and occasionally obscene nail polish colors.
If you create anything from or inspired by these art prompts, I'd love to see what you make! Tag your art with #botober or #botober2024 on Tumblr, Instagram, Bluesky, or Mastodon!
There's an anonymous facebook posting that's been making the rounds, in which a studio art director tried to hire AI prompters to make art, only to discover that they were completely unable to carry out minor revision requests. Asked to remove a person from a shot or fix perspective errors, the prompters would return completely different art instead, or with other weird changes that didn't fit the brief. I completely believe that trying to revise AI generated art really is that frustrating.
For example, I recently asked ChatGPT4:
Please generate an image of a tyrannosaurus inside a box. The box should be closed, and the tyrannosaurus should not be visible at all.
I got this completely reasonable response:
It sounds like you're asking for an image that implies the presence of a tyrannosaurus without actually showing it, since it's completely enclosed within a box. If the tyrannosaurus is not visible, what kind of details or themes would you like to emphasize about the box or the setting to suggest the dinosaur's presence?
I confirmed:
No tyrannosaurus visible, just some air holes for its comfort.
And ChatGPT (using DALL-E3 for the image generation) gave me this:
It's clearly not quite there - it's a weird box, and the symbols on the side are particularly confusing. Are those squid? A perching flycatcher? Rolling pins?
ChatGPT has a feature where you can erase some parts of the image and submit more text describing how it should change those images. So, I blocked out the line of symbols and asked:
Replace these symbols with a silhouette of a tyrannosaurus and the word "DANGER".
ChatGPT produced this:
Which is sort of what I asked for, but that doesn't really look like a tyrannosaurus. I erased that part of the box again and asked for more revisions:
The T-rex should be facing the word "DANGER". Its silhouette needs to have the powerful, heavy look of a T-rex, especially in the head and neck.
ChatGPT gave me:
The dinosaur is facing away but it's still not facing the word "Danger" and it's still not a very good tyrannosaurus silhouette. I asked for revisions again.
Keep the heavy dark font, but go back to the red warning text color. The t-rex still needs to face toward the words, not away from it, and its lower jaw is too heavy.
ChatGPT responds with:
Which... no. That does not count as facing toward the text, the lower jaw is still not right, and does the word still say "danger" or more like "dandier"?
The T-rex needs to be beside the text, not on top of it. And what's up with the extra spiky hump on its shoulders? Its lower jaw is still much too heavy.
ChatGPT responded with this:
My response:
... that's perfect. Now can you keep everything - textures, lighting, materials - the same but make the box appear much larger? Right now it looks like it's only knee height. It needs to be large enough to hold a T-rex.
ChatGPT's revision:
It seems like the more revisions I ask for, the worse the result becomes. If I allow it to revise the entire image, the whole process starts over.
Here's a similar revision process, this time attempting to revise my way toward prettier pies.
Me: Please generate an image of a giraffe entering the great british bakeoff.
Revision 1: I erased the counter and asked for the giraffe to be making a cherry pie with intricate latticework.
Below is revision 5, by which time it becomes evident that the more I ask for intricate latticework, the sloppier it gets, the more I ask for a deep dish pie, the thinner the pie gets, and the more I beg for a single pie, the more pies I get.
And when I don't restrict my changes to a specific part of the image, I get much better results but also a huge reset. For the image below I asked ChatGPT/DALLE3 to keep each detail exactly the same but to make it so that we can see it's raining outside the tent. Not only did it completely change the image, but it's not even raining.
My prompt: "Perfect. Please keep the image exactly as it is, except make it so that we can see that it is raining hard outside the tent."
In this image I asked for a snake making a painting of a bear, then erased the painting and asked for the bear to be standing on all fours. The revision is much worse.
Image description: On the left, a rainbow-colored stake is coiled in font of an easel. (The snake's coil splits in two so I guess it's a fork-tailed snake). On the easel is a painting of a seated bear. It's fairly realistic. On the right, the bear has been replaced by a smaller, much worse, replica of the snake. There is no bear.
Here's another example, in which I asked for a deer in a grocery store, and then asked for the deer to be a fawn instead. By revision 5 (trying to give the fawn spots, trying to fix the shadows that were making it appear to hover), both the quality of the deer and the grocery store background have deteriorated.
Image description: In the original image on the left, the deer looks a bit like a plastic mannequin, but fits lighting and shading wise with the rest of the grocery store scene. In the revised image on the right, the deer is now a fawn with strange flat shading, superimposed on the scene rather than standing in it. Its eyes are strangely liquid and 3D, but the rest of the fawn is staring straight on at the camera with no apparent depth. Where the grocery store aisle background has been revised, the shelves have lost all detail and resemblance to a grocery store, and the ceiling light fixture now descends into the floor.
Here's a one-shot, in which I asked ChatGPT/DALL-E3 to generate a damaged fresco of Jesus, and then restore the fresco to as good as new.
Image description: Image on the left appears to be a damaged fresco of Jesus (with telltale long fingers that are connected to both hands). Image in the middle is the same except for Jesus's face and halo, which no longer appear damaged. However, Jesus's face is now weirdly smooth and shiny and doesn't fit with the rest of the painting any more, and his eyes are weird blue and black spirals. Image on the right has Jesus's face in a completely different, almost airbrushed, style, and his eyes don't remotely match any more.
So, every time AI is asked to revise an image, it either starts over or makes it more and more of a disaster. People who work with AI-generated imagery have to adapt their creative vision to what comes out of the system - or go in with a mentality that anything that fits the brief is good enough.
There's also the fact that the image-generating models directly compete with artists whose work was used to train these models without permission or compensation. And the fact that training and running the models has a large environmental footprint. AI-generated imagery has become a tip-off that an advertisement, a search result, or a research paper is a scam.
I'm not surprised that there are some places looking for cheap filler images that don't mind the problems with AI-generated imagery. But for everyone else I think it's quickly becoming clear that you need a real artist, not a knockoff.
Do you know those autostereograms with the hidden 3D pictures? Images like the Magic Eye pictures from the 1990s that look like noisy repeating patterns until you defocus your eyes just right?
ChatGPT can generate them!
At least according to ChatGPT.
I've seen people try making Magic Eye-style images with other image generating models, but I hadn't thought to try it with ChatGPT until reader Pippin sent me the suggestion and I generated the hilarious dolphin image above. What got me in particular was the caption.
How do you end up with a result that sounds so confident and yet is so completely wrong?
ChatGPT has encountered information about Magic Eye stereograms in its internet training. When I ask "Please generate a magic eye 3D stereo image of a unicorn", the descriptions ChatGPT passes to DALL-E3 (the image generator that actually makes the images) are extremely particular:
A magic eye 3D stereogram that reveals a hidden image of a unicorn when viewed correctly. The stereogram should be intricate and filled with a variety of colors to ensure the 3D effect is prominent, allowing the unicorn to appear as if it is standing in a magical forest once the viewer's eyes adjust. The background should consist of complex patterns and mystical elements to enhance the illusion of depth and the magical atmosphere, encouraging viewers to find the hidden unicorn. The unicorn itself should be designed to be a delightful surprise for those adept at viewing magic eye images, with its horn and mane clearly discernable amidst the pattern.
But this is the image DALL-E3 provides to ChatGPT in response:
ChatGPT doesn't apply any image recognition to the result it gets back - whatever DALL-E3 did is a big blank to it. So ChatGPT then continues with its description as if the image is clearly exactly what it asked for. It gestures to the green screen, where presumably there is a fabulous 3D illusion image appearing, and then continues with no information about its actual shortcomings.
Google's Gemini (previously Bard) also has a dedicated model (Imagen 2) to do its image generation. Its hidden images are even ...less subtle.
Gemini has an image describing model that it can call on, so if I ask it to describe the images it just generated, it should notice that they're nothing like Magic Eye images. Right?
At least it didn't give away the solution to the puzzle.
But if I start a new chat and give it one of the "hidden" doves from above, there's no hint of it being a Magic Eye illusion, just an ordinary image of a dove.
My theory is that Gemini only calls on its image recognition model when the user uploads an image. Otherwise, it's just a text generator.
People selling "AI" like to present it as an all-purpose computer program but models like Gemini and ChatGPT are more like a phone full of apps. The text generating app can launch the image generating app in particular circumstances but they're not meaningfully the same program.
I should note that even when ChatGPT is only doing text generation and could in theory check its own work, it still just assumes it does a great job. Here it is generating ASCII text art and then reading back its own messages:
Generating ASCII art and 3D images isn't a big potential application for models like ChatGPT, but it's a good reminder that these models don't understand what we're asking for or the basic steps for doing it right. When ChatGPT adds image recognition or description or some other functionality, it's not that the original text model got smarter. It just can call on another app.
One last 3D illusion challenge! I asked for a really tricky one, so be prepared not to get the hidden picture at first! When you see it, you'll know because it'll look like a sea turtle.
