AI cake fails

AI cake fails are unlike any cake fails I've seen before. What do you get when your cake is being generated by something that's seen lots of pictures labeled "cake" but has never had a cake or experienced physics?

CLIP+VQGAN are two algorithms that, when teamed up, take turns generating images and judging whether they match a prompt, based on what images appear with what text in online training data.

Here's "Unicorn cake with rainbow sprinkles" as generated by CLIP+VQGAN (tutorial for this version here).

The image looks simultaneously like cake, unicorn, and sprinkles, which is technically what I asked for. I didn't specify whether I wanted the inside or the outside of the cake, so I kind of got both.

Randall Munroe, who put me up to generating cakes, suggested I try specifying a wedding cake since people tend not to post pictures of the insides of those. It helped - I got a similar level of chaos but at least less inside-out cake. What also helped was to give it a sketch of a single cake layer as a starting image, and to give it a negative penalty for anything resembling "slice".

Here's "unicorn cake with golden horn and rainbow sprinkles" with a negative penalty for "slice".

It still managed to show cake interior but now it's in a glass of some sort and the cake has sprouted a unicorn head that is ...drinking out of the glass?

I wanted to generate cakes that are at least physically possible, if highly improbable. So I talked with bearsharktopusdev who helped me optimize their Zoetrope version of CLIP+VQGAN for cake generation (and they have a Patreon where you can help them develop new bells and whistles). With the ability to control noise and optimization method, the ability to turn the learning rate way down (so the AI only takes small, careful steps away from the default plain cake), and experimentation that indicated that the best prompts for getting a showcase cake would include the words "food photography" and "by Janelle's bakery" (or anyone's bakery, really), I got this cake:

It's a technically impressive cake, I'll give it that. The rainbow interior reveal is dramatic and I didn't even know you COULD get fondant to look like the underside of an albino alligator's belly.

I decided to give the AI another classic cake challenge: a galaxy glaze cake. They're tricky to get right, but ideally a galaxy glaze cake would have a glossy dark mirror finish patterned with stars and nebulae. Here's what I got:

I also tried requesting a classic hedgehog cake:

The AI, despite having seen many classic cakes online, doesn't seem to reach for this knowledge when asked for cakes. There are many cute mushroom cake designs online. Then there's the AI-generated version.

"Golden retriever" cake uses an impressive melting fondant effect and I appreciate the bits of gold leaf it added for decoration.

I tried a "neural network" cake but it was pretty horrifying.

It can also do movie-themed cakes. Here's a Frozen-themed cake.

I'm not sure what's up with the eyeball, or the brussels sprouts. One of the problems with doing a themed cake is, unlike a human baker who might pick one aspect of the theme and shape the cake like that, the AI tries to do ALL of them at once. Its "Star Wars themed" cake was so much of a mess that it looked less like a cake than a themed pile of junk.

My attempts to focus its efforts on "Kylo Ren themed cake" proved ineffectual, as the resulting cake was still very busy. As bearsharktopusdev put it, it's not so much a cake about Kylo Ren as it is a cake made of Kylo Ren.

The AI seems not to be able to leave well enough alone. Here's "Cyperpunk neon" cake which I stopped fairly early on in generation. It's weird, but it's at least a coherent cake.

Here's the "Cyberpunk neon" cake later.

It's possible to reduce the chaos by tweaking the noise settings, though I'm not sure the resulting cake is any more bakeable:

In image generation as in text generation, AI excels at details and at overall vibes. Understanding of reality, such as physics of actual cake, is not AI's strong point.

After all, none of its cakes are real. They don't have to be obey physics or be edible - it wasn't rewarded for that during training. If it can convince itself that this is cake, that's good enough.

Bonus content: more of my favorite cake fails, including Thomas the Tank Engine, giraffe cake, and "trout".