Neural networks vs the Bake-off technical challenge
There's this baking competition I really like, and one of the elements in every show is what they call the Technical Challenge.
In the Technical Challenge, Great British Bakeoff contestants have to bake something they may never have seen before, based solely on a brief description and a very sparse set of instructions.
I decided to give two programs the technical challenge. Both generate images from my descriptions by using feedback from an algorithm called CLIP to improve their images. CLIP gives feedback based on images it's seen online during training. (More on diffusion-guided CLIP, and CLIP+VQGAN here, as well a tutorial on how to use them for free.)
Let's see how our bakers did!
Challenge 1 (Season 4, episode 2):
Description given to the neural nets: Mary Berry’s Viennese Whirls | Push the boat out with these melt-in-the-mouth biscuits filled with homemade raspberry jam and vanilla buttercream.
Diffusion-guided CLIP:
A bit of a swirl there, nice flavor on the raspberry jam. It seemed this baker was a bit confused and managed somehow to spread molten biscuit between layers of raspberry. Could be neater, and the buttercream seems to have dried into shards. Good effort.
CLIP+VQGAN:
This baker has spent a lot of time on presentation, but the bake itself is perhaps not quite so good. The raspberry jam is a bit dry and thin - I don't know how it came out quite so dark. Perhaps they overcooked the raspberry. I do see a bit of a swirl - good definition there from the bake. I'm not sure I agree with the way they chose to make everything look like rows of jagged teeth.
Challenge 2: Savarin with Chantilly Cream (Season 4, Episode 9)
Description given to the neural nets: Savarin with Chantilly Cream | Not just another teatime sponge, this citrus-flavored yeasted cake is soaked in orange liqueur syrup and crowned with Chantilly cream and fresh fruit.
Diffusion-guided CLIP:
This cake has lost some definition in the bake. Good color, though, and plenty of orange liqueur syrup. What happened to the fresh fruit? Presentation is lacking a bit on this one.
CLIP+VQGAN:
This baker has made an effort on the plating - that gold really does make it look special and festive, although I'm not as sure about the broccoli. Plenty of chantilly cream, which I like. The orange slices are there, although it could be a bit neater. I would have left off the shark fins - they don't add to the bake and make it look rather a bit spooky.
Challenge 3: Choux Buns Religieuse
Description given to the neural nets: Choux Buns Religieuse | A stack of custard-filled choux buns, dipped in chocolate and arranged to look like nuns
Diffusion-guided CLIP:
Uh-oh, where are this baker's buns? This is quite a shame. If we can't see a baker's buns, we can't judge them.
CLIP+VQGAN:
Very good effort, if slightly terrifying for reasons we can't exactly pinpoint. We have the choux pastry, we have plenty of cream, and the chocolate looks well-tempered and glossy. We also have a few other things in here that I can't quite identify and am not sure I should sample. Nice golden color on the choux pastry.
And the winner of the neural net technical baking challenge is....
....
.... CLIP+VQGAN!
You presented something for every challenge, in all cases quite a lot of something, and rather a few extra unidentifiable somethings. Please take them all away with you.
Bonus content: a series of technical challenges including floating islands, spanische windtorte, and Mary Berry's famous Victoria sponge cake.