This neural net knows your secret recipe
Could you look at a picture of food and write down the recipe?
There are people working on trying to train AI to do exactly this. Facebook recently made headway on this tough problem by realizing that there are so many possible recipes that it doesn’t make sense to try to match every picture to a recipe the recipe has seen before. What if someone makes a turnip, haggis, and mandarin orange pizza, and the AI hasn’t seen that recipe before? So Facebook trained an AI with one part that looks at a picture and tries to figure out the ingredients it sees. Then, another part of the AI tries to turn the ingredients into a title and directions.
They recently posted their trained model, and Jonathan Fly turned them into an easy-to-use colab notebook (only a web browser needed, and no coding experience). So of course I tried it out.
It worked pretty well on this picture of cake:
Title: Cherry Cake
Ingredients: sugar, flour, egg, salt, butter, extract, baking_soda, cinnamon, cherries, baking_powder
Instructions:
-Preheat oven to 350 degrees f (175 degrees c).
-Grease and flour one 9 inch round cake pan.
-In a large bowl, cream together butter and sugar.
-Beat in eggs one at a time, then beat in vanilla and cherries.
-Combine flour, baking powder, soda, and salt; stir into sugar mixture alternately with the buttermilk and cherries.
-Pour batter into prepared pan.
-In a small bowl, mix brown sugar and cinnamon for topping, sprinkle over batter.
-Bake in the preheated oven 25 to 30 minutes, until cake springs back when lightly touched.
-Let cool in 10 minutes before serving.
Now I don’t know for sure that this is the actual recipe that made the cake in the picture. What if I try it on a recipe where I do know the entire ingredients list?
This is “Chocolate Baked and Serves”, created by a recipe-generating neural net that I trained. It is the worst chocolate thing I have EVER tasted, thanks to the CUP OF HORSERADISH that the neural net threw in there at the end. (Some people claim it’s delicious but clearly they’re space aliens with a weird fondness for horseradish).
[photo credit: Jono Ellis, lover of horseradish and alleged human]
Title: Easy peanut butter brownies
Ingredients:
sugar, egg, butter, flour, salt, cocoa, extract, baking_powder, vanilla
Instructions:
-Melt chocolate, butter, and peanut butter together in microwave.
-Combine all other ingredients and pour into a greased 9x13 baking pan.
-Bake in 350-degree oven for about 20 minutes (until brownies appear a little underdone on top, but not gooey ! !
-).
-Makes 12.
It gets the peanut butter (but probably by chance, since the ingredient-identifying part of the AI didn’t list any peanut butter). But it is completely (blissfully) oblivious to the horseradish. Does it recognize brownie and then grab a generic brownie recipe that it’s seen before?
Here’s proof that the algorithm is generating the recipes on the fly rather than looking them up from a list of existing recipes. It figured out that it was supposed to be making easter eggs, but apparently didn’t know how and sort of panicked.
Title: Easter eggs
Ingredients:
egg, vinegar, food_coloring, pepper
Instructions:
-Place eggs in a saucepan and cover with cold water.
-Bring water to a boil and immediately remove from heat.
-Cover and let eggs stand in hot water for 10 to 12 minutes.
-Remove from hot water, cool, peel and slice.
-Place eggs in a bowl and cover with cold water.
-Let eggs stand in hot water for 10 to 12 minutes.
-Drain and rinse with cold water.
-Place eggs in a saucepan and cover with cold water.
-Bring water to a boil and immediately remove from heat.
-Cover and let eggs stand in hot water for 10 to 12 minutes.
-Remove from hot water, cool, peel and cut into ¼-inch slices.
Things get really quirky because of the two-part nature of this AI. Since one part generates its recipe from the list of ingredients that another part recognizes, rather than from the picture itself, it can suffer from pancake/waffle confusion. Seeing a picture of a pancake, the ingredient-recognizer turns it into pancake ingredients - flour, eggs, butter, etc. Then the 2nd part, not knowing what the 1st part saw, has to guess whether this is a pancake, waffle, scones, or what.
Sometimes the 1st part of the AI can really sabotage the 2nd part. When the ingredient-recognizing part misinterprets this bowl of pretzels, apricots, and a lemon slice as “lemon + bread + juice”, then recipe-generating part is stuck with the task of turning it into a recipe.
Title: Easy homemade lemon
Ingredients:
lemon, bread, juice
Instructions:
-Slice lemon.
-Dip bread into lemon juice to coat completely.
-Eat !
What if I give it something that’s not even food? It still generates a recipe, because “what the heck” is not in its list of responses.
Title: Homemade cleaner
Ingredients:
water
Instructions:
-Mix all ingredients in a bucket.
-Apply to clean silver dollar or silver dollar store.
Title: Super easy homemade water
Ingredients:
water
Instructions:
-Boil water add a little salt, add a couple teaspoons of good quality vanilla extract let it boil for 15 minutes
-Put in a jar let cool
-Enjoy with your favorite bread
AI’s never going to be perfect at this task - after all, you can hide anything at the bottom of a salad, and you can always frost a meatloaf like a cake. If humans can’t do the task perfectly in the first place, you won’t be able to train an AI to perfect performance by copying them.
You can try this out yourself with the colab notebook.
AI Weirdness supporters get bonus content: more recipes, including more recipes for Emperor Palpatine. Or become a free subscriber to get new AI Weirdness posts in your inbox!
