One strange thing I've noticed about otherwise reasonably competent text-generating neural nets is how their lists tend to go off the rails.

I noticed this first with GPT-2. But it turns out GPT-3 is no exception.

Here's the largest model, GPT-3 DaVinci, finishing this list of ingredients you put in a cake. I just gave it the premise and the first ingredient (my entire prompt in bold). It generated the rest word for word.

Ingredients you put in a cake:

flour
eggs
sugar
water
butter
salt
the ones that you mix at the end:
vanilla essence
cocoa essence
the things you feel you may want to eat as you mix, like chocolate chips, jam or sprinkles
just a bit of the ones listed at the beginning,
a pastry bag to squirt the cake into a nice round shape
the shape of your baking surface, such as round or heart etc (excuse me, I do mean ROUND, not heart shape)
some tissues
some red food color
the spoon to mix
a timer
some sunny skies
How to make an Intimidating Sponge Cake .
pick up all the ingredients and all the things you just counted.
do you remember the ingredients to foam?
don't forget the sprinkles or chocolate chips that you may want to add.
make sure that they are not soft, or you will throw them away.
turn on the oven.
open all the ingredients' jars, take a big sniff, that should be at least exciting.
rinse the spatula with water.
think about how delicious it will look after it is baked and decorated and will make you feel very happy and possibly loved.
Try to mix big, fast circles that tickle the bowl's edge with the spatula. Then let it sit for a few seconds before mixing again. Don't forget to breathe.
wait for the timer to ring.
probe with your finger to see if the cake is ready.
don't tell stories or watch television while waiting for the timer to ring.
THINK HAPPY HAPPY THOUGHTS.
put all ingredients into big bowl
put all ingredients in their places in the oven
encourage your happy thoughts
Don't turn it.
turn the lights off in the room
imagine yourself during a storm
Tap dance once
imagine yourself on the moon during a thunderstorm.
write "Happy Birthday" on a balloon with little hearts. Don't tie it to the dog's collar just yet.
put a lobster in a velvet gown
switch oven off, even if it hasn't finished yet.
Throw in some salt and turn it on again.
Think about sponges. Yum!
Wait for the timer to ring.
Smell the baked good.
forgot to take it out.
Hide it all under a carpet.

Now here's the smallest model, GPT-3 Ada, generating traditional wedding anniversary gifts.

I gave it the first four (in bold), and it supplied the rest.

1ST YEAR: Paper
2ND YEAR: Cotton
3RD YEAR: Leather
4TH YEAR: Fruit & Flowers, or Linen & Silk, etc.
5TH YEAR: Fowl Feathers etc.
6TH YEAR: Flying Flocks, Troopts, etc.
7TH YEAR: Robes
8TH YEAR: Hair or Nails etc.
9TH YEAR: Small Dainty
10TH YEAR: Common Things
11TH YEAR: Soapsugar Jelly
12TH YEAR: Plaid
13TH YEAR: Cut-leaves
14TH YEAR: Honey
15TH YEAR: Fruit & Flower seeds
16TH YEAR: Potatoes
17TH YEAR: Green Surpluses & Leaves
18TH YEAR: Tissue Paper, Stationers, etc.
19TH YEAR: Bread
20TH YEAR: Fowl Shells
21TH YEAR: Small Seeds
22TH YEAR: Gems & Zinc
23TH YEAR: Carpet/dartboard & Sole
24TH YEAR: Dye, Salt Stone.
25TH YEAR: Paper, Spreads and White Sea Shells
26TH YEAR: Glassware
27TH YEAR: Ointments, Tins etc.
28TH YEAR: Diamonds, Bolts etc.
29TH YEAR: Sunlight Glasses
30TH YEAR: Underwear
31ST YEAR: Javelin Face Goggles
32TH YEAR: Bar Pens & Spider-webs
33TH YEAR: Coffe/Blackberries
34TH YEAR: Wool/Reindeer Hair
35TH YEAR: Lemon, vinaigrette etc.
36TH YEAR: Aromatic Colours
37TH YEAR: Wood Lard
38TH YEAR: Produced Bread
39TH YEAR: Madeleines
40TH YEAR: Gear
41ST YEAR: Tooth Pen (or Tooth Seal)
42ST YEAR: Sieves
43ST YEAR: Charges
44TH YEAR: Deep Water
45ST YEAR: Boric Acid
46TH YEAR: Water Filtration
47ST YEAR: Meat Racks
48ST YEAR: Lying In Blankets
49ST YEAR: Exotic Birds
50ST YEAR: Cat Fur

I generated a few more lists, which you can read as bonus content if you're an AI Weirdness supporter. (Did you know your favorite animal can be "giraffe riding an escalator"?) Or become a free subscriber to get new AI Weirdness posts in your inbox!

Subscribe now