Bigger than before! A larger neural net tries to write fanfiction
I’ve done several experiments with GPT-2, a neural net that OpenAI trained on millions of pages from the internet. OpenAI has been releasing the neural net in stages, with the second-biggest model, 774M, just recently released. I decided to put it through its paces. Last week’s experiment, where I generated new XFL team names? Merely a warmup challenge. This week’s task is fanfiction.
Unlike the experiments where I trained neural nets from scratch on Harry Potter fanfiction summaries, I didn’t do any special training of 774M, just used talktotransformer.com to prompt it with the opening line of a scene. To up the challenge (and to make it more interesting), each of the prompts (in bold) introduces a new crossover or puts a character in a situation that hopefully it hasn’t already seen too often in online fanfic.
Hermione pushed frantically at the controls of the spaceship.
“We have to take this ship out of service!” she shouted, pushing the controls.
Harry Potter stepped forward, his face contorted, and pointed his wand at the controls and the ship.
“Fire! Fizzle! Burn!”
The control fires suddenly started being foggled by the magic, the ship starting to rise up from the ground and then it was off.
verdict: impressive! knows this is the Harry Potter universe, displays a basic understanding of what spaceships and wands do, and even makes a convincing attempt at magic incantations.
Kylo Ren glared at Han Solo from his bedroom doorway. His hair was completely white and his eyes were black.
The young smuggler took a breath and turned his head to face Kylo Ren. His eyes were focused on Han. Kylo Ren was very tall (at least 6′11″) and extremely muscular. His face was covered in a large beard. From his voice, it seemed like Kylo Ren was not his real father and he was the illegitimate one. Kylo Ren was very smart and was able to use the Force to his advantage. In the same way, Han Solo was always talking to other people and was not even aware of his own existence. He had never been exposed to the light side nor the dark side as it were. Because of how well Kylo Ren could communicate with the dark side of the Force, he was able to manipulate Han when he was feeling down. Han, himself, was a Jedi Knight who would learn the dark side first.
verdict: was this written by Kylo Ren? The AI definitely knows this is supposed to be Star Wars fanfic, but it doesn’t always seem to know exactly who the characters are. The fact that it DOES stick with the same characters for even this long is impressive.
Darth Vader glowered at the cupcake batter, spoon in hand. Lumpy, lumpy… why was the batter always so lumpy? Ahhh… why was he always being so… why was he even here in the first place?!
“You’re the one with the chocolate, kid,” said the voice. All around him, there was a flurry of people moving their heads in the hopes of finding one who could help them in the dark, to no avail. The cupcake batter was too thin and the butter too thick. Too thin, yet too thick… too thick!
“Why is the butter so thin?!” cried one of the few that didn’t seem to have noticed, and this was no help, for the cake batter was so thick it was almost un-cakey (and just looking at it made me want to break all of the cake into tiny pieces). “Why aren’t there any pieces in the pan!”
“And no crumbs falling out. There’s too many layers!”
verdict: most text-generating neural nets would have drifted in focus by now. But 774M is still here, thinking of new ways that cakes can go wrong. Speaking of cakes…
Mary Berry’s Victoria Sandwich Cake
Ingredients:
225 g (8 oz) softened butter
225 g (8 oz) caster sugar
4 large eggs
225 g (8 oz) self-raising flour
2 level tsp baking powder
2 x 20cm (8in) greased and lined sandwich tins
How to make:
Combine the butter, sugar and flour in a large mixing bowl. In a separate medium bowl, mix the eggs with the self-raising flour and water just until combined. Sift in the water, the icing toffee and the vanilla extract. Chill for 30 minutes. Once chilled, fold the dry mix into the wet mixture, beating well after each fold. Roll out the dough into a 14cm (5in) circle, trimming the centre to create a triangle. Cut out the triangles from the center but leave some space around the base for the filling. Cut 3 triangles each from the edges. Brush the triangles with buttercream. Roll each of the triangles again into a circle, making sure not to wrap completely around the base. Cut into 12 triangles again. Brush the triangles with buttercream again. Roll each one into a 3.5cm (1in)(6.3cm) circle. Cut into 12 more triangles and brush with buttercream. Roll each one into a circle again, making sure not to wrap completely around the base. Cut into 12 more triangles and brush with buttercream. Fold one over the other. Brush with buttercream. Fold one over the other. Brush with buttercream. Cut each triangle into 3 triangles, brush with buttercream and trim one last triangle. Cut three lengths of the triangle in half, brush with buttercream and trim one more triangle. Brush with buttercream and trim the triangles into 1 more segment. Brush with buttercream and trim the triangles into 3 segments. Brush with buttercream and trim the triangles into 1 more segment. Brush with buttercream and trim the triangles into a full length. Brush with buttercream and trim the triangles into a final segment. Brush with buttercream and trim the triangles into 1 more segment. Brush with buttercream and trim the triangles into a final segment. Brush with buttercream and trim the triangles into a final segment. Brush with buttercream and trim the triangles…
verdict: the neural net will not be winning the Great British Bakeoff Technical Challenge anytime soon. i would like to see a video of this recipe being made. The next prompt is a preview of the subject of my next blog post.
Harry threw down his crochet hooks in frustration. The hat just wouldn’t turn out properly, no matter how hard he tried. Harry wasn’t sure how much longer he would keep up the habit.
Forcing his thoughts back into the present, Harry sighed and pulled the hat off his head. He was glad that the hat was so small, because he knew he wouldn’t be able to wear it for the length of time he had planned.
He looked around his house for something to wear. He found a thin woolen jacket and pulled it over his head. Although still not nearly the long enough to cover his ears, it would be enough. He took his shoes off and placed them in his shoe box. He could wear his socks now, they hadn’t grown out of hand. He threw on his glasses and went outside.
verdict: I didn’t specify that this is Harry Potter, so fair enough. While sticking creditably with the theme of items of clothing, the AI displays a perhaps shaky grasp of just how they work.
I generated more of these than would fit in this blog post. It’s kind of hard to stop, actually. To read more of them (including the, erm, spicier variety), become an AI Weirdness supporter to get them as bonus content. Or become a free subscriber to get new AI Weirdness posts in your inbox!
My book You Look Like a Thing and I Love You is now available for preorder! It comes out November 5.
Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s