Harry Potter and the word-level recurrent neural network
Welcome Home by Alisanne | Severus will quite continue in a muggle book.
In a previous experiment, I trained a character-level recurrent neural network (char-rnn) to generate titles and summaries of Harry Potter fan fiction, based on over 100,000 examples scraped (with permission) from Ao3. (Big thanks to @b8horpet for the dataset.)
In a character-level neural network, the algorithm can work with any character (mostly letters, numbers, and punctuation) that appear in the original dataset. During the first stages of the training, the neural network has to learn to spell even common words like “the” before it can progress to writing sentences. I wondered if I could get better fan fiction summaries if I shortcutted this process.
A word-level recurrent neural network (word-rnn) lets the algorithm use entire words as its building blocks rather than individual letters. However, we usually have to restrict the number of possible words to a number that the neural network can handle without choking - in my case, I found that I had to use only words that appear at least 200 times.
This 200-word threshold gets rid of self-insertions (i.e. people adding themselves as foreign exchange students), but also gets rid of some of the inventiveness we saw in char-rnn (so sadly, no indescribennings). However, since the neural network no longer has to learn to spell individual words, it’s much better at producing readable summaries.
Char-rnn:
Harry Potter and the Dark Doored Lives of Both (Odd chickens) by Cherus | Hermione gets to follow her terror. He just lost his heroe before him, he’s not crossed intentions. THONG SEXUAL! (Post-War) ((_All holes of DPAGonder Summoning, I want overly fenrir and a shape three and since Himself) for now, sorts, no things, they may have.
Word-rnn:
A Game of Happy - Heart by orphan _ account | Harry Potter is a sixth year student at Hogwarts. And he has a plan. When a strange new teacher shows up with his cousin he finds out that he is not an orphan, but he is not always a werewolf .
The word-level neural network is also helped by a longer memory, since it can now analyze 50 words at a time rather than merely 75 characters. We get longer sentences that still manage to be grammatical, and it remembers to close most of its quotation marks. It even manages to write summaries of identifiably different styles.
A Day on the Night by articcat621 | Hermione and Draco get stuck in a potion, which might have turned out to be the best change in the world.
Sweet by alafaye | Remus and Sirius are in a relationship, and while Remus is in want of a date, he has no idea what to say.
The Secrets We Have by orphan _ account | “I’m not sure what you think,“ Harry said, and he heard it as a smile. "You’re not going to be able to keep it all around.”
Your Name Is A Wish by 1001 | I’m a witch. And you’re not a girl, and I’m a Hufflepuff.
The Secrets We Get by orphan _ account | What if Harry Potter was born? What if Harry Potter had been raised by the Dursleys and had a few friends and a few friends? What if he hadn’t been a Death Eater? What if he had been raised by his godfather? What if he was raised by the Dursleys? What if he had been raised as a Half-blood Prince? What if he didn’t know that he’d been pregnant? What if he was raised in the dark and he became a Death Eater? What if he wanted his parents to be in Gryffindor? What if he was a hero? What if he had found out? What if Harry Potter was not a father? What if he had a twin sister, a very different boyfriend.
That’s not to say that the grammar is flawless, however:
Who am you? by orphan _ account | Harry and Draco are both in the same room as “The Boy Who Lived”.
Draco to go by chaos [ archived by thequidditchpitch_archivist ] | Drarry. Seamus doesn’t mind at a gay hospital. For the prompt “the dark Lord was bed.”
Happy Birthday, Harry by orphan _ account | In which Harry is a cat, and Draco is a little bit of a little over.
Another consequence of the 200-word frequency threshold is that individual authors only made the cut if they were prolific enough to write at least 200 stories. So, anyone appearing in the word-rnn results has written hundreds of these - with the exception of screennames containing underscores. In those, every word is treated separately and so the neural network is able to improvise new names, some of which would be seriously annoying to type in:
why_you_i_because_a_gay_gay
hell_and_the_Luna_life
The_Marauder_of_angry_and_this_are_the_who_though
books_and_go
1000000000000000
coffee_n_and_a_Future
__________________________Dumbledore_and_magic_Black_magic
The_Marauder_Snape_Of_Sherlock
oh_little_boy
Tumblr_and_many_sky_by_the_great_sort
The_Dramione_of_time
The_Lost_and_happy_than_live_with_a_writing_is_not_going_to_you_love
One of the things that must have happened at least 200 times is this crossover:
We’re Just a Wizard by orphan _ account | The Doctor and Lily Potter find themselves in a strange new world.
And “Happy Birthday” and “more than they bargained for” are two of the neural network’s very favorite phrases, which makes sense if you’ve read enough Harry Potter fan fiction.
Happy Birthday, Harry by orphan _ account | Harry and Draco get a little more than they bargained for.
Happy Birthday, Neville by ChokolatteJedi | Harry gets a little more than he bargained for.
Christmas, of course, happens All The Time.
You Don’t Know (I’ll Hell You) by digthewriter | The night before Christmas, it is a mistake on the other side.
Slytherin. by pekeleke for Alisanne | Harry is happy to tell you how he has Secret Santa, and Draco can’t sleep.
Headmaster Snape’s First Years by Elfflame | Hermione has a Secret Santa to a lover for Ron. It’s open to you - it’s exactly what you expect (It’s not quite) /HG/HP
Just matter by This Day Is ( filled ) for digthewriter | A few years after the death of Harry and Ron, Draco goes to the only Wizarding Christmas bar.
In fact, characters end up in bars quite often.
Not a Wizard by me _ and | What if Severus is in the bar? Harry doesn’t know how to tell him that.
My First Chance by coffee _ n _ and _ a _ Future [ archived by ISF _ Archivist ] | Sirius is feeling drunk and alone with a certain Remus later, and soon runs away from the Marauders’ sixth year.
Even if the neural network plays it safe with grammar and spelling, it will often come up with scenarios that seem a bit… unlikely.
We’ll Be Boys by orphan _ account | Sirius Black is a small girl who has no idea what he’s going to do to save the Wizarding World.
The Secrets We Are by orphan _ account | Harry Potter, twin brother of Sirius Black, is a girl with a few friends, a girl who has no idea what is going on.
Harry and Tom : The Boy Who Lived by orphan _ account | Harry Potter returns to Hogwarts behind his parents, then his sixth year at Hogwarts being sorted into Slytherin and more. Now, with his best friend, Severus Snape, Harry finally finds out his greatest plans : his friendship with his friends there are no answers to what went on during history . A young wizard Draco finds on his son and lover, along with Bill Weasley and Draco Malfoy. Will they win? Also, it will be going not expected.
Harry is the Master of Death and Snape is tired of it.
Summer Summer by Alisanne | Severus and Severus have a little fun in the snow.
But I think my favorite thing this time around is the output the neural network produces when it’s instructed to play it Really Really Safe, and choose the most likely next word in each sentence. With the temperature setting turned down to just 0.3, we get:
A Hero’s Tale by orphan _ account | Harry Potter is a wizard, and he is a wizard. He is a wizard. He is a wizard, a wizard, a wizard, and a son. He is also a Slytherin, and he is a wizard. He is a wizard, and he is a wizard. He is also a wizard, and he has not been the one to be a father.
A Hero’s Tale by 1001Angel | Harry’s life is turned upside down when he finds out that he is a wizard, and is a wizard.
A Hero’s Tale by orphan _ account | The last thing Harry Potter expected to be was a boy who was the most powerful Dark Lord ever. He was not the only one to do the impossible.
A Hero’s Tale by orphan _ account | Harry is a Slytherin, and Draco is a vampire. Harry is a Slytherin, and Draco is a Slytherin .
A Hero’s Tale by 1 9 7 6 | The Dark Lord has won. The Dark Lord has won. Voldemort is dead. Harry Potter is dead. The Dark Lord is defeated. Harry is finally free to live his life. He is not the only one with the power to destroy his own destiny.
Become an AI Weirdness supporter to get bonus content! Or become a free subscriber to get new posts in your inbox.