Letting neural networks be weird

Christmas Carols, generated by a neural network

Neural networks are a type of computer program that imitate the way that brains learn to solve problems. They’re used for face recognition, self-driving cars, language translation, financial decisions, and more. I mainly use them to write humor.

My process starts with a dataset - something that the neural network has to figure out how to imitate. Rather unfairly, I give it no instructions about whether it’s trying to write knock-knock jokes or invent Halloween costumes or begin a novel. It doesn’t even have any built-in knowledge of English. It definitely doesn’t have any clue what Christmas carols are, which is why this week’s experiment was so much fun.

The Times of London teamed up with reader/neural net hobbyist Erik Svensson to collect a mix of ancient and modern carols, about 240 carols in all, from “What Child is This?” to “Grandma Got Run Over by a Reindeer”.

When the neural network begins learning, it starts with a set of random rules about how to put one letter after another to make a Christmas carol. Since they are random, they are terrible rules, and when the neural network tries to apply them, it gets junk:

a    a a na snn aca naa  i       s                 a           i aa a a           n a  uugna   nn na    i      a  uaa              a   a     i  a    a          a            a a    sna aagt o aa 

But it can check its rules against the real songs in its database and then make little tweaks to them that make them work slightly better. It learns to capitalize the first word of a line, it learns how often to do line breaks… and soon it begins to learn actual words - the most common ones first.

Hart fon the be the he br wong on the stor Christmas br he, or the wang
Christ, Christ, on bn a me the stord
Hont on thr st bong the wor
I he a s de poog the stow tome on be ser snur

After the neural network has spent many more rounds refining its rules, it begins to look a lot like Christmas.

Now, neural networks tend to pick up tone and vocabulary pretty easily, but struggle with making sense. So, the carol-trained neural network learned to produce a lot of lines that sound - well, joyful, at least.

The cattle around the Christmas will be
A very special Christmas with me

Hurry Christmas to you

Cup on the earth!

Still the loudly candlelight
Would praise His name.

The babe, the Son of Mary.
He sumbled their flowers and all.

The lord of the glory dawns
Give us the leave all away
A star is spent and red
Shake a cup a strend from the sky
Christmas is coming, the wind is come to you

Walkin’ him love, Dingle bells, jingle bells, jingle bells with bells are ringing

With a heart reindeer
But no more a stranger.
Santa baby, and Dancer, and Curry down

Happy Holiday
When the snowflakes will call the world wakes to bring
Glory bears and asses the air the angels sang
And Christmas tree

(Curry is not in the input data. Nor is Dingle. The neural net likes to invent words that it thinks sound sort of carol-y)

Other carols it generated sounded a bit more morose.

The fire is sleeping.
And crying,
Love love
What a King

Let’s take the little children of the grave!

For some reason, the Sandman figures very prominently in the neural net’s Christmas mythology, despite having been mentioned in the dataset only once. Sometimes the neural net latches on to particular words for no reason I can see. Maybe it’s a Neil Gaiman fan.

The sandman so be joyful now it was born today!
Gloria in excelsis Deo.

The sandman bright before Him.
The holly bears a berry bears
And star in the snow is born today!

The sandman so love to seek the world

The sandman so love so deep and sing and the sun

And this? In retrospect, I should have seen this coming:

The world and joy of the sleigh
Santa baby bore sweet Jesus Christ

The holly bears a berry,
And all the reindeer of the sky

The holly bears a berry and reindeer
He was born today!

And Santa baby bore sweet Jesus Christ,
And the chimney the angels sing.

When the snowman brings the snow
Christmas tree
Let’s take that road before
And Santa Claus comes tonight
He will bring us goodness and light

Santa baby, a blitzen,
And he was the sun and reindeer and earth.
The Saviour of the chimney tonight

The story of the chimney see
Santa baby, and blood and joyous so world and joy and good will to see
Santa baby bore sweet Jesus Christ
Fa la la la la la la, la la la la la la la la.

King of toys and hippopotamuses [sic] full of the light of that stood at the dear Son of Santa Claus
He was born in a wonderful christmas tree

Run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolf the new born King.

The neural network also produced a few lines that, um, strained the bounds of good taste. If you want to read them, sign up here and I’ll send them to you.