Letting neural networks be weird

A computer tries its best to learn psalms

There’s a type of computer program called a neural network that tries to imitate whatever dataset you give it. The programmer gives it no explicit instructions about what the dataset is like; whether it’s poems or prose or a list of pies. The neural network has to figure all that out for itself, as best it can.

I recently helped with Rose Eveleth’s podcast Flash Forward, which is a science fiction/technology speculation podcast in which she and her guests talk about a new hypothetical future each week. In this recent episode, we imagined what would happen if a scientist one day tried to create a new religion by training a computer on all the existing texts of world religions. I simulated this on a much more modest scale using a neural network on my laptop, just to see what the results would look like. The results were mostly incomprehensible, but there were a few interesting bits:

The camel (might), in this day, face the lord’s light. He is never precious (to the camel).
You are a dog, o my soul, and most sooth.
The chicken listed of the enmity is not a guest of the chicken of his people, as the universe had met the rain of that day.

Here you can hear the entire science fiction story, and some real religious scholars talking about the computer-generated text.

I also trained the neural network on a much smaller dataset, which is much easier for it. This time, I gave it just the set of 150 psalms from the Jewish and Western Christian tradition, verse-form songs of praise, thanksgiving, or lamentation. The neural network would ideally love to memorize these and read them back word-for-word, but because it didn’t have enough brainpower, it had to generalize as best it could. Here’s a typical example of its attempts to recreate a psalm:

O My soul from the ends of the mountains;
And the wild beasts of the wilderness;
They that see me about from the wilderness?
They shall be abundantly satisfied with the sighs of the mountains;
And the wild beasts of the field of the wilderness;
They shake the wind out it in the mountains;
And the wild beasts of the field of the wilderness;
They that see me about with mine own flesh.
The wicked shall be abundantly satisfied with thy servants.
For his lovingkindness endureth for ever.
Oh give thanks unto the God of gods;
And the mountains of the wicked perish at the fire,
And the full of the river of the wilderness;
They shake the wind out it in the mountains;
And the wild beasts of the field of thy house;
And the measts of the wicked shall not reproach unto thee.
They that see me about from the earth,
And the first-born is the deserts;
Hide not the wild beasts of the mighty:
He sent them food the sand.
And the wild beasts of the field of the wilderness;
They that see me about with me, but they hear not;
Neither is the fountains of God.

If you are a choir, you can sing one of these, thanks to Owain Park, who arranged a neural network psalm as a chant based on the Tetris theme. (I assembled this particular psalm from a bunch of individual lines that the neural network produced).

Feel free to share and/or record (oh please oh please record this).

Want more neural network psalms? Sign up here and I’ll email them to you.

Fulltext of Psalm 10010111:

For thou art my God, O God:
I will observe thy statutes:
I will not reprove thee for thy sacrifices;
And my tongue will rejoice with my transgressions;
And thy testimonies be clothed with frogs;
The faithful was in the lands of the earth;
And the peoples he gave them the dust,
He sent them wonders of the dunghills of God.
And he brought forth his praise to pants.
Then they went up a highway for him that rideth through the mountains.
And the wild beasts of the seas;
And the wild beasts of the field of eat,
And the bars of the river of the river of thy wings.
And the cattle upon me, O God;
And the cattle upon the house of God, ye bread with thee.
Oh let them do no bungers in the mountains;
And the wild beasts of the field exult thee about the sheep of thine anger
Their heart shall be abundantly satisfied with frogs.