Ancient wisdom from the neural network

What happens when really old advice meets really new technology?

A recurrent neural network (like the open-source char-rnn framework used here) can teach itself to imitate recipes, paint colors, band names, and even guinea pig names. By examining a dataset, it learns to formulate its own rules about it, and can use these rules to generate new text that - according to the neural network - resembles the dataset. But since the neural network is doing all this without cultural context, or any knowledge of what the words really mean, the results are often a bit bizarre.

In this example, the dataset is a list of more than 2000 ancient proverbs, collected by reader Anthony Mandelli. Some of these are well-known, such as “You can lead a horse to water, but you can’t make it drink.” and “Where there’s a will, there’s a way.” Others are frankly a bit strange: “Where there’s muck there’s brass.” and “A curst cow has short horns.” and “Be not a baker if your head is made of butter.”

What will a neural network make of this ancient wisdom?

If you answered “Really really weird proverbs”, you are correct.

A fox smells it better than a fool’s for a day.
No songer in a teacuper.
A fool in a teacup is a silent for a needle in the sale.
No man is the better pan on the hunder.
A mouse is a good bound to receive.
Do not come to the cow.

Some of them almost make sense:

A good wine makes the best sermon.
A good fear is never known till needed.
Death when it comes will have no sheep.
An ounce of the heart comes without an exception.
A good face is a letter to get out of the fire.
No wise man ever wishes to be sick.
A good excuse is as good as a rest.
There is no smoke without the best sin.
A good man is worth doing well.
A good anvil does not make the most noise.

While others would be more difficult to pass off as real proverbs:

We can serve no smort.
A good face is a letter like a dog.
A good earse makes a good ending.
Gnow will not go out.
Ung.
A fox smeep is the horse of the best sermon.
No sweet is half the barn door after the cat.
There is not fire and step on your dog and stains the best sermon.
An ox is a new dogn not sing in a haystar.

One of the oddest things to emerge from the proverb-trained neural network is a strange obsession with oxen. I checked, and there were only three oxen-related proverbs in the dataset, yet they appear frequently in the neural network’s version, and usually as rather powerful creatures.

An ox can lever an enemies are dangerous and restens at home.
An ox is not to be given with a single stone.
An ox is never known till needed.
An ox is as good as a best.
An ox is not to be that wound is hot.
An ox is a silent for the gain of the bush.
An ox is not fill when he will eat forever.

Whatever the internal mythos the neural network has learned from these ancient proverbs, oxen are mysteriously important.