Raw Material & Manual Sorting

For source material, Time’s feature draws on the user-generated keywords from IMDB for each of 242 movies nominated for best picture since 1970 (including this year’s lot). These tags have a really admirable level of detail for most films. For The Godfather, for example, there are 210 keywords ranging from “mafia” to “orange peel.” For the list of recent nominees, there are 12,144 unique tags.

After paring down the universe of keywords to those that appeared in at least 3 of the 242 movies, there were 2,601 candidates for recombination. I went through that list manually and categorized them as characters (“astronaut”, “sheriff”), adjectives (“French”, “autistic”), period (“World War II”, “1940s”), location (“Berlin”, “the desert”) and theme (“unrequited love”, “mob violence”). I also threw out nearly 2,000 of them that referred to specific objects or events on-screen. (I do not know who generates these keywords, but they are meticulous in parsing the forms of female nudity.)

Adding Brains

From there, it would have been relatively simple to randomly pair a few characters and adjectives, put them in a random location and time, Mad Libs style, and be done with it. But randomness, left unsupervised, has a penchant for putting astronauts in 18th-century England and emperors in Minnesota. A little implausibility can be very funny, but a lot of it defeats the purpose.

To get better results, I used the basic idea behind Markov chains. By calculating the odds that any two tags would appear in the same movie, I was able to worm through an enormous network of tags one plausible step at a time.

The process begins by randomly selecting a tag from the list of 669 that made the final cut. From there, it randomly selects a keyword from the pool of all other keywords that commonly appear in the same movies, weighted slightly toward those that have the closest relationship. Then it repeats the process for that second tag, and so on, creating a chain of 15 keywords for the hypothetical movie.

For example, were the script to select the keyword “melancholy” as the first keyword, the chain might go like this:

melancholy -> singer -> love -> 1950s -> scientist -> villain -> classical music -> hero -> faith -> washington d.c. -> new york city -> revenge -> liar -> boy -> courage -> honor -> captain

As one can see, 15 connections can cover a lot of a ground, and they don’t always make a lot of sense: What’s the connection between faith and Washington, D.C.? But the data is always there. Those two keywords connect in Forrest Gump, Raiders of the Lost Ark, and The Exorcist.

This string of connections occurs live in the browser each time a user generates a new plot. I was able to cram all the data on the connections between the tags into about 600 Kb when minified, so there is no server call necessary for each instance of the algorithm.

Quite to my surprise, this random 15-step stroll through moviedom almost always produces a nice variety of keyword types, with sufficient adjectives, characters, settings and themes to make for a detailed plot. I had thought that I would need to artificially control for a mixture of types so as not to end up with 14 characters and one adjective or a dozen themes and three locations, but the appropriate mixture arises naturally from the Markov chain. The only constraint I placed on the algorithm is that it needs to end up with at least one character, and will continue running extra times if it still doesn’t have one after 15 loops. I’ve never seen it resort to this measure.

After the keywords are assembled, the program then pairs adjectives to characters in the order they appear in the chain, attaches a time period to a location (if both are present), and spits out the raw ingredients. The code to generate the blurbs is less sophisticated. I wanted the structure of the synopses that the program output to vary as much as possible, so I built a prototype for each order of themes, characters, and period.

var madlibs = {

"themes-characters-period": "$themes $haunt $characters in $period",

"themes-period-characters": "$themes $mix in $period for $characters",

"period-characters-themes": "In $period, $characters $confront $themes",

"period-themes-characters": "In $period, $themes $mix for $characters",

"characters-themes-period": "$characters $confront $themes in $period",

"characters-period-themes": "$characters in $period $confront $themes",

"themes-characters": "$themes $haunt $characters",

"themes-period": "$themes $mix in $period",

"period-characters": "The story of $characters $confront in $period",

"period-themes": "In $period, a story of $themes",

"characters-themes": "$characters $confront $themes",

"characters-period": "The story of $characters $confront in $period"

};

The “glue” words—$mix, $confront, $haunt—are just further random variables chosen from a small list of synonyms for a little extra variety.

The output of that particular chain is this:

In 1950s Washington, D.C., melancholy, love and classical music mix for a singer and a scientist

It Works! Mostly

There are still plenty of stinkers that show up, like the “pregnant baby” I see now and again. (Though certain aphids are born pregnant, I do not believe any aphid-themed movies have been nominated for best picture.) That sort of thing is unavoidable by this method, since pregnancy and babies would quite reasonably show up in the same movie. By and large, I was extremely pleased with the results. Though I do notice reporters ending up in the desert with an alarming frequency.