Earlier this year, a former Wall Street quant named Alexis Kirke premiered a film at the Peninsula Arts Contemporary Music Festival with some special requirements: Audience members had to fit special sensors so that Kirke could monitor their brain waves, heart rates, perspiration levels, and muscle tension. But he’s not doing this for science—he’s doing it for the science of entertainment.
Your bodily response to the Many Worlds film indicates your physical arousal, or the amount that the movie makes you feel feelings, so to speak. An absence of variation would indicate boredom, while a white-knuckle seat-gripper would produce huge spikes in readings. These readings are then fed in to a computer, where they are averaged and analyzed in real time, and used to alter the direction of the film’s narrative based upon the emotional response of the audience.
It’s a choose-your-own adventure film—without the element of conscious choice. Your body (and the bodies of those around you) chooses what happens next. Weird, yes. But in a world of personalized entertainment, it may be just the antidote to Hollywood’s blues.
For every Iron Man movie—that is, a Hollywood blockbuster that appeals to vast numbers without sacrificing quality—there are dozens of films in which every ounce of originality is airbrushed out in an effort to appease the widest possible audience, most of which is overseas.
"A fixed film appeals to the lowest common denominator," says Kirke. "What it does is to plot an average path through the audience’s emotional experience, and this has to translate across all audiences in all countries. Too often it can end in compromise."
Many Worlds is part of a groundbreaking field referred to as Emotional Optimization. EO marks a major transition in the limits of traditional cinema by transforming audiences from passive consumers into active participants. It also, perhaps, represents an end to the belief that there need be such a thing as a work of art that will appeal to everyone. Which has made it something of a quandary for people who like to think about art.
The relationship between science and the humanities is a complex one. In his famous essay, "The Two Cultures," the late British scientist C. P. Snow argued that "the intellectual life of the whole of western society" was split into these two distinct camps, and that it's their inability to comprehend one another that accounts for a large percentage of the world’s problems.
In a sense, both art and technology help to bridge this divide. Although one rarely thinks of it this way, a musical instrument like the flute—just one example—is as much an instrument of technology as it is a tool for creating "art." Similarly, a major tenet of the "hacker ethic" described in Steven Levy’s classic 1984 book, Hackers: Heroes of the Computer Revolution, was that one could "create art and beauty on a computer," an idea famously seized upon by Steve Jobs and the folks at Pixar.
Levy was hardly the first person to express such an insight. The Greek word technē, meaning "craftsmanship," refers both to aspects of art and technology. Then during the Enlightenment, academic genres became en vogue, and people have been trying to reconcile the two fields ever since. In the early 20th century, Hannah Höch—one of the pioneers of photomontage, a precursor to both today’s digital photo manipulation and the idea of immersive augmented reality—wrote, "Our whole purpose [is] to integrate objects from the world of machines and industry in the world of art." Fast-forward to 2013 and modern thinkers are echoing the sentiment. So could films like Many Worlds be the first signs of modern consilience?
Making alternate endings to films isn’t simply a ploy to make movies profitable again. In fact, the profit motive is hardly there at all; Kirke’s days as a quantitative analyst left him feeling disenfranchised about making money for the sake of it. He wanted to be an artist.
Even still, Kirke’s research could be a panacea for the entertainment industry’s woefully inefficient business model. In Hollywood, as in commercial publishing and (to a lesser extent) the record industry, statistically rare "superstar" hits like the Harry Potter films serve to offset the cost of the rest of the studio’s flops, which are much more common. These days, a movie studio that scores one hit out of five films is actually happy with the result.
But what would happen if that same studio ramped up its spending by shooting alternate scenes and commissioning several possible soundtracks? Let’s say the marginal cost for all this is 50% of the film’s original budget—it would still be worthwhile if the producers could increase the likelihood of a hit. If "branching plot" films could be all things to all people, then studios might get two or three hits for every five films—a major improvement in success rate.
After quitting Wall Street, Kirke left the United States and returned to his hometown of Plymouth, England to enroll in a music degree. Today he is a Research Fellow in Music at Plymouth University’s Interdisciplinary Centre for Computer Music Research, as well as being the composer-in-residence for the Plymouth Marine Institute, the U.K.’s leading research facility regarding marine pollution and conservation, where he researches Emotional Optimization. While we’re throwing around jargon, let’s define it.
The term Emotional Optimization relates to the discovery that certain parts of the brain correspond to different emotions. By asking test subjects to wear electroencephalography (EEG) brain caps, neuroscientists can measure the electrical activity that results from ionic current flows within the neurons of the brain. These readings can then be used to uncover the positive and negative reactions experienced by a person as they listen to a piece of music or watch a scene from a film. Through the addition of machine learning tools, the possibility of discovering which low-level features in art prompt particular emotional responses now becomes a reality.
Paired with a simple response device, for a feedback loop—say, a smartphone app where the viewer can upvote scenes they like—users would be able to passively and actively dictate the mood they wanted to achieve. Instead of having playlists to match our general tastes—as is the case with tools like iTunes Genius and Google Instant Mix—a person could have the option of entering their desired emotion into a computer or smartphone and have the media match their mood.
The applications go beyond entertainment; Kirke says this technology may have particular applications in the therapeutic world, as a means of helping those who suffer from stress or forms of depression. Or athletics—a runner could have his or her pulse rate measured by the headphones they’re wearing, with music then selected according to whether heart rate rises or falls. Translated to literature, electronic novels could monitor the electrical activity of neurons in the brain while they are being read, with sections rewritten according to reactions. Just as a stand-up comic or live musician will subtly alter their performance to best appeal to an audience, so too will media increasingly alter to resemble its consumer. The medium might stay the same, but the message will change depending on who is listening.
An article in the New Statesman earlier this year referred to Emotional Optimization as "aural pill-popping," in which there will be "one [music] track to bring us up [and] another to bring us down."
But isn’t this the value of art in the first place? We might feel calm looking at Mark Rothko’s Untitled (Green on Blue) painting, but does this relegate it to the artistic equivalent of Valium? If art evokes empathy, or generates pleasure, then the devices around us might sense when to display a quick pick-me-up—say, a painting by Pierre-Auguste Renoir. Technological skeptics like Evgeny Mozorov say this is a dystopian future in which our emotions are regulated by our machines. In his book To Save Everything, Click Here, he says:
"Well, Google doesn’t exactly ‘know’ it; it knows only that you are missing 124 units of ‘art’ and that, according to Google’s own measurement system, Renoir’s paintings happen to average in the 120s. You see the picture and—boom!—your mood stays intact."
Mozorov continues his line of enquiry by asking the pertinent questions that arise from such a proposition. Would keeping our mood levels stabilized by looking at the paintings of Renoir turn us into a world of art lovers? Would it expand our horizons? Or would such attempts to consume art in the manner of self-help literature only serve to demean artistic endeavors?
There are other questions not touched on by Mozorov. If we accept that Renoir’s work gives us a happiness boost of, say, 122, while Pablo Picasso’s score languishes at a mere 98, why bother with Picasso’s work at all? Similarly, let’s imagine for a moment that the complexity of Beethoven’s 7th Symphony turns out to produce higher neurological highs than Justin Bieber’s hit song "Baby," thereby giving us the ability to draw a mathematical distinction between the fields of "high" and "low" art. Should this prove to be the case, could we receive the same dosage of artistic nourishment—albeit in a less efficient time frame—by watching multiple episodes of Friends as we could from reading Leo Tolstoy’s War and Peace?
These are the types of questions those exploring the crossover between art and technology will have to grapple with. Technologists are rightly excited about the possibilities on offer. As for those creatives who break out in a cold sweat at the thought of "quantifying" art? Well, every good artist needs something to react against.