Automate This - WSJ.com

The Tyranny of Algorithms
Do we want a world where a software program picks the next pop-music star and legal systems run on opaque pieces of code?

By EVGENY MOROZOV


In "Player Piano," his 1952 dystopian novel, Kurt Vonnegut rebelled against automation. For Vonnegut, the metaphor of the player piano—where the instrument plays itself, without any intervention from humans—stood for all that was wrong with the cold, mechanical and efficiency-maximizing environment around him.


Vonnegut would probably be terrified by Christopher Steiner's provocative "Automate This," a book about our growing reliance on algorithms. By encoding knowledge about the world into simple rules that computers can follow, algorithms produce faster decisions. A gadget like a player piano seems trivial in comparison with Music Xray, a trendy company that uses algorithms to rate new songs based on their "hit-appeal" by isolating their patterns of melody, beat, tempo and fullness of sound and comparing those with earlier hits. If the rating is too low, record companies—the bulk of Music Xray's clientele—probably shouldn't bother with the artist.


As we think through the role that algorithms should play in our lives—and the various feats of automation that they enable—two questions are particularly important. First, is a given instance of automation feasible? Second, is it desirable? Computer scientists have been asking both questions for decades in the context of artificial intelligence.


Many early pioneers reached gloomy conclusions. In the mid-1970s, Joseph Weizenbaum of MIT railed against depriving humans of their capacity to choose, even if computers could decide everything for us. For Weizenbaum, choosing and deciding were different activities—and no algorithm should be allowed to blur the difference. A decade later, Stanford's Terry Winograd attacked the philosophical foundations of artificial intelligence, arguing that everyday human behavior was too complex and too spontaneous to be captured in rules. The philosopher Hubert Dreyfus said as much in the 1960s, when he compared artificial intelligence to alchemy. But Mr. Winograd's critique, coming from a respected computer scientist, was particularly devastating.






Automate This


By Christopher Steiner

(Portfolio Penguin, 248 pages, $25.95)


Mr. Steiner, a former reporter for Forbes and currently an Internet entrepreneur, glosses over his subject's historical background. He does introduce us to the first known algorithm—found on clay tablets near Baghdad and dating to roughly 2500 B.C., it recorded Sumerian instructions for how to equally divide grain harvest between a varying number of men—but, alas, he doesn't go much further. For most of his book's 10 chapters he simply explores different sorts of contemporary algorithms and their uses, from their embrace by record labels to their potential to transform health care.


The author explains "the algorithmic takeover" of the past three decades by linking it to Wall Street's fascination with algorithmic trading, whereby traders recede into the background and leave it to the algorithms to identify and act on arbitrage opportunities. Judging by the recent Knight Capital debacle—one of the main cheerleaders for algorithmic trading squandered $440 million when one of its algorithms went rogue—this is, indeed, an important subject. But is Wall Street the driving force behind the culture-wide algorithmic fetish so aptly diagnosed by Mr. Steiner? Or is it just along for the ride?


Mr. Steiner does air some qualms with the proliferation of algorithmic decision making, and some of these are on target. Writing of companies like Music Xray, he wonders whether algorithms will "lead to a music world of forced homogenization" rather than promote innovative artists. But the author goes too far: "Algorithms may bring us new artists, but because they build their judgment on what was popular in the past, we will likely end up with some of the same kind of forgettable pop we already have." An important concern—but why blame the algorithms? After all, record labels could also employ algorithms to identify music that is fresh and diverse. Instead of spotting consensus items, they could highlight risky outliers.



It isn't the algorithms that favor the mainstream over the avant-garde but the music industry.


Algorithms don't build their judgments on anything—their creators do. One can easily imagine a very different music industry that would still profit from algorithms but favor very different kinds of artists. The inherent risk associated with Mr. Steiner's technology-centric approach is that the institutional logic inscribed in the algorithms suddenly becomes invisible, as we direct our fury at the technology instead.


On the whole, though, Mr. Steiner believes that we need to accept our algorithmic overlords.


Accept them we might—but first we should vigorously, and transparently, debate the rules they are imposing. Following several high-profile scandals involving algorithmic trading, regulators in Hong Kong have recently proposed that all such algorithms be audited and tested every year. Similar calls have been made with regard to independent audits of Google's search algorithms—if only to avoid the impression that the company might be favoring its own services in its search results.



Consider predictive policing—an area that Mr. Steiner doesn't discuss but one that captures just how tricky the politics of algorithms could get. Police departments across America are rapidly embracing software that, by drawing on past crime data, suggests where and when crimes might happen next. It all sounds fine in theory—but will it open the door to even more racial profiling?


Police could blame their algorithms and say: "My algorithm told me to arrest this man!" Some legal scholars already seriously entertain this possibility. Private companies, moreover, might eventually step in with proprietary algorithms. Do we want our legal system to run on opaque code?



While "Automate This" hints at some of these thorny issues, it says very little about the ways to resolve them. The real question isn't whether to live with algorithms—the Sumerians got that much right—but how to live with them. As Vonnegut understood over a half-century ago, an uncritical embrace of automation, for all the efficiency that it offers, is just a prelude to dystopia.



Mr. Morozov is the author of "The Net Delusion: The Dark Side of Internet Freedom."


A version of this article appeared September 20, 2012, on page A15 in the U.S. edition of The Wall Street Journal, with the headline: The Tyranny Of Algorithms.



Book Review: Automate This - WSJ.com


 Link:  http://online.wsj.com/article/SB10000872396390443686004577633491013088640.html?mod=googlenews_wsj