A Better Worst-Case Scenario

Dystopian literature is all the rage right now, but this excellent article argues that we are worrying about the wrong sort of future. Most of us associate the surveillance state with Orwell's 1984, and if you're North Korean, that analogy is on the nose. Americans and Europeans, however, live in a surveillance state that's not gray and Stalinist. You are being watched, yes—not by people who are itching to put you in prison, but by people who want to seduce you: believe my ideas, they whisper. Buy my stuff. Vote for my candidate. Sign my petition. All the data that we're sucking up in our giant surveillance machines isn't going to land you in the Gulag. But your brain may end up hijacked.

How you look to a marketer

"To make sense of the surveillance states that we live in, we need to do better than allegories and thought experiments, especially those that derive from a very different system of control. We need to consider how the power of surveillance is being imagined and used, right now, by governments and corporations.

"We need to update our nightmares . . ."

The original article, which the io9 piece excerpts, is here. I have never worried terribly about Big Data for the very reason Zeynep Tufekci says here: nobody is going to come arrest me for my opinions or my phone calls. They're just going to prod me to buy products or join movements, which doesn't seem quite as pernicious. (I'm still not convinced it is as pernicious.) Big Data as a tool is, in itself, neutral; it can be and is used for good as well as evil. I'm a technophile, so I've been focused on the good. But when I see the war of ideas exploding across the world, a war which is becoming literally bloody, I see the potential downsides. It's easier to sell an idea to people when you know everything about them. Ask any marketer. These days it's not just products that are being peddled to us in ever-more sophisticated ways: it's ideas. Religious ideas, political ideas, economic ideas. Think about Rand Paul whispering to Mitch McConnell (they didn't realize the mic was on) that he couldn't believe the Dems hadn't "poll tested" a particular stance. If you don't think every message you're receiving hasn't been carefully manufactured by marketing geniuses who know exactly how to push all your emotional buttons so you'll begin nodding like a puppet ... well.

The villain of classic dystopian stories is Big Brother: he has a goal and a fist. You will believe what he believes, do what he says, or you will end up imprisoned or dead. He doesn't care what you think: people are bodies, a labor force—not brains. This is still the kind of villain many people face ... just look at Kim Jong Un. But in modern western democratic societies, the villain is more nebulous: he cares desperately what you think. What you think is everything. He's trying to lure you in, not batter you down. In fact, he might look like that most classic of villains, the Devil. Satan, as a literary figure, didn't throw people into prisons and torture them into submission. (The threat of Hell, it might be argued, is God's tactic, not Satan's.) Satan seduced, he whispered, he tempted. Satan was a master marketer, not an autocrat. Look at the body language in the image here: Satan is a bottom-up sort of bad guy, wheedling and persuading. Autocrats are top-down: they rely on fear and force. Maybe our new dystopian stories ought to hearken back to our oldest literature?

Marketing is nothing new, but what is new is the data behind it. The humans who run stuff (corporations, governments, churches, social media) have gotten so much better at eliciting beliefs from the rest of us, finding and pulling the strings that make people twitch one way or another. The combination of brain science, marketing knowhow, and reams of data is very difficult to beat. We may have free will (it's an open question among philosophers), but it's becoming clear we have less of it than we'd like to believe. Humans are, in the end, distressingly easy to manipulate. And we like it! We enjoy having the certainties of our convictions. We embody our beliefs as if we came up with them ourselves, and then we bask in the pleasure of hearing our thoughts echoed by those around us. This is especially true when opinions are polarized; your beliefs become even more precious to you when they are challenged. It's not just about belief anymore, it's about winning. It's about intellectual tribalism. Marketers know how to utilize that, too.

Dystopian literature, especially cautionary tales like 1984, can train a new generation to recognize and prevent oppressive government. That was critical in the 20th century, when the USSR had global aims and tremendous power. Any society that teaches Orwell in its high schools is very unlikely to wind up with an Orwellian state running things: the book inoculates a culture against that version of government. But since that's not the problem we're facing, what's a better vaccination? Tufekci proposes Aldous Huxley's Brave New World, which is not typically taught these days (my daughter was assigned Orwell, not Huxley), and which isn't selling especially well, certainly not compared to 1984—which has been flying off the shelves since Snowden's revelations. BNW puts forth the meme of "soma," a drug that keeps citizens happy, anxiety-free, and compliant, which might be a good stand-in for any number of seductive ideas sold to us by marketing-savvy institutions. I haven't read BNW since high school so I can't assess it, but I think Minority Report might also be a good book to dust off (and the movie ain't half bad, either): remember the mall scene? Very personalized advertisements.

What do you think? Do we need to retool our dystopian books and movies to warn us against the sweet whispers of insidious marketing instead of hard fist of autocratic regimes? Can we use literature to energize a new generation to be good critical thinkers, more skeptical of the ideas sold to them? Can "manipulative advertisers" ever compete with "tyrannical leader" for villainy? How would the hero fight back?