The Age of Surveillance Capitalism

The Fight for a Human Future at the New Frontier of Power

Shoshana Zuboff

 

Central Ideas:

 

1 – Under surveillance capitalism, automated machine processes not only know our behavior but also shape it at scale. With this orientation, which turns knowledge into power, it is not enough to automate the flow of information about us; the goal is to automate us.

2 – In 46 of 48 surveys of prominent researchers – conducted between 2008 and 2017 to probe public reaction to the loss of privacy and other elements of capitalist surveillance practices – the majority of participants support taking steps to defend privacy and user control of their personal data.

3 – The Big Other (the name given by the author to the digital apparatus that collects data) does not care what we think, feel, and do, so long as its millions, billions, or trillions of sensitive, acting, computational eyes and ears can observe, render, transform into data, and instrument the vast reservoirs of behavioral surplus.

4 – With respect to the European Union’s GDPR (General Data Protection Regulation), the only possible answer is that everything will depend on how European societies interpret the new regulatory regime in legislation and in the courts. It is the local popular movements that will shape the interpretations.

5 – The fact that surveillance capitalism has usurped our rights to privacy is a scandalous abuse of digital capabilities, despite the promise of democratizing knowledge and meeting the frustrated needs of effective living. Let there be, yes, a digital future, but let it be a human future.

 

About the author:

Shoshana Zuboff is the Charles Edward Wilson Chair Professor Emeritus at Harvard Business School and former Associate Professor at the Berkman Klein Center for Internet & Society at Harvard Law School. She has also written the book In the Age of the Smart Machine.

 

Introduction

Surveillance capitalism unilaterally claims human experience as free raw material in behavioral data. While some of this data is applied to the improvement of products and services, the rest is declared as the owner’s behavioral surplus, feeding advanced manufacturing processes known as “machine intelligence.” This data is manufactured into predictive products that anticipate what a given individual would do now, in the near or more distant future. Ultimately, these products developed through these predictions are marketed in a new type of market that anticipates possible user behavior, which I call future behavior markets. Surveillance capitalists have amassed enormous wealth from these commercial operations, as many companies are eager to discover our future behavior.

As we will see throughout the chapters, the competitive dynamics of these new markets lead surveillance capitalists to acquire increasingly predictive sources of behavioral surplus: our voices, personalities, and emotions. Surveillance capitalists have found that the most predictive behavioral data comes from intervening in the game in ways that encourage, persuade, tune, and harness behavior for the purpose of profitable outcomes. Pressures of a competitive nature have brought about the shift in which automated machine processes not only know our behavior but also shape it at scale. With such guidance turning knowledge into power, it is not enough to automate the flow of information about us; the goal now is to automate us.

 

PART I – The foundations of surveillance capitalism

On August 9, 2011, at about the same time that celebrations erupted in the Apple conference room, 16,000 police officers took to the streets of London, determined to quell “the widest and most protracted breakdown of order in London’s history since the Gordon Riots in 1780.” The unrest had begun four nights earlier when a peaceful vigil in honor of a boy who had been shot by police suddenly turned violent. In the days that followed, the number of protesters grew impressively, with looting and arson spreading to 22 of the 33 districts of London and other major cities across Britain. During four days of street mayhem, thousands of people caused more than $50 million in property damage and 3,000 people were arrested.

At the same time that the rise of Apple seemed to ratify the claims of individuals of the second modernity, the streets of London narrated the grim legacy of a three-decade experiment in economic growth entailing social exclusion.

Surveillance capitalism recruited the wonders of the digital world to meet our needs regarding leading an effective life, promising the magic of unlimited information and thousands of ways to anticipate our demands, easing the complexities of our troubled lives. We welcome it with open arms. As we will explore in more detail throughout the explanation of this book, it is because of surveillance capitalism that the resources for living an effective life, which we seek to obtain in the digital world, come burdened by a new kind of threat. Under this new regime, the precise moment when our needs are met is also the moment when our lives are “plundered” for behavioral data, with the added aggravation that it will be used for someone else’s profit. The result is a perverse amalgam of empowerment inextricably overlaid with the undermining of XXX. In the absence of a decisive response by society that constrains or makes illegal this logic of accumulation, surveillance capitalism looks set to become the dominant form of capitalism of our time.

What is crucial now is for us to identify this new form of capitalism on its own terms and in its own words. It is inevitable that this search will take us back to Silicon Valley, where things change so fast that few people know what has just happened. It is the habitat for progress “at the speed of dreams,” as a Google engineer once described it. My goal here is to slow down the action in order to expand space for such debate and to debunk the tendencies of these new creations as they increase inequality, intensify social hierarchy, exacerbate exclusion, usurp rights, and strip personal life of that which makes it personal to you or me. If the digital future is to be our home, then it is up to us to make it the way we want it. We will need to know how to do it. We will need to decide. And we will need to define who will decide for us. This is our fight for a human future.

Surveillance capitalism in action. Projects like Street View taught Google that the company could assume the role of arbiter of the future and get away with it. It has learned to sustain even the most contested dispossession efforts when they are necessary to secure new supply routes. For example, as protests against Street View erupted around the world and just months before Germany announced that Street View was secretly capturing personal information from unprotected Wi-Fi networks, Google introduced Buzz – a platform that was intended to float through Google’s networks in the path of the coveted behavioral surplus flowing from social media. It was invasive practices introduced with Buzz – which expropriated private information from users in order to connect to their social networks and thus begin a new round of the dispossession cycle and its dramatic disputes.

Surveillance capitalism was born digital, but, as we will see later, it is no longer confined to companies born digital. The logic for converting investment into revenue is extremely adaptive and generates exceptional profits as long as raw material supplies remain free and the law is kept at bay. The rapid migration to surveillance revenues now underway harks back to a shift that occurred at the end of the 20th century, when revenues derived from goods and services became revenues from financial capitalism, derived from the mastery of speculative strategies and aimed at maximizing shareholder value. At that time, every company was forced to obey the same commandments: downsizing, setting up manufacturing and services abroad, reducing expenses that affected product and service quality, reducing responsibilities to employees and customers, and automating the interface with customers – all radical cost-cutting strategies programmed to support the company’s share price, which was held hostage to an increasingly narrow and exclusionary vision of the company and its role in society.

With competition for surveillance assets, new laws of motion gain prominence. Eventually, they shape an even more ruthless imperative to predict future behavior with greater certainty and detail, forcing the whole project to break free of the virtual world in favor of another that we call “real”. In Part II, we follow this migration into the real world, as competitive dynamics force the expansion of supply operations and an increasingly complex extraction architecture reaches into new, more distant, and deeper territories of human experience.

 

PART II – The Advance of Surveillance Capitalism

Google/Alphabet, Facebook, Microsoft and many more companies attracted by surveillance revenues have based their claims on the “demise” of the Internet as a matter of necessity. Forced to improve predictions, surveillance capitalists like Google realized that they needed to expand and diversify their data extraction architectures to accommodate new sources of surplus and new supply operations. Economies of scale would still be vital, of course, but in this new phase supply operations were developed and intensified to accommodate economies of scope and economies of action. What does this imply?

Economies of scope. The shift toward economies of scope defines a new set of goals: the behavioral surplus must be vast but also varied. This effort must be designed with two dimensions in mind: first, the extension of extraction operations from the virtual world to the “real” world, where we actually lead our lives. Surveillance capitalists understood that their future wealth would depend on new supply routes extending into real life on roads, between trees, and through cities. The extension wants to be in your bloodstream and in your bed, in your conversation, in your breakfast, in your means of transport, in your jogging, in your refrigerator, in your parking space, and in your living room.

Economies of action. Just as the scale became necessary but insufficient for higher quality predictions, it was also clear that economies of scope would be necessary but insufficient for higher quality prediction products capable of sustaining a competitive advantage in the new future behavioral markets. The behavioral surplus needs to be vast and varied, but the surest way to predict behavior is to intervene at its source and shape it. The processes invented to achieve this result are what I call action economies. To achieve such economies, machine processes are configured to intervene in the state of play in the real world between real people and real things. These interventions are designed to increase certainty through certain activities: they encourage, tune, watch, manipulate, and modify behavior in specific directions by performing subtle actions, such as inserting a specific phrase into your Facebook News Feed, programming the appearance of a BUY button on your cell phone screen, or turning off your car engine when insurance payment is overdue.

Right to future time. Free will is the existential bone structure that holds up the moral muscle of all promise, and my insistence on its integrity is not nostalgic complacency or a view that privileges pre-digital human history as something more truly human. That is the only kind of freedom we can guarantee ourselves, no matter the weight of entropy or inertia, and whatever the forces and fears that try to shatter time and turn it into eternal make-believe now, and now, and now. These “bones” are the necessary condition for the possibility of civilization as a “moral environment” that fosters the dignity of the individual and respects the distinctly human capacities for dialogue and problem-solving. Any person, idea, or practice that breaks these “bones” and destroys this “muscle” robs us of a future of individual and collective authorship.

Such principles are not eccentric accessories, as Hal Varian and others suggest. Instead, they are hard-won achievements that have come to fruition over millennia of human strife and sacrifice. Our freedom flourishes only when we strive to bridge the gap between making promises and keeping them. Implicit in this action is an assertion that through my will I can influence my future. This does not imply total authority over the future, of course, only over my part in it. In this way, the affirmation of free will also affirms the right to future time as a condition of fully human life.

In the vast majority of surveys designed to probe public attitudes toward the loss of privacy and other elements of capitalist surveillance practices, few of us are in favor of the status quo. In 46 of the 48 most prominent surveys administered between 2008 and 2017, most participants support taking steps to improve privacy and user control of their personal data (only two early surveys were somewhat less conclusive because many participants indicated that they did not understand how and what personal information was being collected), Indeed, as early as 2008 the notion was enshrined that the more knowledge a person has of “internet privacy practices,” the more likely they are to be very concerned about their privacy.

 

PART III – Instrumental power for a third modernity

The Big Other. Surveillance capitalism is the puppet that imposes its will through the ubiquitous digital apparatus. I now name the apparatus the Big Other: it is the perceptible, computational, connected puppet that renders, monitors, computers, and modifies human behavior. The Big Other combines these functions of knowing and doing to achieve a pervasive and unprecedented means of behavioral modification. The economic logic of surveillance capitalism is directed by the capabilities of the Big Other to generate instrumented power, replacing the engineering of souls with the engineering of behavior.

The radical indifference of instrumentalism is operationalized in the Big Other’s dehumanized methods of evaluation, which produce equivalence without equality. They reduce individuals to the lowest common denominator of equality – an organism among organisms – despite all the crucial ways in which we differ. From the point of view of the Big Other, we are strictly Others: organisms that behave. The Big Other encodes the Other’s point of view as a global presence. Here there is no brother of any kind, big or small, good or bad; there are no family ties, however, discontented these may be. There is no relationship between Big Brother and his altered objects, just as there was no relationship between B. F. Skinner’s “scientists and subjects.” There is no mastery of the soul that trades all elements of intimacy and affection for terror – it is much more the unraveling of a large number of relationships. The Big Other doesn’t care what we think, feel or do, so long as its millions, billions, or trillions of sensitive, acting, computational eyes and ears can observe, render, transform into data, and instrument the vast reservoirs of behavioral surplus generated in the tumult of connection and communication.

Right to the sanctuary. In the march of institutional interests intent on implanting the Big Other, the first citadel to fall is the oldest: the principle of the sanctuary. The privilege of the sanctuary has stood as an antidote to power since the beginning of human history. Even in ancient societies where tyranny prevailed, the right to sanctuary stood unshaken. There was a way out from totalitarian power, and that way out was to enter a sanctuary, whether it was a city, a community, or a temple. In the time of the Greeks, shrines were sacred places built across the territories of Ancient Greece and consecrated as places of religious asylum and sacrifice. The Greek word asylon means “impregnable” and underlies the notion of the sanctuary as an inviolable space. The right of asylum survived until the 18th century in many parts of Europe, tied to sacred places, churches, and monasteries. The disappearance of the sanctuary privilege was not an act of repudiation, but rather a reflection of social evolution and the firm establishment of the rule of law.

Fourth Amendment. When American academics and jurists evaluate the ways in which digital capabilities challenge existing law, the focus is on the Fourth Amendment doctrine as it circumscribes the relationship between individuals and the state. Clearly, it is important that Fourth Amendment protections be updated for the 21st century, protecting us from the search and appropriation of our information in ways that demonstrate the contemporary realities of data production. The problem is that even the extension of state protections does not defend us from the assault on the sanctuary devised by instrumental power and put in place by the economic imperatives of surveillance capitalism.

GDPR of the European Union. Many hopes today are pinned on the new body of European Union regulation known as GDPR, or General Data Protection Regulation, which became applicable in May 2018. The European Union’s approach differs fundamentally from that of the United States in the sense that companies must justify their data activities within the GDPR regulatory framework. The guidelines introduce several substantial new basic substantive and procedural features, including a requirement to notify individuals when personal data is breached; a high tolerance for the definition of “consent” that restricts the extent to which a company can rely on this tactic to approve the use of personal data; a prohibition on making any personal information public as a default; a requirement to use the selected privacy option when building systems; the right to the erasure of data; and expanded protections against decisions made by automated systems that impose “significant” effects on an individual’s life. The new regulatory framework also imposes substantial fines for violations, which could be as high as 4% of a company’s global revenue, and allows class action lawsuits, in which users can band together to defend their privacy and data protection rights.

Dehaye’s Request for Investigation. This topic appears in the odyssey of mathematician and data protection activist, Belgian Paul-Olivier Dehaye, who in December 2016 filed a request for his personal data collected through Facebook’s Personalized Audiences and tracking pixel tools, which would reveal the web pages on which Facebook had tracked him. It is likely that Dehaye knew more about Cambridge Analytics’ illicit data operations than anyone in the world except the employees and their masterminds. The Belgian’s goal was to take a bottom-up investigative approach aimed at uncovering the secrets of illegitimate means of behavior modification for political purposes.

In March 2018, fifteen months after his initial request, Dehaye finally received an email from Facebook’s Privacy Operations Team. He was notified that the information he sought was “not available through our self-service tools,” but rather, stored in the “Hive,” Facebook’s “virtual storage area,” where it is retained for “data analytics” and kept separate from the “databases that power the Facebook website.” The company insisted that in order to access this data, “enormous technical challenges” had to be overcome. “This data,” the company writes, “is also not used to serve Facebook directly, live, as users experience.”

In her discussion of “the life of the law,” anthropologist Laura Nader reminds us that the law projects “possibilities for democratic empowerment,” but that these are brought to real life only when citizens actively challenge injustice and use legislation as a means to a higher purpose.

This brings us back to GDPR and the question of its impact. The only possible answer is that it will all depend on how European societies interpret the new regulatory regime in legislation and in the courts. It will not be the wording of the regulations, but rather local popular movements that will shape the interpretations. A century ago, workers organized for collective action and altered the balance of power, and today’s “users” will have to mobilize in new ways that reflect our unique 21st-century “conditions of existence.” We need synthetic statements that are institutionalized in new centers of power, knowledge, and democratic contestation and that challenge current knowledge and power asymmetries. The character of collective action will be necessary if we are to finally replace lawlessness with others that affirm the right to the sanctuary and the right to future time as essential to effective human life.

Only time will tell whether the GDPR will be a catalyst for a new phase of combat capable of confronting and taming an illegitimate marketplace in behavioral futures, the data operations that fuel it, and the instrumental society they aim at. In the absence of new synthetic statements, we may be disappointed by the intransigence of the status quo. If the past serves as a prologue, then privacy, data collection, and antitrust laws will not be enough to disrupt surveillance capitalism. Reflections in the face of the question, “How did they get away with it?”, suggest that the immense and intricate structures of surveillance capitalism and its imperatives will require a more direct challenge.

 

Conclusion

If democracy is to be revived in the coming decades, it is up to us to revive our sense of outrage and loss over what is being taken from us. And I’m not just talking about our “personal information”. What is at stake here is the human expectation of sovereignty over one’s own life and the authorship of one’s own existence. What is at stake is the dominant principle of social ordering in an information civilization and our rights as individuals and societies to answer the questions Who knows? Who decides? Who decides who decides? The fact that surveillance capitalism has usurped so many of our rights in these domains is an outrageous abuse of digital capabilities and the once grand promise of democratizing knowledge and meeting our frustrated needs for an effective life. Let there be a digital future, but let it be first and foremost a human future. 

I do not accept the inevitability and my hope is that you, as a result of this journey of ours, will not accept it either. We are at the beginning of this story, not the end. If we get involved now with older issues, there is still time to take the reins and redirect the action towards a human future that we can call home.

Surely, the age of surveillance capitalism will meet the same fate in teaching us how we do not want to live. It instructs us on the irreplaceable value of our greatest moral and political achievements by threatening to destroy them. It reminds us that shared trust is the only real protection against uncertainty. It demonstrates that power without the rule of democracy can only breed exile and despair. Friedman’s cycle of public opinion and durable law now reverts to us: it is up to us to use our knowledge, recover our sense of direction, urge others to do the same and found another beginning. In the conquest of nature, the victims of industrial capitalism were mute. Those who try to conquer human nature will see their intended victims full of voice, ready to name the danger and defeat it. This book is intended as a contribution to this collective effort.

 

FACTSHEET:

Title: The Age of Surveillance Capitalism

Author: Shoshana Zuboff

 

Review: Rogério H. Jönck