<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1 20120330//EN" "http://jats.nlm.nih.gov/publishing/1.1/JATS-journalpublishing1.dtd">
<!--<?xml-stylesheet type="text/xsl" href="article.xsl"?>-->
<article article-type="research-article" dtd-version="1.1" xml:lang="en" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<front>
<journal-meta>
<journal-id journal-id-type="issn">2515-2076</journal-id>
<journal-title-group>
<journal-title>Le foucaldien</journal-title>
</journal-title-group>
<issn pub-type="epub">2515-2076</issn>
<publisher>
<publisher-name>Open Library of Humanities</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.16995/lefou.62</article-id>
<article-categories>
<subj-group>
<subject>Research</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Algorithmic Decision-Making, Spectrogenic Profiling, and Hyper-Facticity in the Age of Post-Truth</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname>Weiskopf</surname>
<given-names>Richard</given-names>
</name>
<email>richard.weiskopf@uibk.ac.at</email>
<xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
</contrib-group>
<aff id="aff-1"><label>1</label>University of Innsbruck, AT</aff>
<pub-date publication-format="electronic" date-type="pub" iso-8601-date="2020-03-09">
<day>09</day>
<month>03</month>
<year>2020</year>
</pub-date>
<pub-date pub-type="collection">
<year>2020</year>
</pub-date>
<volume>6</volume>
<issue>1</issue>
<elocation-id>3</elocation-id>
<history>
<date date-type="received" iso-8601-date="2019-06-29">
<day>29</day>
<month>06</month>
<year>2019</year>
</date>
<date date-type="accepted" iso-8601-date="2020-02-24">
<day>24</day>
<month>02</month>
<year>2020</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright: &#x00A9; 2020 The Author(s)</copyright-statement>
<copyright-year>2020</copyright-year>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/4.0/">
<license-p>This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See <uri xlink:href="http://creativecommons.org/licenses/by/4.0/">http://creativecommons.org/licenses/by/4.0/</uri>.</license-p>
</license>
</permissions>
<self-uri xlink:href="http://foucaldien.net/articles/10.16995/lefou.62/"/>
<abstract>
<p>This paper investigates algorithmic decision-making and data-driven profiling as particular ways of producing truth by which &#34;(wo)men govern themselves and others.&#34; It starts with problematizing some of the fundamental assumptions on which algorithmic decision-making relies. It then conceptualizes profiling as a &#34;spectrogenic process&#34; in which abstractions are produced that haunt the world, thereby generating material effects of sorting people in/out from a distance. In the final section, the paper discusses emerging forms of governance and the modes of subjectification associated with the current condition of multiple profiling machines. Paradoxically, in the context of post-truth, these forms produce a hyper-facticity that governs by circumventing reflexivity, grounding government in computational truth, and substituting ethico-political decisions by calculations.</p>
</abstract>
<kwd-group>
<kwd>algorithmic decisions</kwd>
<kwd>profiling</kwd>
<kwd>post-truth</kwd>
<kwd>specters</kwd>
<kwd>hauntology</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<disp-quote>
<p>Power is in tearing human minds to pieces and putting them together again in new shapes of your own choosing. Do you begin to see, then, what kind of world we are creating? (George Orwell)<xref ref-type="fn" rid="n1">1</xref></p>
</disp-quote>
<sec>
<title>1. Introduction</title>
<p>The recent scandal around <italic>Cambridge Analytica</italic> (<italic>CA</italic>) has brought the manipulative potential of Big Data analyses as well as the practices of monetization of personal information and data to light. The discussion has been dominated by issues of privacy and the (mis)use of data and the role of <italic>Facebook</italic> and various data brokers in this context. <italic>CA</italic> is (or was) specialized on data mining, psychometrics and the development of psychological profiles that allow predicting behavior of users and provide a basis for (micro-)targeting people. According to its (now closed) own website, <italic>CA</italic> &#34;use(s) data to change audience behavior&#34; both in commercial and political contexts.</p>
<p>A theoretical basis for <italic>CA</italic>&#39;s work can be found in a paper published in 2015.<xref ref-type="fn" rid="n2">2</xref> Here, the authors attempt to show &#34;that computer&#39;s judgments of people&#39;s personalities based on their digital footprints are more accurate and valid than judgments made by their close others or acquaintances (friend, family, spouse, colleagues).&#34;</p>
<p>Based on Big Data technologies, so the argument goes, personality assessment is not only more accurate than subjective judgments, but also more accurate than conventional psychological diagnoses based on expert knowledge or standardized procedures for measuring personality traits. A high practical relevance is attributed to this enhanced capacity to know individuals:</p>
<disp-quote>
<p>Automated, accurate, and cheap personality assessment tools could affect society in many ways: marketing messages could be tailored to user&#39;s personalities; recruiters could better match candidates with jobs based on their personality; products and services could adjust their behavior to best match their users&#39; characters and changing moods, and scientists could collect personality data without burdening participants with lengthy questionnaires. Furthermore, in the future, people might abandon their own psychological judgments and rely on computers when making important life decisions, such as choosing activities, career paths, or even romantic partners. It is possible that such data driven decisions will improve people&#39;s lives.<xref ref-type="fn" rid="n3">3</xref></p>
</disp-quote>
<p>The example of <italic>CA</italic> illustrates the trust in the power of algorithmic procedures and data mining, to produce evidence and better decisions. This is shared by theorists of Big Data. Mayer-Sch&#246;nberger and Cukier, for example, argue that digitalized devices and Big Data procedures &#34;liberate us from profiling&#39;s shortcomings&#34; and hold the promise to make profiling &#34;better, less discriminatory, and more individualized.&#34;<xref ref-type="fn" rid="n4">4</xref> But the <italic>CA</italic> example also illustrates what Youyou et. al. note in passing: &#34;knowledge of people&#39;s personalities can also be used to manipulate and influence them.&#34;<xref ref-type="fn" rid="n5">5</xref> As revealed by the whistleblower Christopher Wylie, profiles can be turned into a &#34;psychological warfare tool&#34;<xref ref-type="fn" rid="n6">6</xref> as well as into big business. More than this, the example demonstrates the power of profiles and profiling for constructing and shaping the social world. It illustrates how profiles can circulate and have intended and unintended effects that are inherently ethical and political such as &#34;sorting people according to their presumed economic and political value.&#34;<xref ref-type="fn" rid="n7">7</xref> Thus, the significance of profiling goes way beyond its potential to improve decisions. In fact, algorithmic decisions and data-driven profiling are part and parcel of a new regime of truth which Rouvroy has termed &#34;data-behaviourism&#34;<xref ref-type="fn" rid="n8">8</xref> and a new form of governmentality which she characterizes as &#34;spectral.&#34;<xref ref-type="fn" rid="n9">9</xref></p>
<p>In this paper I problematize elements of this regime of truth and provide an account of its spectral character. While the critical debate on post-truth and post-facticity mostly focusses on the apparent loss of facts or the production of &#34;bullshit,&#34; which is often used as a &#34;catch-all word to cover misrepresentation, half-truths and outrageous lies alike,&#34;<xref ref-type="fn" rid="n10">10</xref> I take inspiration from Foucault, who defined his &#34;problem (as) to see how men govern (themselves and others) by the production of truth.&#34;<xref ref-type="fn" rid="n11">11</xref> From this perspective, the critical point is not that certain things or claims are false, but rather that they are accepted as true. I argue that (data-driven) profiling and algorithmic decision-making are new ways of producing truth by which &#34;(wo)men govern themselves and others.&#34; It is thus not the loss of facticity that is at stake here, but the production of a new kind of facticity that I call hyper-facticity. As such it implies a new way of &#34;defining the relations between the manifestation of truth and the exercise of power.&#34;<xref ref-type="fn" rid="n12">12</xref> It governs behavior by circumventing reflexivity, by grounding government in computational truth rather than ethical-political debate, and ultimately by substituting ethical-political decisions by calculations.</p>
<p>I start with a discussion of &#34;algorithmic decision-making&#34;<xref ref-type="fn" rid="n13">13</xref> in the context of Derrida&#39;s view on decisions that deserve the name. This opens the space for problematizing assumptions on which algorithmic decision-making and profiling rely. In transforming decisions into calculations, they create &#34;ghostly demarcations&#34;<xref ref-type="fn" rid="n14">14</xref> that discriminate and sort in/out based on abstractions and make the ethics and politics involved in these processes disappear. In the following section I look at the example of profiling, an important technology of organizing that is increasingly used for creating knowledge about individuals and populations and for informing decisions in various contexts. While traditional profiling generated knowledge about individuals or groups, based on predefined criteria and expert knowledge, &#34;new profiling&#34;<xref ref-type="fn" rid="n15">15</xref> increasingly generates a specific type of knowledge, based on detecting and discriminating patterns in huge data sets, using algorithmic procedures. I argue that algorithmic profiling, which is designed to create evidence/knowledge/transparency in (preemptive) decision-making, can be understood as a &#34;spectrogenic process&#34;<xref ref-type="fn" rid="n16">16</xref> in which abstractions are produced that haunt the world. Independently of whether profiles accurately represent individuals or groups, they circulate and have material effects. The application of profiles and profiling has a spectral structure, generating effects in various contexts, which cannot be mastered in advance. In the final section, I reflect on the emerging forms of governance and the modes of subjectification associated with the condition of multiple profiling (machines).</p>
</sec>
<sec>
<title>2. Big Data, Algorithmic Decision-Making, and the Ghost of the Undecidable</title>
<p>In recent years, quantification and measurement have been intensively discussed in different areas.<xref ref-type="fn" rid="n17">17</xref> Quantification has taken a new dimension with the movement of &#34;datafication,&#34; which implies transforming the flux and flow of real-life events&#8212;behaviors, actions, motions, bodies, etc.&#8212;into a quantified format that allows it to be tabulated and analyzed. The label &#34;Big Data&#34; is often characterized by the &#34;4Vs&#34; (huge in volume, diverse in variety, high in velocity and veracity of data) and captures a development that was made possible by growing computational power and technological advancements in data collection (&#34;harvesting&#34;), processing, storing and &#34;mining.&#34; The transformation of human experience into a datafied and machine-readable form is driven and intensified by the economic imperatives of &#34;surveillance capitalism.&#34;<xref ref-type="fn" rid="n18">18</xref> Zuboff has argued that this &#34;mutation of capitalism&#34; relies on transforming human experience into data, providing material for the production of &#34;prediction products,&#34; which are &#34;designed to forecast what we will feel, think and do: now, soon, and later.&#34;<xref ref-type="fn" rid="n19">19</xref> Here, &#34;machine intelligence&#34;<xref ref-type="fn" rid="n20">20</xref> (a broad term referring to a wide range of computational operations, like machine learning, &#34;classical&#34; algorithmic production, predictive analytics or artificial intelligence and others) is the central means of production, which again relies on extracting masses of data for creating accurate behavioral predictions as a source of profit.</p>
<sec>
<title>Predictions and Evidence-based Decisions</title>
<p>Today, Big Data analyses and data-driven profiling are applied in many different areas. In general, &#34;(Big) data captured through digitalized devices are processed by algorithms aimed at predicting what a person will do, think and like on the basis of their current (or past) behaviors.&#34;<xref ref-type="fn" rid="n21">21</xref> Such algorithms are for example used for predicting security threats and risks, forecasting crimes in predictive policing, in insurance and employment screening, or in establishing credit scores for predicting the likelihood of loan default. In managerial contexts, data and numbers are seen as an essential basis for organizational steering and transparency. Big Data analyses are used in the areas of marketing and consumer research. Increasingly, psychometrical categories are used to create clusters of consumer types (profiles) and to predict consumer behavior.<xref ref-type="fn" rid="n22">22</xref> The idea of &#34;predicting people&#34; has also become central in Human Capital Management.<xref ref-type="fn" rid="n23">23</xref></p>
<p>In all of these areas, Big Data analyses promise to deliver a basis for &#34;evidence-based decisions.&#34; Data are frequently presented as neutral forms of information that represent objects in the world and provide insight into social, economic and environmental phenomena. The assumption is that (big) data comprehensively map reality, while algorithmic procedures (computer-based data mining) allow identifying and &#34;discovering&#34; the underlying structures and patterns of this reality. McAfee and Brynjolfsson formulated the conviction of Big Data proponents succinctly: &#34;Data-driven decisions are better decisions&#8212;it&#39;s as simple as that. Using big data enables managers to decide on the basis of evidence rather than intuition.&#34;<xref ref-type="fn" rid="n24">24</xref> Critical researchers in various fields instead have questioned this assumption on the grounds that (big) data and algorithms might be inherently biased, creating, reproducing or even intensifying unfair discriminations and established inequalities.<xref ref-type="fn" rid="n25">25</xref> In addition, Kitchin reminds us that (big) data are neither simply given nor representing the world. Instead, &#34;data are <italic>captured</italic> from the world, but in turn do work in the world.&#34;<xref ref-type="fn" rid="n26">26</xref></p>
</sec>
<sec>
<title>Algorithmic Decisions, Algorithms, and the Ghost of the Undecidable</title>
<p>For Derrida, a decision that deserves the name presupposes &#34;undecidability&#34; and requires going through the &#34;ordeal of the undecidable.&#34; Algorithmic decisions are in his sense not decisions at all since &#34;a decision that didn&#39;t go through the ordeal of the undecidable [&#8230;] would only be a programmable application or unfolding of a calculable process.&#34;<xref ref-type="fn" rid="n27">27</xref> So called decision trees that characterise algorithmic decisions effectively automate responses and force a yes/no or if/then choice. In this sense, the agonism and radical uncertainty of a true decision is annulled or rather denied. There is no hesitation or fear, no struggling heart, no ambiguity. Similarly, an &#34;apparent decision taken on the basis of what is &#39;seen&#39; evidently, via the calculation of experts, or in the screened results of algorithmic visualization, is not a decision at all.&#34;<xref ref-type="fn" rid="n28">28</xref> In fact, algorithmic procedures that produce a decision (or rather a yes/no switch) as the outcome of a calculated series of steps (e.g. decision trees) reframe ethical and political questions as technical problems of systems engineering. Operating through logical and mathematical procedures, they become &#34;supercarriers of formal rationality&#34;<xref ref-type="fn" rid="n29">29</xref> that promise to overcome the &#34;bounded rationality&#34; of humans. While proponents of algorithmic decision-making assume that the inherent undecidability, which traverses human systems, can be&#8212;or ideally is&#8212;removed, Derrida reminds us that the undecidability is &#34;not a moment to be traversed and overcome.&#34; It continues to inhabit the decision &#34;as an essential ghost.&#34;<xref ref-type="fn" rid="n30">30</xref> The programmed decision and the objects, divisions and demarcations it creates are both haunting the world and haunted by its own shadow. In the context of data-driven decisions, the ghost(ly) and the notion of haunting draw attention to all that exceeds measurement and meaning, to the &#34;noise&#34; in the system, to all that cannot be captured and computed, to that which occasionally and unexpectedly interrupts systems and sometimes also allows for novelty, creativity and the creation of the new.<xref ref-type="fn" rid="n31">31</xref> As Cheyney-Lippold has demonstrated, even &#34;algorithmic identities&#34;<xref ref-type="fn" rid="n32">32</xref> inferred upon individuals can never finally remove the undecidability and fix the subjects they refer to. On the contrary, they are pervaded by undecidability. For example, the ratio male/female or other characterizations is constantly changing according to the feed-back data; definitions of maleness/femaleness etc. can shift according to the logic of the algorithm.</p>
<p>Algorithms can be understood as &#34;series of generalized procedures for turning disorganized data inputs into manageable outputs through a series of logical rules.&#34;<xref ref-type="fn" rid="n33">33</xref> As such, algorithms are organizers that are both socially produced and socially productive. They organize and structure the reality producing process as they select, filter, and frame information and <italic>create facticity</italic>. Hence, no algorithm is neutral. &#34;Pattern discrimination,&#34;<xref ref-type="fn" rid="n34">34</xref> the ability to filter information from data is never innocent. It is inherently tied to value judgments and it often reproduces or amplifies the very phenomenon it seeks to describe or represent. Values are thus not external to the algorithm, but they are folded into it. Even the (seemingly) trivial acts of naming, of storing data, of &#34;deciding where to make cuts in the system,&#34;<xref ref-type="fn" rid="n35">35</xref> contain value judgements. Moreover, algorithms that are &#34;inherently framed and shaped by all kinds of decisions, politics and ideology&#34; are created for purposes that &#34;are often far from neutral: to create value and capital; to nudge behaviour and structure preferences in a certain way; and to identify, sort, classify people.&#34;<xref ref-type="fn" rid="n36">36</xref> Algorithms codify assumptions and are socially productive. A data-mining algorithm for example that is designed to find or &#34;discover&#34; valuable data and patterns, sorts, selects, filters data based on specific <italic>assumptions</italic> about what makes data &#34;valuable&#34; and for whom. In Hacking&#39;s terms, it is not just an &#34;engine of discovery&#34;&#8212;as the term &#34;Knowledge Discovery in Data Base&#34; (KDD) suggests&#8212;but also at the same time an &#34;engine of making up people.&#34;<xref ref-type="fn" rid="n37">37</xref> Algorithms are thus performative. They <italic>generate</italic> objects of knowledge that loop back and shape the world. There are many different&#8212;highly complex and sophisticated&#8212;technical methods of data mining (e.g. decision trees, cluster analysis, neural nets, text mining, anomaly detection and many others). What is distinctive about these methods is the specific <italic>interested</italic> perspective that guides the search for &#34;valuable&#34; data and patterns.<xref ref-type="fn" rid="n38">38</xref> From these data it allows to construct a &#34;person of interest,&#34; for different actors, institutions, organizations for different reasons. A hiring algorithm for example mines data from the interested perspective of an employer and <italic>creates</italic> an image of the attractive/unattractive candidate; a security algorithm makes up an image of the potential security risk; in consumer profiling, types of more or less profitable consumers are created and sorted; in credit scoring an image of the more or less creditworthy or trustworthy person is produced. While it is often assumed that the outcomes represent a preexisting and underlying reality in a neutral way, critical researchers have argued that the outcomes are rather &#34;carefully crafted fictions&#34;<xref ref-type="fn" rid="n39">39</xref> that help to make us overlook or forget that underlying data are often messy, incomplete, full of gaps and errors, as well as the fact that algorithms may work in biased or even systematically distorted ways. However, such fictions are not simply illusions, but rather social objects and artefacts that have concrete material effects in the world. This will be theorised and illustrated in the following section on profiling.</p>
</sec>
</sec>
<sec>
<title>3. Profiles, Data-doubles, and the Spectrogenic Process</title>
<p>Profiling is a technology and practice that is increasingly &#34;used to make decisions, sometimes even without human intervention.&#34;<xref ref-type="fn" rid="n40">40</xref> Profiles and profiling have a long history. Genealogically, they can be traced back to their use in police work and behaviorist psychology of the early 20<sup>th</sup> century.<xref ref-type="fn" rid="n41">41</xref> Harcourt has shown how during the second half of the 20<sup>th</sup> century, actuarial methods have been increasingly used in the context of criminal justice, creating profiles from statistics.<xref ref-type="fn" rid="n42">42</xref> Koopman&#39;s genealogy of the &#34;informational person&#34; points to various systems of &#34;human bookkeeping&#34; (registration systems that define &#34;who the person is&#34;), personality testing, psychometrics and trait measuring, as well as to (racialized) credit scoring systems in the early 20<sup>th</sup> century as precursors of modern profiling.<xref ref-type="fn" rid="n43">43</xref> Thus, long before the emergence of Big Data, profiles were used as a knowledge tool in a wide range of human sciences. Today, it is no longer only about deviant subjects in the psychological, legal or moral sense, but above all about the identification of (potentially) &#34;valuable&#34; subjects. In modern marketing for example profiles are created for classifying customers according to their potential worth. Equally, in Human Resource Management (HRM), profiling of employees and classifying them according to their performance and their &#34;potential to perform&#34; has a long tradition. Identifying &#34;high potentials&#34; and future &#34;stars&#34; and separating them from &#34;question marks,&#34; &#34;cash-cows,&#34; and &#34;deadwood&#34; has been proposed as a tool for supporting decisions in &#34;strategic HRM.&#34;<xref ref-type="fn" rid="n44">44</xref> Increasingly, profiling of prospective and/or current employees draws on online data and algorithmic procedures to generate knowledge concerning the &#34;fitness for the job.&#34;<xref ref-type="fn" rid="n45">45</xref> Similarly, in the context of marketing and consumer profiling, researchers observed a shift from classifying and segmenting customers according to demographic criteria to the use of psychometric profiling for the purpose of matching a product, price or marketing strategy to particular individuals or target groups.<xref ref-type="fn" rid="n46">46</xref> In all of these areas it is assumed that decision-making dramatically improves, since it is &#34;based on highly informed prediction of future behavior.&#34;<xref ref-type="fn" rid="n47">47</xref></p>
<sec>
<title>Theorizing Profiles and Profiling</title>
<p>All profiles are abstractions. In the process of profiling images of the person are created for the purpose of diagnosis or prediction. A profile is never a representation of an individual but a selective image that is constructed for specific purposes. In the process of profiling the complexity of the person is reduced to a finite number of traits, indicators, etc. Profiles are in this sense &#34;model(s) or figure(s) that organize [&#8230;] multiple sources of information to scan for matching or exceptional cases.&#34;<xref ref-type="fn" rid="n48">48</xref> Bogard locates these &#34;models or figures&#34; between fiction and reality at the intersection of virtual and actual worlds. Such figures may be fictions (in the sense of not representing an underlying reality) but these are &#34;a fiction difficult to separate from fact, whose effects are indiscernibly real.&#34;<xref ref-type="fn" rid="n49">49</xref> Fictions create mental images and expectations. They are operationally effective, as they influence decisions, shape and intervene in the world. Consequently, profiles should not be understood as descriptive, but as performative.</p>
<p>Harding has taken a similar perspective and argued that profiles that are assembled and derived from data should not be understood as representations of the self, but as &#34;mere specters of our actual selves&#8212;something metaphorically akin to the theatrical illusion called Pepper&#39;s Ghost.&#34;<xref ref-type="fn" rid="n50">50</xref> Understanding profiles as &#34;mere specters of our actual selves&#34; reminds us that they should not be confused with the embodied human being, yet at the same time, they are more than simply illusions. I argue that such categories may not be able to capture the world, but nevertheless they may have material effects in the world. In her seminal book on &#34;ghostly matters,&#34; Gordon<xref ref-type="fn" rid="n51">51</xref> has introduced the &#34;ghost&#34; as a social figure that helps understanding and theorizing phenomena that are somehow located <italic>between</italic> the dead and the living, between the abstract and the concrete; phenomena which are absent, but nevertheless make their presence felt, phenomena which are illusionary or unreal, but are at the same time real in the sense of producing material effects. The figure of the &#34;ghost&#34; and the &#34;ghostly&#34; has also been taken up by organization scholars<xref ref-type="fn" rid="n52">52</xref> and data-scientists<xref ref-type="fn" rid="n53">53</xref> for theorizing a &#34;spectral working&#34; of forces that destabilize and contaminate any self-sufficient present, for disrupting linear concepts of time and for attending to how policies or data become &#34;haunted&#34; when their implementation disrupts local cultures, traditions and histories. Continuing this work and taking some inspiration from Derrida&#39;s analysis of the ghost and the &#34;ghost effect&#34; in <italic>The Specters of Marx</italic>, I suggest that profiles can be understood as ghosts that are the outcome of a &#34;spectrogenic process.&#34;<xref ref-type="fn" rid="n54">54</xref> Derrida describes this as a process in which thoughts, ideas, data, etc. are extracted from the living body and integrated into a more abstract body. &#34;Ghost effects&#34; are produced with every step of abstraction. Transferred to our context this means, that ghost effects are engendered on multiple levels. Firstly, when human experience is abstracted and turned into &#34;information,&#34; secondly, when data are extracted, resulting in what Haggerty and Ericson called a &#34;decorporealized body, a &#39;data double&#39; of pure virtuality.&#34;<xref ref-type="fn" rid="n55">55</xref> The data-double as the aggregation of personal information represents already a high level of abstraction; an even higher level is reached when the data-double is &#34;mined&#34; and selective information is assembled and reassembled into a profile that allows institutions and organizations to make discriminations and separations among people. In the process of application, institutionally and technically formed and shaped data return to the world of real-life events, where they haunt those with whom profiles are associated. As Gordon put it: &#34;[i]n haunting, organized forces and systemic structures that appear removed from us make their impact felt in everyday life in a way that confounds the social separations themselves.&#34;<xref ref-type="fn" rid="n56">56</xref></p>
<p>While all profiles are abstractions, profiling can be distinguished first, by the way how these abstractions are produced and second by the way how these abstractions (profiles) are applied in decision-making processes. I elaborate this in the following.</p>
</sec>
<sec>
<title>Production of Profiles</title>
<p>There are different forms of profiling. Traditional profiling is a process of classification, which is based on ex-ante categorizations of individuals. It assumes that the individual can be characterized by a set of traits that can be identified by professional/scientific methods. A specific type of knowledge is created that typically results from experience or the tradition of expert knowledge. Typically, such knowledge is deductive and derived from specific predefined hypotheses. In the traditional sense, the profile results from targeted questioning based on specific hypotheses. Profiling involves the definition of categories that are assumed to characterize the individual and the establishment of methods for measuring the individual along these categories. Measurement allows to identify differences and to determine the distance to the norm. Profiling allows to position individuals in a normative matrix of behavior and to make them comparable to others.</p>
<p>Traditional profiling resembles the classical disciplinary technology of <italic>examination</italic>. This &#34;tiny operational schema that has become so widespread (from psychiatry to pedagogy, from the diagnosis of diseases to the hiring of labour),&#34; which Foucault saw at the heart of the procedures of discipline, &#34;establishes over individuals a visibility through which one differentiates them and judges them.&#34;<xref ref-type="fn" rid="n57">57</xref> It is the scientific or quasi-scientific fixing of differences and their representation in numbers, charts, and reports, which provides the basis for administrative decision-making.</p>
<p>As an administrative technology, described by Foucault as a &#34;scientific or administrative inquisition which determines who one is,&#34;<xref ref-type="fn" rid="n58">58</xref> classical profiling works as a productive technology of power. It creates objects of knowledge, allows classifying and ranking these objects, and turns individuals into &#34;cases&#34; that can be handled and managed in particular ways. Emerging profiles are inherently normative constructions that have intrinsic moral value since they are &#34;kinds that people want to be or not.&#34;<xref ref-type="fn" rid="n59">59</xref> Like stereotyping, profiling works as an a priori ordering that tells us who &#34;is&#34; good/bad, trustworthy/untrustworthy, healthy/unhealthy, suspicious/un-suspicious, profitable/not-profitable, etc. Profiles also provide those who are profiled with an image of themselves. They allow subjects to reflect on themselves and to monitor their behavior in the light of normative expectations. In this sense, profiles create an imaginary ideal (&#34;<italic>id&#233;al speculative</italic>&#34;) that works as a subjectifying force.<xref ref-type="fn" rid="n60">60</xref></p>
<p>In &#34;new profiling,&#34;<xref ref-type="fn" rid="n61">61</xref> profiles are often the result of automated and algorithmically enhanced systems of pattern recognition and -discrimination.<xref ref-type="fn" rid="n62">62</xref> This presupposes the transformation of the flux, flow, and mess of real-life events into data that are readable by machines. Knowledge in this case is inductive. It is shaped by the algorithms, which direct attention, focus on specific points in the data set and cancel out other data. Profiles in this sense are less about measurement but about &#34;detection.&#34; In contrast to traditional profiling, categorization emerges ex-post, from clustering detected patterns. As Hildebrandt explains, &#34;Pattern recognition, based on &#39;blind&#39; correlations (i.e., correlations that do not derive form predefined hypotheses and do not necessarily imply causes or reasons) allows those that use the ensuing profiles to anticipate the state or behavior of the objects or subjects that are profiled.&#34;<xref ref-type="fn" rid="n63">63</xref></p>
<p>Traditional profiling follows the logic of inspection. It seeks to identify the presence of certain predefined characteristics in the individual and categorize it accordingly. In contrast, new profiling follows the forward-looking logic of prospection. It is based on the assumption that detected patterns and correlations allow creating &#34;predictive transparency,&#34;<xref ref-type="fn" rid="n64">64</xref> which provides the basis for preemptive decision-making and anticipatory governing. Predictions are derived from patterns in past behavior or they are derived from similar patterns of &#34;groups&#34; or &#34;neighbors.&#34; Categorizations thus not only depend on individual actions, behaviors and histories, but on those of others who are similar to him or her.<xref ref-type="fn" rid="n65">65</xref> The generated knowledge is based on probability rather than on the construction of causalities. In short, while traditional profiling seeks to determine, who or what the person <italic>is</italic>, new profiling &#34;render(s) us <italic>transparent</italic> in a rather counterintuitive manner. We become transparent in the sense that the profiling software looks straight <italic>through</italic> us to &#39;what we are like,&#39; instead of &#39;what or who we are.&#39;&#34;<xref ref-type="fn" rid="n66">66</xref></p>
<p>New profiling, based on pattern recognition, constructs or assembles a person of interest by connecting data-points and visualizes the person for example on a screen, a scoring sheet or a risk map. Algorithms function as means for directing attention and for focusing on specific data while neglecting or cancelling out all other data. In the case of new profiling, they make up people in the form of &#34;data doubles&#34; or &#34;data derivatives.&#34;<xref ref-type="fn" rid="n67">67</xref> Such objects of knowledge can be generated from a variety of different and heterogeneous sources (e.g. discrete digital acts like searching, messaging, blogging, purchasing, liking, tweeting, and posting, etc.). Profiles are composed by combining these actions and connecting bits and pieces of data. They are not representations of the &#39;real&#39; person or of &#39;individuals,&#39; but selective constructions which consist themselves out of sub-individual elements, the fragments of registered behavior, which are extracted from the flow of data for specific purposes. They resemble the coded &#34;dividual&#34;<xref ref-type="fn" rid="n68">68</xref> which, according to Deleuze, replaces the undivided in-dividual in the &#34;society of control.&#34; The <italic>dividual</italic> consists of separate data-points, which can be scrutinized, calculated, circulated, assembled, and reassembled for specific purposes. As Hildebrandt explains, &#34;these elements or properties are not given; they are attributed by whoever writes the algorithms of data analytics; taking into account that whatever dividuals are sought after they must be inferred from machine-readable data by machine-readable algorithms.&#34;<xref ref-type="fn" rid="n69">69</xref></p>
<p>The aim is not to produce exhaustive knowledge about a particular individual and his or her intrinsic characteristics, but to create a basis for acting on similar individuals. These profiles are not a reflection of a given identity, but a projection of possible future behavior. Amoore stresses that the digital alter ego that emerges from algorithmic profiling &#34;is precisely a projected person: an image of a potential future person as yet to come,&#34;<xref ref-type="fn" rid="n70">70</xref> e.g., the potential terrorist, the potential criminal, the potential buyer, the potential toxic employee, the potential credit risk, etc.</p>
<p>While traditional profiling fixes the individual in a relatively stable normative matrix, profiles that emerge from clustering patterns &#34;create momentary groupings that might disappear into the white noise of the database in the next moment.&#34;<xref ref-type="fn" rid="n71">71</xref> They therefore do not constitute a stable and fixed norm that defines and disciplines bodies according to a predefined model, but rather create a fluctuating network of categories that modulate behavior. In terms of Deleuze, such &#34;modulation&#34; functions &#34;like a self-transmuting molding continually changing from one moment to the next, or like a sieve whose mesh varies form one point to another.&#34;<xref ref-type="fn" rid="n72">72</xref> In such dynamic networks, subjects can be assessed and sorted for specific purposes on a continuous basis. Categorizations are themselves evolving and shifting depending on feed-back data and the logic embedded in the algorithm.</p>
<p>What is called &#34;autonomic profiling&#34;<xref ref-type="fn" rid="n73">73</xref> goes one step further. Here, algorithmic functions are not only applied for the purpose of detecting patterns and generating profiles, but also for adapting the algorithms to changing environments and for automatic sorting and treatment of profiles, for example in the form of a differentiated or &#34;(micro)targeted&#34; response that is machine-generated.</p>
<p>An organizational example that comes close to autonomic profiling can be seen in concepts of &#34;people analytics.&#34;<xref ref-type="fn" rid="n74">74</xref> The individual here is a &#34;walking data generator.&#34;<xref ref-type="fn" rid="n75">75</xref> Sensor technologies automatically extract data from moving bodies. The algorithm built into the technology mines the data and autonomously transforms extracted data into a virtual profile, which is continuously updated and modified. People analytics visualizes individuals and teams and makes them comparable, measurable, and manageable. In this version, data-traces left by the moving bodies are calibrated as &#34;derivatives.&#34; Amoore has described the specific form of abstraction that characterizes this form of creating profiles: &#34;The data derivative comes into being from an amalgam of disaggregated data&#8212;reaggregated via mobile algorithm-based association rules and visualized in &#39;real time&#39; as risk map, score or colour coded flag.&#34;<xref ref-type="fn" rid="n76">76</xref> In Table <xref ref-type="table" rid="T1">1</xref> the main differences are summarized.</p>
<table-wrap id="T1">
<label>Table 1</label>
<caption>
<p>Traditional and Data-driven Profiling.</p>
</caption>
<table>
<tr>
<th align="left" valign="top">Traditional/Disciplinary</th>
<th align="left" valign="top">&#39;New profiling&#39;/Data-driven</th>
</tr>
<tr>
<td colspan="2"><hr/></td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Deductive</td>
<td align="left" valign="top">&#8226; Inductive</td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Causality</td>
<td align="left" valign="top">&#8226; Correlation</td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Inspection</td>
<td align="left" valign="top">&#8226; Prospection</td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Fixing of difference</td>
<td align="left" valign="top">&#8226; Momentary patterns</td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Relatively stable matrix of identities</td>
<td align="left" valign="top">&#8226; Web of fluctuating categories</td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Examination</td>
<td align="left" valign="top">&#8226; Continuous assessment</td>
</tr>
<tr>
<td align="left" valign="top">&#8226; Administrative decision-making</td>
<td align="left" valign="top">&#8226; Preemptive decision-making</td>
</tr>
</table>
</table-wrap>
<p>In sum: In particular data-driven profiling based on (algorithmic) pattern recognition illustrates how emerging profiles are increasingly decoupled from the social relations from which they are drawn. Both, the process of arriving at categories and the multiple decisions about what becomes visible or invisible, what to select and what to exclude within the system become black-boxed and disappear in the created image. Such profiles emerge as &#34;artefactual bodies&#34; that can be and are disconnected from the individual. They can be combined with other profiles and they can be used for different purposes in different contexts. They can be continuously modified and they are extremely mobile, particularly in their electronic form. Thus, profiles become &#34;prediction products&#34;<xref ref-type="fn" rid="n77">77</xref> that circulate in cyberspace, can be sold and traded and turned into capital.</p>
</sec>
<sec>
<title>Application of Profiles</title>
<p>As pointed out, profiles are not descriptive but performative. They are not only a product of worldly and material processes, but in turn they do work <italic>in</italic> the world where they have material effects. They generate a frame for decision makers, produce the &#34;evidence&#34; to which decision-makers can refer to, they instruct decision-makers on what to see and what to ignore and they &#34;act back on those with whom data are associated, informing us who we are, what we should desire or hope for, including who we should become.&#34;<xref ref-type="fn" rid="n78">78</xref> Application is the process in which profiles perform their work of informing or shaping decision-making in various places. Application is often seen as a technical, linear process in which context-specific &#39;solutions&#39; or decisions are derived from abstract rules or concepts, or in our case, from profiles. By contrast, Derrida has suggested &#34;a new concept of application which is in agreement with dissemination&#34; and implies &#34;a total spectral structure.&#34;<xref ref-type="fn" rid="n79">79</xref> Such a concept of application leads beyond the mere technical or instrumentalist understandings and considers application as a creative and performative process, which generates something unpredictable in &#34;contexts which nobody can master in advance.&#34;<xref ref-type="fn" rid="n80">80</xref></p>
<p>In this process, profiles travel through time and space and have expected and unexpected effects. As ghosts, they are not bound by time and space. They can be created now, but return later; they can be produced here, but return elsewhere. Similarly, Amoore reminds us:</p>
<disp-quote>
<p>Data are not only gathered in the strict sense of data collection, but they assemble themselves with multiple other elements and things. Data do unexpected things with unanticipated effects. Stop the wrong person at the border, fail to stop the right person, gather false positives and then let them loose into the world, invite some intuitions and inferences from observers and banish others.<xref ref-type="fn" rid="n81">81</xref></p>
</disp-quote>
<p>As data and profiles move from context to context, they can be interpreted and framed in different ways. Profiles can mean different things in different contexts. A regular buyer of antidepressant drugs can be an attractive customer for the pharmaceutical industry, but can be classified as an employment risk in other cases; a credit score can become a proxy for the trustworthiness of persons etc. Even though, any interpretation or sensemaking is context-bound, context cannot be fixed. Every profile is a sign or a mark that signifies some social status&#8212;creditworthiness, employability, riskiness, profitability, etc. As such, it can be removed from its &#34;original&#34; context. It can break from a given context and can be redeployed in new contexts; data collected for one specific purpose can be subsequently used for another authorized or unauthorized purpose, a phenomenon that has been described as &#34;function creep.&#34;<xref ref-type="fn" rid="n82">82</xref> As is frequently demonstrated, profiles can easily flow between so-called &#34;data sharing partnerships&#34; that are created for optimizing the user experience. In moving from context to context profiles do not move in a vacuum, but they can be, and actively are transferred and transformed in the context of other profiles, actors, traditions, and institutions. The de/re-contextualization of profiles may change not only the content (the data-body) but also the ascribed meaning of the profile. Thus, when profiles travel form context to context, they may also be reframed according to specific interests and ideologies. Profiles are scrutinized, assembled, and reassembled for specific purposes in a variety of distributed &#34;centres of calculation,&#34;<xref ref-type="fn" rid="n83">83</xref> which&#8212;like <italic>Cambridge Analytica</italic>&#8212;seek to develop a knowledge base for commercial or political strategies for influencing and shaping behavior of customers or voters or, more generally, for nudging people to make decisions, which decision architects consider good decisions.</p>
<p>In the context of &#34;surveillance capitalism,&#34;<xref ref-type="fn" rid="n84">84</xref> profiles are the form of capitalizing data. Digital traces are mined and transformed into profiles that can be bought and sold, packaged and repackaged for specific purposes. Since every piece of data is of potential value for someone it might end up in some profile that is useful and can be sold to somebody. The monetization is the business of various data brokers, direct-marketers and other organizations and companies that sell and resell profiles to interested parties.<xref ref-type="fn" rid="n85">85</xref> In these conditions the emerging profiles are further removed from their primary source or context. As the legal scholar Frank Pasquale says, there are endless permutations:</p>
<disp-quote>
<p>Profiling might begin with the original collectors of information, but it can be elaborated by numerous data brokers, including credit bureaus, analytic firms, catalog co-ops, direct marketers, list brokers, affiliates, and others. Brokers combine, swap, and recombine the data they acquire into new profiles, which they then can sell back to the original collectors or to other firms.<xref ref-type="fn" rid="n86">86</xref></p>
</disp-quote>
<p>What Pasquale has called &#34;runaway profiles&#34; illustrates the spectral logic of profiling, where data are decoupled and torn loose from their context in ways that make it impossible to understand how data from various sources are transformed into a profile, on what kinds of decisions, assumptions and prerogatives the constructed object has been formed, how it has been transformed and modified according to technical, economic or political considerations. While the decision process in itself becomes obscure and black-boxed, profiling and profiles&#8212;particularly when produced by &#34;data-intelligence&#34;&#8212;produce and reproduce the &#34;illusions of transparency,&#34;<xref ref-type="fn" rid="n87">87</xref> the promises of predictability and governability that support a (neoliberal) regime of truth and justify decisions and demarcations on the basis of seemingly rational and neutral calculations.</p>
</sec>
<sec>
<title>Profiles as Ghostly Demarcations</title>
<p>When abstractions circulate and are materialized (i.e., when the profiles are &#39;applied&#39;) they have powerful effects. They serve to allow or to deny access to services and benefits, they generate credibility or suspicion, and they provide the basis for a differential treatment of people. In particular, runaway profiles &#34;can lead to <italic>cascading disadvantages</italic> as digital alchemy creates new analogous realities.&#34;<xref ref-type="fn" rid="n88">88</xref> Once a person is attributed the label of credit risk, toxic worker, marginal consumer, potential dropout or high-risk traveler, that attribute may appear with decisional force on various occasions throughout the social field. Profiles&#8212;regardless of whether we are talking about traditional (disciplinary) or about data-driven profiling, based on pattern recognition, mining of data, etc.&#8212;are discriminatory technologies that work as &#34;social sorting.&#34;<xref ref-type="fn" rid="n89">89</xref> Profiling sorts people into social categories, so that the classification allows distributing opportunities and risks. While algorithmic and data-driven profiling is promoted as means to overcome human irrationality, to optimize judgmental accuracy and to produce more fine grained and more objective analyses, it is perhaps better understood as a movement in which mechanisms that generate demarcations become increasingly opaque and incomprehensible for those who are objects of profiling.</p>
<p>In the context of security governance, Leese has identified a triple &#34;loss&#34; that is associated with data-driven profiling (in comparison to traditional profiling).<xref ref-type="fn" rid="n90">90</xref> First, there is a &#34;<italic>loss of traceability</italic>.&#34; With data-driven profiling, it is increasingly difficult to determine where data from which profiles are assembled come from, since they can literally come from everywhere. It is increasingly difficult to know and understand how categories are defined and how classifications are produced. Second, there is a &#34;<italic>loss of visibility</italic>.&#34; Profiling increasingly operates out of sight. Data-mining procedures produce &#34;artificial categories,&#34; the individual is likely not even to notice when or how he or she becomes categorized as risky, profitable, suspicious, etc. Consequently, most of the time people will not even notice if or how they have been discriminated against. Third, there is a &#34;<italic>loss of accountability</italic>.&#34; In many cases, the human decision-maker is &#34;rendered likely to comply with the truth claims of the algorithm&#34;<xref ref-type="fn" rid="n91">91</xref> even if &#34;autonomy&#34; is attributed to him or her. The ability of individuals to reflect or challenge decisions vanishes to the degree that the process of making them becomes impenetrable. An inherent opacity characterizes predictive algorithms, particularly in the case of machine learning.<xref ref-type="fn" rid="n92">92</xref> Even experts are often overwhelmed by the complexity to comprehend the internal procedures of algorithmic systems. In many cases, it is difficult, if not impossible, to give an account of what the profile means and how it has been constructed.</p>
<p>I would add that there is&#8212;paradoxically&#8212;also a <italic>loss of predictability</italic>. While data-driven profiling seeks to make actions, thoughts, feeling, and behavior of citizens (customers, voters, debtors, employees, etc.) predictable, it becomes highly unpredictable how, when and for what reason created profiles will affect those who are profiled. Occasionally, we realize that we are in the visor of specific organizations and are targets of specific profiling machines. This happens, for example, when we are sent an individualized or personalized message, when we are offered a tailored product, are singled out as a risky passenger at the airport, or, when we are denied access to public spaces, insurance, employment, credit etc. because we (or our profile) does not fit to the criteria defined in advance by anonymous agencies. Moreover, there is a <italic>loss of response-ability</italic>. The ability to respond to the concrete other is a precondition for exercising or enacting moral responsibility. In his analysis of the Holocaust Bauman<xref ref-type="fn" rid="n93">93</xref> has analyzed how bureaucratic procedures and abstract classifications work as &#34;moral sleeping pills.&#34; According to Bauman, dehumanization starts when people are objectified and reduced to a set of quantitative measures. In data-driven profiling where people and social relations are transformed into bits of information and electronic impulses, dehumanization works on a more abstract level. When the person appears on screen as a result of algorithmic visualization, the Other as a &#34;face&#34; (in Levinas&#39; sense of &#34;an authority without force&#34;<xref ref-type="fn" rid="n94">94</xref> that demands a moral response) disappears completely and moral responsibility for the Other is suspended and rendered ineffective.</p>
<p>To sum up: Profiling increasingly creates an &#34;invisible visibility&#34; that is associated with what Derrida has called the &#34;visor effect: we do not see who looks at us.&#34;<xref ref-type="fn" rid="n95">95</xref> We are observed and increasingly denied the possibility to understand, question, or challenge how, why, when, by whom we are observed and evaluated by what criteria. This also has consequences for the modes of subjectification and the specific ways in which truth and power are linked to each other.</p>
</sec>
</sec>
<sec>
<title>4. Multiple Profiling(machines): Proto-paranoiac Subjectification and Hyper-facticity</title>
<p>Today, profiles and profiling are used in many different contexts: customer profiling, profiling for employment screening, credit scoring, criminal investigations, immigration policy, healthcare management, forensic biometrics, etc. The emerging arrangement of multiple profiles and &#34;profiling machines&#34;<xref ref-type="fn" rid="n96">96</xref> that create <italic>specific</italic> forms of visibility of subjects modifies the &#34;play of light and darkness&#34; which Foucault has related to the transparency ideal embodied in the panopticon.<xref ref-type="fn" rid="n97">97</xref> Hierarchical observation and normalizing judgement characterize its diagram. Activities are captured in reports and registers which provide the basis for comparisons and judgements. The context of multiple profiling machines displays a new diagram, in which the hierarchical and fixing panoptic gaze, which oversees the field from above, is replaced by a multiplicity of gazes, coming from a multiplicity of observing agencies, which in addition are movable, and continuously changing their position. This is the condition emerging from <italic>multiple profiling machines</italic>, in which subjects appear in the light of various and changing searchlights. In the contemporary &#34;digital prism&#34;<xref ref-type="fn" rid="n98">98</xref> of the datafied world, each of these searchlights constitutes subjects as persons of interest for specific purposes. While the panoptic gaze fixes individuals and objectifies them by pigeonholing the observed into predefined boxes (categories), &#34;new profiling&#34; generates observational categories from below, by mining and detecting patterns in data traces and footprints. Kitchin has noted that these footprints and shadows are &#34;dispersed, divided across dozens of organizations and servers, and are subject to integration and division. [&#8230;] At best they constitute oligopticons&#8212;limited views form partial vantage points from fixed positions with defined view sheds.&#34;<xref ref-type="fn" rid="n99">99</xref> Ellerbrok has described a similar, but perhaps more dynamic scenario of &#34;multiple variable visibilities&#34;<xref ref-type="fn" rid="n100">100</xref> that is generated by new media technologies that render individuals visible to diverse interests simultaneously, on multiple levels that are interacting dynamically.</p>
<p>The multiplication of profiling (machines) and the proliferation of data-driven decision making fundamentally affects processes of subjectification and the way the truth-game is linked to the &#34;art of governing.&#34;</p>
<sec>
<title>Subjectification</title>
<p>Rouvroy has argued that algorithmic forms of profiling affect people in so many different contexts, yet <italic>without</italic> addressing them as persons directly. In her view, data-driven profiling circumvents subjectivity and subjectification, and in some sense even works &#34;without subjectivity.&#34;<xref ref-type="fn" rid="n101">101</xref> Algorithmic governmentality, she says, &#34;bypasses consciousness and reflexivity, and operates on the mode of alerts and reflexes.&#34;<xref ref-type="fn" rid="n102">102</xref> Yet it can also be argued that the emerging regime of truth creates conditions that are related to a specific mode of subjectification which &#381;i&#382;ek has called &#34;proto-paranoiac&#34;:</p>
<disp-quote>
<p>If, then, today, in the guise of detailed databases [profiles, R.W.] that circulate in the corporate cyberspace [&#8230;] we are [&#8230;] &#39;interpellated&#39; by institutions even without being aware of it, one should nevertheless insist that this &#39;objective interpellation&#39; actually affects my subjectivity only by means of the fact that I <italic>myself am well aware of how, outside the grasp of my knowledge, databases circulate which determine my symbolic identity in the eyes of the social</italic> &#39;<italic>big Other</italic>&#39;. My very awareness of the fact that &#39;the truth is out there&#39;, that files on me circulate which, even if they are factually &#39;inaccurate&#39;, none the less performatively determine my socio-symbolic status, is what gives rise to the specific proto-paranoiac mode of subjectification characteristic of today&#39;s subject: it constitutes me as a subject inherently related to and hassled by an elusive piece of database in which, beyond my reach, &#39;my fate is writ large&#39;.<xref ref-type="fn" rid="n103">103</xref></p>
</disp-quote>
<p>But the conditions of subjectification are even more paradox. On the one hand there is the phenomenon that &#381;i&#382;ek describes. On the other hand, these data practices are widely regarded as normal, necessary or beneficial. There are a lot of incentives for making ourselves visible to various others, or to (actively) profile ourselves. In order to proof our creditworthiness or attractiveness to employers or potential partners, for example we provide our data and actively create our profiles. This is a new form of truth obligation that modifies the confession which according to Foucault &#34;became one of the West&#8217;s most highly valued techniques for producing the truth,&#34;<xref ref-type="fn" rid="n104">104</xref> which constitutes subjects in the double sense of the word. In the digital context one confesses in the form of providing data. Often this is not forced, but happens because of ignorance, laziness or indifference. The promise here is not salvation in the other world, but the promise is to get access to relevant services (e.g. credit, employment, friends, etc.) and pleasures in <italic>this</italic> world. In the digital context, the obligatory act of speech turns in many cases into a desiring self-exposure and exhibition of intimate details.<xref ref-type="fn" rid="n105">105</xref></p>
</sec>
<sec>
<title>Governing according to the rules of evidence</title>
<p>Data-driven profiling provides the basis for a new mode of governing or, in Foucault&#39;s sense, a new &#34;way of linking the art of governing and the game of truth.&#34;<xref ref-type="fn" rid="n106">106</xref> It is a way of linking them, which at first sight seems paradox:</p>
<disp-quote>
<p>It is the idea that, if in actual fact the government governs not through wisdom in general, but through the truth, that is to say through the exact knowledge of the processes that characterize the reality [, &#8230;] then it will have to govern even less. The more it pegs its action to the truth, the less it will have to take decisions that have to be imposed from above, in accordance with more or less uncertain calculations on people who will accept them more or less well.<xref ref-type="fn" rid="n107">107</xref></p>
</disp-quote>
<p>Foucault traces this way of linking the art of governing and truth to the (physiocratic) idea, of &#34;govern(ing) according to the rules of evidence.&#34; According to this idea, it would ideally be &#34;things in themselves, rather than men, that govern.&#34; This, of course implies a utopian moment in which the exercise of power is nothing more than an &#34;indicator of the truth.&#34;<xref ref-type="fn" rid="n108">108</xref></p>
<p>Today, in the context of the regime of practices which Zuboff<xref ref-type="fn" rid="n109">109</xref> has called &#34;surveillance capitalism,&#34; a similar utopian moment is present in the work of prophets of the new data-driven society, who believe that total knowledge is within reach. They argue that &#34;Big data give us a chance to view society in all its complexity, through the millions of networks of person-to-person exchanges [&#8230;] we could potentially arrive at a true understanding of how society works and take steps to fix our problems.&#34;<xref ref-type="fn" rid="n110">110</xref> &#34;Evidence&#34; (i.e., truth) is derived from a multiplicity of data (Big data), drawn from a huge variety of sources and brought into a specific form by algorithms. It is thus, in the terms of Foucault, a specific form of evidence that is linked to a specific &#34;modality of veridiction.&#34;<xref ref-type="fn" rid="n111">111</xref> Evidence here is not produced by or attributed to personal experience or courageous truth-telling (the parrhesiastic modality), it is also not produced with reference to some general form of knowledge (the modality of wisdom) nor by referring to professional expertise and (causal) knowledge (the technical modality). Rather, evidence is associated with the &#34;objectivity&#34; of data and the effectiveness of computational procedures. In this context, there is a multiplicity of generators of evidence, which provide the basis for a form of governance that is based on the assumption that &#34;computational truth must necessarily replace politics as the basis of instrumentarian governance.&#34;<xref ref-type="fn" rid="n112">112</xref> In the context of ubiquitous computing, the idea that things in themselves govern takes on a new form that goes beyond Foucault&#39;s analyses. The more the things that surround us become &#34;intelligent&#34;&#8212;from cars to refrigerators, from cellphones to Alexa, etc. the more, Zuboff remarks, the &#34;things that <italic>we</italic> have&#34; are turned into &#34;things that have <italic>us</italic>.&#34;<xref ref-type="fn" rid="n113">113</xref> Increasingly, driven by the utopia of &#34;N = all&#34; the whole spectrum of our experience, our homes and our bodies, our movements and our emotions, etc. are transformed into objects that can be calculated, processed and managed in the name of profit. Equipped with what Zuboff calls &#34;actuation capability,&#34; technologies like smart sensors are interlinked and can register and analyze any kind of human behavior and then actually figure out how to change it. Automatically generated impulses and &#34;nudges&#34; condition and incentivize behavior and bypass human reflexivity. Governing the conduct of individuals, influencing, shaping, forming conduct may ultimately become automatized and freed from the incertitude and strain of interpretation, negotiation, and responsibility. This is ultimately hyper-facticity at work.</p>
</sec>
</sec>
<sec>
<title>5. Conclusion</title>
<p>Rouvroy speaks of an emerging &#34;algorithmic governmentality,&#34; which allows the exercise of a specific form of power based on knowledge generated by data-driven profiling. She characterized this specific regime of power as &#34;spectral.&#34;<xref ref-type="fn" rid="n114">114</xref> In this paper I have added Derrida&#39;s analysis of the ghost and the &#34;spectrogenic process&#34; as a supplement for illuminating the spectral character of this form of governmentality and for understanding the effects of power of algorithmic profiling in particular. I have shown how algorithmic profiling creates &#34;ghostly demarcations&#34; that haunt our present condition. The specific modus operandi of profiling signifies a shift in exercising power. While classical profiling follows the disciplinary logic of positing an ideal model and defining a norm against which reality is measured and corrected, new profiling follows the logic of security,<xref ref-type="fn" rid="n115">115</xref> which is based on the calculation of probabilities. The norm is derived from the observation of the real, using statistical methods and data-processing systems, for generating a dynamic and fluctuating network of categories that organize observation from various interested perspectives. Power in this sense is neither prohibitive (like the law) nor prescriptive (like discipline). It rather has the character of anticipatory regulation and preemption. In Foucault&#39;s words, it is a matter of &#34;allowing circulations to take place, of controlling them, sifting the good and the bad, ensuring that things are always in movement, constantly moving around, continually going from one point to another, but in such a way that the inherent dangers of this circulation are cancelled out.&#34;<xref ref-type="fn" rid="n116">116</xref></p>
<p>In the introduction I have mentioned the example of <italic>Cambridge Analytica</italic>. Such companies have built their business model on optimizing the flow of populations and on developing and selling predictive profiles that allow to &#34;sift the good and the bad&#34; and to influence and even produce specific forms of behavior and subjectivity behavior (&#34;We use data to change audience behavior&#34;). In the final statement at the end of a YouTube Video, which shows Alexander Nix, the former CEO of CA celebrating the range of possibilities and promoting methods for influencing and shaping the &#34;target audience,&#34; Nix insinuates that it was one candidate in particular who made use of these methods in the 2016 US election campaign. Retrospectively, it is remarkable, that it was exactly the one candidate that has become the epitome and symbolic figure of post-facticity and post-truth: Donald Trump. It seems that post-facticity&#8212;the syndrome of lies, bullshit, simulated truth-telling and &#34;alternative facts&#34;&#8212;and hyper-facticity go very well together. Perhaps, post-facticity is even the ghost that haunts hyper-facticity.</p>
</sec>
</body>
<back>
<fn-group>
<fn id="n1"><p>George Orwell, <italic>Nineteen Eighty Four</italic> (<xref ref-type="bibr" rid="B59">Essex: Heinemann, 1949/1990</xref>), 206.</p></fn>
<fn id="n2"><p>Wu Youyou, Michal Kosinski, and David Stillwell, &#34;Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans,&#34; <italic>PNAS</italic> 112, no. 4 (<xref ref-type="bibr" rid="B67">2015</xref>): 1036&#8211;39.</p></fn>
<fn id="n3"><p>Youyou, Kosinski, and Stillwell, &#34;Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans,&#34; 1039.</p></fn>
<fn id="n4"><p>Viktor Mayer-Sch&#246;nberger and Kenneth Cukier, <italic>Big Data. A Revolution That Will Transform How We Live, Work and Think</italic> (<xref ref-type="bibr" rid="B54">London: John Murray, 2013</xref>), 161.</p></fn>
<fn id="n5"><p>Youyou, Kosinski, and Stillwell, &#34;Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans,&#34; 1039.</p></fn>
<fn id="n6"><p>Carol Cadwalladr, &#34;The Cambridge Analytica Files. &#39;I Made Steve Bannon&#39;s Psychological Warfare Tool&#39;: Meet the Data War Whistleblower,&#34; <italic>The Guardian</italic>, <xref ref-type="bibr" rid="B12">March 18, 2018</xref>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.theguardian.com/news/2018/mar/17/data-war-whistleblower-christopher-wylie-faceook-nix-bannon-trump">https://www.theguardian.com/news/2018/mar/17/data-war-whistleblower-christopher-wylie-faceook-nix-bannon-trump</ext-link>.</p></fn>
<fn id="n7"><p>Oscar H. Gandy, <italic>The Panoptic Sort: A Political Economy of Personal Information. Critical Studies in Communication and in the Cultural Industries</italic> (<xref ref-type="bibr" rid="B33">Bolder: Westview Press, 1993</xref>).</p></fn>
<fn id="n8"><p>Antoinette Rouvroy, &#34;The End(s) of Critique. Data-Behaviourism Vs. Due-Process,&#34; in <italic>Privacy, Due Process and the Computational Turn. Philosophers of Law Meet Philosophers of Technology</italic>, ed. Marielle Hildebrandt and Katja de Vries (<xref ref-type="bibr" rid="B42">London: Routledge, 2013</xref>), 146.</p></fn>
<fn id="n9"><p>Rouvroy, &#34;The End(s) of Critique. Data-Behaviourism Vs. Due-Process,&#34; 152.</p></fn>
<fn id="n10"><p>James Ball, <italic>Post-Truth. How Bullshit Conquered the World</italic> (<xref ref-type="bibr" rid="B5">London: Biteback, 2017</xref>), 5.</p></fn>
<fn id="n11"><p>Michel Foucault, &#34;Questions of Method,&#34; in <italic>The Foucault Effect</italic>, ed. Graham Burchell, Colin Gordon, and Peter Miller (<xref ref-type="bibr" rid="B29">London: Harvester Wheatsheaf, 1991</xref>), 79.</p></fn>
<fn id="n12"><p>Michel Foucault, <italic>On the Government of the Living. Lectures at the Coll&#232;ge the France 1979</italic>&#8211;<italic>1980 and Oedipal Knowledge</italic> (<xref ref-type="bibr" rid="B28">New York: Picador, 2016</xref>), 15.</p></fn>
<fn id="n13"><p>Dirk Lindebaum, Mikko Vesa, and Frank den Hond, &#34;Insights from &#39;The Machine Stops&#39; to Better Understand Rational Assumptions in Algorithmic Decision Making and Its Implications for Organizations,&#34; <italic>Academy of Management Review, no.</italic> 1 (<xref ref-type="bibr" rid="B50">January 2020</xref>): 247&#8211;263; Sue Newell and Marco Marabelli, &#34;Strategic Opportunities (and Challenges) of Algorithmic Decision-Making: A Call for Action on the Long-Term Societal Effects of &#39;Datafication&#39;,&#34; <italic>Journal of Strategic Information Systems</italic> 24 (<xref ref-type="bibr" rid="B57">2015</xref>).</p></fn>
<fn id="n14"><p>Michael Sprinker, ed. <italic>Ghostly Demarcations. A Symposium on Jacques Derrida&#39;s Specters of Marx</italic> (<xref ref-type="bibr" rid="B65">London: Verso, 1999</xref>).</p></fn>
<fn id="n15"><p>Mireille Hildebrandt, &#34;Defining Profiling: A New Type of Knowledge,&#34; in <italic>Profiling the European Citizen</italic>, ed. Marielle Hildebrandt and Serge Gutwirth (<xref ref-type="bibr" rid="B41">Berlin: Springer, 2008</xref>). Mathias Leese, &#34;The New Profiling: Algorithm, Black Boxes, and the Failure of Anti-Discriminatory Safeguards in the European Union,&#34; <italic>Security Dialogue</italic> 45, no. 5 (<xref ref-type="bibr" rid="B49">2014</xref>).</p></fn>
<fn id="n16"><p>Jacques Derrida, <italic>Specters of Marx. The State of the Debt, the Work of Mourning, &amp; the New International</italic> (<xref ref-type="bibr" rid="B20">New York, London: Routledge, 1994</xref>).</p></fn>
<fn id="n17"><p>Steffen Mau, <italic>Das metrische Wir. &#220;ber die Quantifizierung des Sozialen</italic> (<xref ref-type="bibr" rid="B53">Frankfurt a.M.: Suhrkamp, 2017</xref>).</p></fn>
<fn id="n18"><p>Shoshana Zuboff, <italic>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</italic> (<xref ref-type="bibr" rid="B69">New York: Public Affairs, 2019</xref>).</p></fn>
<fn id="n19"><p>Zuboff, <italic>The Age of Surveillance Capitalism</italic>, 96.</p></fn>
<fn id="n20"><p>Zuboff, <italic>The Age of Surveillance Capitalism</italic>, 65.</p></fn>
<fn id="n21"><p>Newell and Marabelli, &#34;Strategic Opportunities (and Challenges) of Algorithmic Decision-Making: A Call for Action on the Long-Term Societal Effects of &#39;Datafication&#39;,&#34; 4.</p></fn>
<fn id="n22"><p>John Cheney-Lippold, &#34;A New Algorithmic Identity. Soft Biopolitics and the Modulation of Control,&#34; <italic>Theory, Culture &amp; Society</italic> 28, no. 6 (<xref ref-type="bibr" rid="B13">2011</xref>): 167&#8211;72.</p></fn>
<fn id="n23"><p>Martin Edwards and Kirstin Edwards, <italic>Predictive HR Analytics: Mastering the HR Metric</italic> (<xref ref-type="bibr" rid="B21">London: Kogan Page, 2019</xref>).</p></fn>
<fn id="n24"><p>Andrew McAfee and Erik Brynjolfsson, &#34;Big Data: The Management Revolution,&#34; <italic>Harvard Business Review</italic> (<xref ref-type="bibr" rid="B55">October 2012</xref>): 5, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://hbr.org/2012/10/big-data-the-management-revolution">https://hbr.org/2012/10/big-data-the-management-revolution</ext-link>.</p></fn>
<fn id="n25"><p>Newell and Marabelli, &#34;Strategic Opportunities (and Challenges) of Algorithmic Decision-Making: A Call for Action on the Long-Term Societal Effects of &#39;Datafication&#39;;&#34; <xref ref-type="bibr" rid="B14">Wendy Hui Kyong Chun</xref>, &#34;Queering Homophily,&#34; in <italic>Pattern Discrimination. In Search of Media</italic>, ed. Clemens Apprich et al. (<xref ref-type="bibr" rid="B4">Minneapolis: University of Minnesota Press, 2019</xref>).</p></fn>
<fn id="n26"><p>Rob Kitchin, <italic>The Data Revolution. Big Data, Data Infrastructures &amp; Their Consequences</italic> (<xref ref-type="bibr" rid="B45">Los Angeles: Sage, 2014</xref>), 21.</p></fn>
<fn id="n27"><p>Jacques Derrida, &#34;Force of Law: The &#39;Mystical Foundation of Authority&#39;,&#34; in <italic>Deconstruction and the Possibility of Justice</italic>, ed. Drucilla Cornell, Michel Rosenfeld, and David Gray Carlson (<xref ref-type="bibr" rid="B19">London: Routledge, 1992</xref>), 24.</p></fn>
<fn id="n28"><p>Louise Amoore, &#34;Lines of sight: on the visualization of unknown futures,&#34; <italic>Citizenship Studies</italic> 13, no. 1 (<xref ref-type="bibr" rid="B2">2009</xref>): 29.</p></fn>
<fn id="n29"><p>Lindebaum, Vesa, and den Hond, &#34;Insights from &#39;The Machine Stops&#39; to Better Understand Rational Assumptions in Algorithmic Decision-Making and Its Implications for Organizations,&#34; 248.</p></fn>
<fn id="n30"><p>Derrida, &#34;Force of Law: The &#39;Mystical Foundation of Authority&#39;,&#34; 24.</p></fn>
<fn id="n31"><p>Lisa Blackman, <italic>Haunted Data. Affect, Transmedia and Weird Science</italic> (<xref ref-type="bibr" rid="B8">London: Bloomsbury, 2016</xref>).</p></fn>
<fn id="n32"><p>Cheney-Lippold, &#34;A New Algorithmic Identity. Soft Biopolitics and the Modulation of Control,&#34; 171.</p></fn>
<fn id="n33"><p>Mikkel Flyverbom and Anders K. Madsen, &#34;Sorting Data Out: Unpacking Big Data Value Chains and Algorithmic Knowledge Production,&#34; in <italic>Die Gesellschaft der Daten. &#220;ber die digitale Transformation der sozialen Ordnung</italic>, ed. F. S&#252;ssenguth (<xref ref-type="bibr" rid="B25">Bielefeld: transcript, 2015</xref>).</p></fn>
<fn id="n34"><p>Clemens Apprich et al., <italic>Pattern Discrimination</italic> (<xref ref-type="bibr" rid="B4">Minneapolis: University of Minnesota Press, 2019</xref>).</p></fn>
<fn id="n35"><p>Geoffrey C. Bowker and Susan Leigh Star, <italic>Sorting Things Out</italic> (<xref ref-type="bibr" rid="B10">Cambridge, Mass.: MIT Press, 2000</xref>), 4.</p></fn>
<fn id="n36"><p>Kitchin, <italic>The Data Revolution. Big Data, Data Infrastructures &amp; Their Consequences</italic>, 9&#8211;10; Rob Kitchin, &#34;Thinking Critically About and Researching Algorithms,&#34; <italic>Information, Communication and Society</italic> 20, no. 1 (<xref ref-type="bibr" rid="B46">2017</xref>): 18.</p></fn>
<fn id="n37"><p>Ian Hacking, &#34;Kinds of People: Moving Targets,&#34; in <italic>Proceedings of the British Academy, Volume 151, 2006 Lectures</italic> (<xref ref-type="bibr" rid="B35">2007</xref>).</p></fn>
<fn id="n38"><p>Bernhard Rieder, &#34;Scrutinizing an Algorithmic Technique: The Bayes Classifier as Interested Reading of Reality, Information,&#34; <italic>Communication and Society</italic> 20, no. 1 (<xref ref-type="bibr" rid="B63">2017</xref>).</p></fn>
<fn id="n39"><p>Kitchin, &#34;Thinking Critically About and Researching Algorithms,&#34; 17.</p></fn>
<fn id="n40"><p>Hildebrandt, &#34;Defining Profiling: A New Type of Knowledge,&#34; 18.</p></fn>
<fn id="n41"><p>Andreas Bernard, <italic>The Triumph of Profiling. The Self in Digital Culture</italic> (<xref ref-type="bibr" rid="B7">Cambridge: Polity Press, 2019</xref>).</p></fn>
<fn id="n42"><p>Bernard E. Harcourt, <italic>Against Prediction. Profiling, Policing, and Punishing in an Actuarial Age</italic> (<xref ref-type="bibr" rid="B38">Chicago: University of Chicago Press, 2007</xref>).</p></fn>
<fn id="n43"><p>Colin Koopman, <italic>How We Became Our Data. A Genealogy of the Informational Person</italic> (<xref ref-type="bibr" rid="B47">Chicago: The University of Chicago Press, 2019</xref>).</p></fn>
<fn id="n44"><p>George S. Odiorne, <italic>Strategic Management of Human Resources</italic> (<xref ref-type="bibr" rid="B58">San Francisco: Jossey Bass, 1984</xref>), 66.</p></fn>
<fn id="n45"><p>Paula McDonald and Paul Thompson, &#34;Social Media(tion) and the Reshaping fo the Public/Private Boundaries in Employment Relations,&#34; <italic>International Journal of Management Reviews</italic>, Vol 18, <xref ref-type="bibr" rid="B56">2016: 69</xref>.</p></fn>
<fn id="n46"><p>John Cheeney-Lippold, &#34;A New Algorithmic Identity: Soft Biopolitics and the Modulations of Control,&#34; <italic>Theory, Culture &amp; Society</italic> 28, no. 6 (2011): 164&#8211;81.</p></fn>
<fn id="n47"><p>Hildebrandt, &#34;Defining Profiling: A New Type of Knowledge,&#34; 11.</p></fn>
<fn id="n48"><p>William Bogard, <italic>The Simulation of Surveillance. Hypercontrol in Telematic Societies</italic> (<xref ref-type="bibr" rid="B9">Cambridge: Cambridge University Press, 2010, 27</xref>.</p></fn>
<fn id="n49"><p>Bogard, <italic>The Simulation of Surveillance</italic>, 75, emphasis in original.</p></fn>
<fn id="n50"><p>James M. Harding, <italic>Performance, Transparency and the Cultures of Surveillance</italic> (<xref ref-type="bibr" rid="B40">Ann Arbor: University of Michigan Press, 2018</xref>), pos. 4061.</p></fn>
<fn id="n51"><p>Avery F. Gordon, <italic>Ghostly Matters: Haunting and the Sociological Imagination</italic> (<xref ref-type="bibr" rid="B34">Minneapolis: University of Minnesota Press, 1997</xref>), 19.</p></fn>
<fn id="n52"><p>Justine Gr&#248;nb&#230;k Pors, Lena Olaison, and Birke Otto, &#34;Ghostly Matters in Organizing,&#34; <italic>Ephemera. Critical Dialogs on Organization</italic> 19, no. 1 (<xref ref-type="bibr" rid="B62">2019</xref>).</p></fn>
<fn id="n53"><p>Blackman, <italic>Haunted Data. Affect, Transmedia and Weird Science</italic>.</p></fn>
<fn id="n54"><p>In <italic>Specters of Marx</italic>, Derrida (<xref ref-type="bibr" rid="B20">1994, 126</xref>) explains that the production of the ghost, or the &#34;constitution of the ghost effect&#34; is more than the autonomization of thoughts, ideas, data, etc. &#34;For there to be ghost, there must be a return to the body, but to a body that is more abstract than ever. The spectrogenic process corresponds therefore to a paradoxical <italic>incorporation</italic>. Once ideas or thoughts (<italic>Gedanken</italic>) are detached from their substratum, one engenders some ghost by giving them a body. Not by returning to the living body from which ideas and thoughts have been torn loose, but by incarnating the latter in <italic>another artifactual body, a prosthetic body</italic>&#34;.</p></fn>
<fn id="n55"><p>Kevin D. Haggarty and Richard V. Ericson, &#34;The Surveillant Assemblage,&#34; <italic>British Journal of Sociology</italic> 51, no. 4 (<xref ref-type="bibr" rid="B36">2000</xref>), 611.</p></fn>
<fn id="n56"><p>Gordon, <italic>Ghostly Matters: Haunting and the Sociological Imagination</italic>, 19.</p></fn>
<fn id="n57"><p>Michel Foucault, <italic>Discipline and Punish. The Birth of the Prison</italic>, trans. Alan Sheridan (<xref ref-type="bibr" rid="B27">London: Penguin Books, 1977</xref>), 184.</p></fn>
<fn id="n58"><p>&#34;Afterword by Michel Foucault: The Subject and Power,&#34; in <italic>Michel Foucault. Beyond Structuralism and Hermeneutics</italic>, ed. Hubert L. Dreyfus and Paul Rabinow (<xref ref-type="bibr" rid="B26">Chicago: University of Chicago Press, 1983</xref>), 212.</p></fn>
<fn id="n59"><p>Hacking, &#34;Kinds of People: Moving Targets,&#34; 367.</p></fn>
<fn id="n60"><p>Judith Butler, <italic>The Psychic Life of Power. Theories in Subjection</italic> (<xref ref-type="bibr" rid="B11">Stanford: Stanford University Press, 1997</xref>), 90.</p></fn>
<fn id="n61"><p>Hildebrandt, &#34;Defining Profiling: A New Type of Knowledge;&#34; Leese, &#34;The New Profiling: Algorithm, Black Boxes, and the Failure of Anti-Discriminatory Safeguards in the European Union.&#34;</p></fn>
<fn id="n62"><p>Apprich et al., <italic>Pattern Discrimination</italic>.</p></fn>
<fn id="n63"><p>Mireille Hildebrandt, &#34;Who Is Profiling Who? Invisible Visibility,&#34; in <italic>Reinventing Data Protection</italic>, ed. Serge Gutwirth (<xref ref-type="bibr" rid="B44">Berlin: Springer, 2009</xref>), 241.</p></fn>
<fn id="n64"><p>Hans Krause Hansen, &#34;Numerical Operations, Transparency Illusions and the Datafication of Governance,&#34; <italic>European Journal of Social Theory</italic> 18, no. 2 (<xref ref-type="bibr" rid="B37">2015</xref>): 203.</p></fn>
<fn id="n65"><p>Chun, &#34;Queering Homophily.&#34;</p></fn>
<fn id="n66"><p>Marielle Hildebrandt, &#34;Profile Transparency by Design? Re-Enabling Double Contingency,&#34; in <italic>Privacy, Due Process and the Computational Turn. The Philosophy of Law Meets the Philosophy of Technology</italic>, ed. Marielle Hildebrandt and Katja de Vries (<xref ref-type="bibr" rid="B64">Milton Park: Routledge, 2013</xref>), 221 (original emphasis).</p></fn>
<fn id="n67"><p>Louise Amoore, &#34;Data Derivatives: On the Emergence of a Security Risk Calculus for Our Time,&#34; <italic>Theory, Culture and Society</italic> 28, no. 6 (<xref ref-type="bibr" rid="B1">2011</xref>): 24&#8211;46.</p></fn>
<fn id="n68"><p>Gilles Deleuze, <italic>Negotiations. 1972&#8211;1990</italic> (<xref ref-type="bibr" rid="B17">New York: Columbia University Press, 1995</xref>), 180.</p></fn>
<fn id="n69"><p>Hildebrandt, &#34;Profile Transparency by Design? Re-Enabling Double Contingency,&#34; 227.</p></fn>
<fn id="n70"><p>Amoore, &#34;Lines of Sight: On the Visualization of Unknown Futures,&#34; 18.</p></fn>
<fn id="n71"><p>Leese, &#34;The New Profiling: Algorithm, Black Boxes, and the Failure of Anti-Discriminatory Safeguards in the European Union,&#34; 508.</p></fn>
<fn id="n72"><p>Deleuze, <italic>Negotiations. 1972</italic>&#8211;<italic>1990</italic>, 179.</p></fn>
<fn id="n73"><p>Mireille Hildebrandt, &#34;Profiling: From Data to Knowledge,&#34; <italic>DuD. Datenschutz und Sicherheit</italic> 30, no. 9 (<xref ref-type="bibr" rid="B43">2006</xref>): 550; &#34;Defining Profiling: A New Type of Knowledge,&#34; 27.</p></fn>
<fn id="n74"><p>Benjamin Nathan Waber, <italic>People Analytics: How Social Sensing Technology Will Transform Business and What It Tells Us About the Future of Work</italic> (<xref ref-type="bibr" rid="B66">Upper Saddle River, N.J.: FT Press, 2013</xref>).</p></fn>
<fn id="n75"><p>McAfee and Brynjolfsson, &#34;Big Data: The Management Revolution,&#34; 5.</p></fn>
<fn id="n76"><p>Amoore, &#34;Data Derivatives: On the Emergence of a Security Risk Calculus for Our Time,&#34; 27.</p></fn>
<fn id="n77"><p>Zuboff, <italic>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</italic>, 19.</p></fn>
<fn id="n78"><p>David Lyon, &#34;Surveillance, Snowden, and Big Data: Capacities, Consequences, Critique,&#34; <italic>Big Data &amp; Society</italic> (<xref ref-type="bibr" rid="B52">July&#8211;December 2014</xref>): 7.</p></fn>
<fn id="n79"><p>Jacques Derrida, <italic>As If I Were Dead. An Interview with Jacques Derrida/Als ob ich tot w&#228;re. Ein Interview mit Jacques Derrida</italic>, trans. Ulrike Oud&#233;e Dinkelsb&#252;hler et al. (<xref ref-type="bibr" rid="B18">Vienna: Turia + Kant, 2000</xref>), 26.</p></fn>
<fn id="n80"><p>Derrida, <italic>As If I Were Dead</italic>, 28.</p></fn>
<fn id="n81"><p>Louise Amoore, <italic>The Politics of Possibility</italic> (<xref ref-type="bibr" rid="B3">Durham: Duke University Press, 2013</xref>), 147.</p></fn>
<fn id="n82"><p>Ariane Ellerbrok, &#34;Empowerment: Analysing Technologies of Multiple Variable Visibility,&#34; <italic>Surveillance &amp; Society</italic> 8, no. 2 (<xref ref-type="bibr" rid="B22">2010</xref>).</p></fn>
<fn id="n83"><p>Bruno Latour, <italic>Science in Action</italic> (<xref ref-type="bibr" rid="B48">Milton Keynes: Open University Press, 1987</xref>), 179&#8211;257.</p></fn>
<fn id="n84"><p>Zuboff, <italic>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</italic>.</p></fn>
<fn id="n85"><p>Matthew Crain, &#34;The Limits of Transparency: Data Brokers and Commodification,&#34; <italic>Media and society</italic> 20, no. 1 (<xref ref-type="bibr" rid="B15">2018</xref>).</p></fn>
<fn id="n86"><p>Frank Pasquale, <italic>The Black Box Society. The Secret Algorithms That Control Money and Information</italic> (<xref ref-type="bibr" rid="B60">Cambridge, Massachusetts: Harvard University Press, 2015</xref>), 32.</p></fn>
<fn id="n87"><p>Hansen, &#34;Numerical Operations, Transparency Illusions and the Datafication of Governance,&#34; 206.</p></fn>
<fn id="n88"><p>Pasquale, <italic>The Black Box Society. The Secret Algorithms That Control Money and Information</italic>, 32.</p></fn>
<fn id="n89"><p>David Lyon, &#34;Surveillance as Social Sorting. Computer Codes and Mobile Bodies,&#34; in <italic>Surveillance as Social Sorting</italic>, ed. David Lyon (<xref ref-type="bibr" rid="B51">London, New York: Routledge, 2003</xref>), 20.</p></fn>
<fn id="n90"><p>Leese, &#34;The New Profiling: Algorithm, Black Boxes, and the Failure of Anti-Discriminatory Safeguards in the European Union,&#34; 504&#8211;05.</p></fn>
<fn id="n91"><p>Leese, &#34;The New Profiling: Algorithm, Black Boxes, and the Failure of Anti-Discriminatory Safeguards in the European Union,&#34; 505.</p></fn>
<fn id="n92"><p>Paul B. De Laat, &#34;Algorithmic Decision-Making Based on Machine Learning from Big Data: Can Transparency Restore Accountability,&#34; <italic>Philos. Technol.</italic> 31 (<xref ref-type="bibr" rid="B16">2018</xref>).</p></fn>
<fn id="n93"><p>Zygmunt Bauman, <italic>Modernity and the Holocaust</italic> (<xref ref-type="bibr" rid="B6">Cambridge: Polity Press, 1991</xref>).</p></fn>
<fn id="n94"><p>Bauman, <italic>Modernity and the Holocaust</italic>, 214.</p></fn>
<fn id="n95"><p>Derrida, <italic>Specters of Marx. The State of the Debt, the Work of Mourning, &amp; the New International</italic>, 7.</p></fn>
<fn id="n96"><p>Greg Elmer, <italic>Profiling Machines: Mapping the Personal Information Economy</italic> (<xref ref-type="bibr" rid="B23">Cambridge, MA: MIT Press, 2003</xref>).</p></fn>
<fn id="n97"><p>Foucault, <italic>Discipline and Punish. The Birth of the Prison</italic>.</p></fn>
<fn id="n98"><p>Mikkel Flyverbom, <italic>The Digital Prism. Transparency and Managed Visibilies in a Datafied World</italic> (<xref ref-type="bibr" rid="B24">New York: Cambridge University Press, 2019</xref>).</p></fn>
<fn id="n99"><p>Kitchin, <italic>The Data Revolution. Big Data, Data Infrastructures &amp; Their Consequences</italic>, 167.</p></fn>
<fn id="n100"><p>Ellerbrok, &#34;Empowerment: Analysing Technologies of Multiple Variable Visibility.&#34;</p></fn>
<fn id="n101"><p>Rouvroy, &#34;The End(s) of Critique,&#34; 144&#8211;45.</p></fn>
<fn id="n102"><p>Rouvroy, &#34;The End(s) of Critique,&#34; 153.</p></fn>
<fn id="n103"><p>Slavoj &#381;i&#382;ek, <italic>The Ticklish Subject: The Absent Centre of Political Ontology</italic> (<xref ref-type="bibr" rid="B68">London: Verso, 1999</xref>), 260 (original modified, &#381;i&#382;ek uses the term &#34;mode of subjectivization&#34;).</p></fn>
<fn id="n104"><p>Michel Foucault, <italic>The History of Sexuality. Volume 1</italic> (<xref ref-type="bibr" rid="B32">London: Penguin Books</xref>), 59.</p></fn>
<fn id="n105"><p>Bernard E. Harcourt, <italic>Exposed. Desire and Disobedience in the Digital Age</italic> (<xref ref-type="bibr" rid="B39">Cambridge, Mass.: Harvard University Press, 2015</xref>).</p></fn>
<fn id="n106"><p>Foucault, <italic>On the Government of the Living. Lectures at the Coll&#232;ge the France 1979</italic>&#8211;<italic>1980 and Oedipal Knowledge</italic>, 13.</p></fn>
<fn id="n107"><p>Foucault, <italic>On the Government of the Living. Lectures at the Coll&#232;ge the France 1979</italic>&#8211;<italic>1980 and Oedipal Knowledge</italic>, 13.</p></fn>
<fn id="n108"><p>Foucault, <italic>On the Government of the Living. Lectures at the Coll&#232;ge the France 1979</italic>&#8211;<italic>1980 and Oedipal Knowledge</italic>, 14.</p></fn>
<fn id="n109"><p>Zuboff, <italic>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</italic>.</p></fn>
<fn id="n110"><p>Alex Pentland, &#34;The Data-Driven Society,&#34; <italic>Scientific American</italic> 309, no. 3 (<xref ref-type="bibr" rid="B61">2013</xref>): 10.</p></fn>
<fn id="n111"><p>Michel Foucault, <italic>The Courage of Truth. The Government of Self and Others II. Lectures at the Coll&#232;ge De France 1983</italic>&#8211;<italic>1984</italic> (<xref ref-type="bibr" rid="B31">Palgrave Macmillan, 2011</xref>), 15&#8211;31. Foucault here distinguishes four basic modalities of truth-telling (<italic>veridiction</italic>).</p></fn>
<fn id="n112"><p>Zuboff, <italic>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</italic>, 433.</p></fn>
<fn id="n113"><p>Zuboff, <italic>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</italic>, 254.</p></fn>
<fn id="n114"><p>Rouvroy, &#34;The End(s) of Critique. Data-Behaviourism Versus Due-Process.&#34;</p></fn>
<fn id="n115"><p>Michel Foucault, <italic>Security, Territory, Population. Lectures as the Coll&#232;ge De France 1977</italic>&#8211;<italic>78</italic> (<xref ref-type="bibr" rid="B30">London: Palgrave Macmillan, 2007</xref>), 56&#8211;59.</p></fn>
<fn id="n116"><p>Foucault, <italic>Security, Territory, Population</italic>, 65.</p></fn>
</fn-group>
<ref-list>
<ref id="B1"><label>1</label><mixed-citation publication-type="journal"><string-name><surname>Amoore</surname>, <given-names>Louise</given-names></string-name>. <article-title>&#34;Data Derivatives: On the Emergence of a Security Risk Calculus for Our Time.&#34;</article-title> <source>Theory, Culture and Society</source> <volume>28</volume>, <issue>no. 6</issue> (<year>2011</year>): <fpage>24</fpage>&#8211;<lpage>43</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/0263276411417430</pub-id></mixed-citation></ref>
<ref id="B2"><label>2</label><mixed-citation publication-type="journal"><string-name><surname>Amoore</surname>, <given-names>Louise</given-names></string-name>. <article-title>&#34;Lines of Sight: On the Visualization of Unknown Futures.&#34;</article-title> <source>Citizenship Studies</source> <volume>13</volume>, <issue>no. 1</issue> (<year>2009</year>): <fpage>17</fpage>&#8211;<lpage>30</lpage>. DOI: <pub-id pub-id-type="doi">10.1080/13621020802586628</pub-id></mixed-citation></ref>
<ref id="B3"><label>3</label><mixed-citation publication-type="book"><string-name><surname>Amoore</surname>, <given-names>Louise</given-names></string-name>. <source>The Politics of Possibility</source>. <publisher-loc>Durham</publisher-loc>: <publisher-name>Duke University Press</publisher-name>, <year>2013</year>. DOI: <pub-id pub-id-type="doi">10.1215/9780822377269</pub-id></mixed-citation></ref>
<ref id="B4"><label>4</label><mixed-citation publication-type="book"><string-name><surname>Apprich</surname>, <given-names>Clemens</given-names></string-name>, <string-name><given-names>Wendy Hui</given-names> <surname>Kyong Chun</surname></string-name>, <string-name><given-names>Florian</given-names> <surname>Cramer</surname></string-name>, and <string-name><given-names>Hito</given-names> <surname>Steyerl</surname></string-name>. <source>Pattern Discrimination</source>. <publisher-loc>Minneapolis</publisher-loc>: <publisher-name>University of Minnesota Press</publisher-name>, <year>2019</year>.</mixed-citation></ref>
<ref id="B5"><label>5</label><mixed-citation publication-type="book"><string-name><surname>Ball</surname>, <given-names>James</given-names></string-name>. <source>Post-Truth. How Bullshit Conquered the World</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Biteback</publisher-name>, <year>2017</year>.</mixed-citation></ref>
<ref id="B6"><label>6</label><mixed-citation publication-type="book"><string-name><surname>Bauman</surname>, <given-names>Zygmunt</given-names></string-name>. <source>Modernity and the Holocaust</source>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Polity Press</publisher-name>, <year>1991</year>.</mixed-citation></ref>
<ref id="B7"><label>7</label><mixed-citation publication-type="book"><string-name><surname>Bernard</surname>, <given-names>Andreas</given-names></string-name>. <source>The Triumph of Profiling. The Self in Digital Culture</source>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Polity Press</publisher-name>, <year>2019</year>.</mixed-citation></ref>
<ref id="B8"><label>8</label><mixed-citation publication-type="book"><string-name><surname>Blackman</surname>, <given-names>Lisa</given-names></string-name>. <source>Haunted Data. Affect, Transmedia and Weird Science</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Bloomsbury</publisher-name>, <year>2016</year>.</mixed-citation></ref>
<ref id="B9"><label>9</label><mixed-citation publication-type="book"><string-name><surname>Bogard</surname>, <given-names>William</given-names></string-name>. <source>The Simulation of Surveillance. Hypercontrol in Telematic Societies</source>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>, <year>2010</year>.</mixed-citation></ref>
<ref id="B10"><label>10</label><mixed-citation publication-type="book"><string-name><surname>Bowker</surname>, <given-names>Geoffrey C</given-names></string-name>., and <string-name><given-names>Susan Leigh</given-names> <surname>Star</surname></string-name>. <source>Sorting Things Out</source>. <publisher-loc>Cambridge, Mass</publisher-loc>.: <publisher-name>MIT Press</publisher-name>, <year>2000</year>.</mixed-citation></ref>
<ref id="B11"><label>11</label><mixed-citation publication-type="book"><string-name><surname>Butler</surname>, <given-names>Judith</given-names></string-name>. <source>The Psychic Life of Power. Theories in Subjection</source>. <publisher-loc>Stanford</publisher-loc>: <publisher-name>Stanford University Press</publisher-name>, <year>1997</year>.</mixed-citation></ref>
<ref id="B12"><label>12</label><mixed-citation publication-type="webpage"><string-name><surname>Cadwalladr</surname>, <given-names>Carol</given-names></string-name>. <article-title>&#34;The Cambridge Analytica Files. &#39;I Made Steve Bannon&#39;s Psychological Warfare Tool&#39;: Meet the Data War Whistleblower.&#34;</article-title> <source>The Guardian</source>, <month>March</month> <day>18</day>, <year>2018</year>, <uri>https://www.theguardian.com/news/2018/mar/17/data-war-whistleblower-christopher-wylie-faceook-nix-bannon-trump</uri>.</mixed-citation></ref>
<ref id="B13"><label>13</label><mixed-citation publication-type="journal"><string-name><surname>Cheney-Lippold</surname>, <given-names>John</given-names></string-name>. <article-title>&#34;A New Algorithmic Identity. Soft Biopolitics and the Modulation of Control.&#34;</article-title> <source>Theory, Culture &amp; Society</source> <volume>28</volume>, <issue>no. 6</issue> (<year>2011</year>): <fpage>164</fpage>&#8211;<lpage>81</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/0263276411424420</pub-id></mixed-citation></ref>
<ref id="B14"><label>14</label><mixed-citation publication-type="book"><string-name><surname>Chun</surname>, <given-names>Wendy Hui Kyong</given-names></string-name>. <chapter-title>&#34;Queering Homophily.&#34;</chapter-title> In <source>Pattern Discrimination. In Search of Media</source>, edited by <string-name><given-names>Clemens</given-names> <surname>Apprich</surname></string-name>, <string-name><given-names>Wendy Hui Kyong</given-names> <surname>Chun</surname></string-name>, <string-name><given-names>Florian</given-names> <surname>Cramer</surname></string-name> and <string-name><given-names>Hito</given-names> <surname>Steyerl</surname></string-name>. <publisher-loc>Minneapolis</publisher-loc>: <publisher-name>University of Minnesota Press</publisher-name>, <year>2019</year>.</mixed-citation></ref>
<ref id="B15"><label>15</label><mixed-citation publication-type="journal"><string-name><surname>Crain</surname>, <given-names>Matthew</given-names></string-name>. <article-title>&#34;The Limits of Transparency: Data Brokers and Commodification.&#34;</article-title> <source>Media and society</source> <volume>20</volume>, <issue>no. 1</issue> (<year>2018</year>): <fpage>88</fpage>&#8211;<lpage>104</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/1461444816657096</pub-id></mixed-citation></ref>
<ref id="B16"><label>16</label><mixed-citation publication-type="journal"><string-name><surname>De Laat</surname>, <given-names>Paul B</given-names></string-name>. <article-title>&#34;Algorithmic Decision-Making Based on Machine Learning from Big Data: Can Transparency Restore Accountability.&#34;</article-title> <source>Philos. Technol</source> (<year>2018</year>): <fpage>525</fpage>&#8211;<lpage>541</lpage>. DOI: <pub-id pub-id-type="doi">10.1007/s13347-017-0293-z</pub-id></mixed-citation></ref>
<ref id="B17"><label>17</label><mixed-citation publication-type="book"><string-name><surname>Deleuze</surname>, <given-names>Gilles</given-names></string-name>. <source>Negotiations</source>. <fpage>1972</fpage>&#8211;<lpage>1990</lpage>. <publisher-loc>New York</publisher-loc>: <publisher-name>Columbia University Press</publisher-name>, <year>1995</year>.</mixed-citation></ref>
<ref id="B18"><label>18</label><mixed-citation publication-type="book"><string-name><surname>Derrida</surname>, <given-names>Jacques</given-names></string-name>. <source>As If I Were Dead. An Interview with Jacques Derrida/Als ob ich tot w&#228;re. Ein Interview mit Jacques Derrida</source>. Translated by <string-name><given-names>Ulrike Oud&#233;e</given-names> <surname>Dinkelsb&#252;hler</surname></string-name> et al. <publisher-loc>Vienna</publisher-loc>: <publisher-name>Turia + Kant</publisher-name>, <year>2000</year>.</mixed-citation></ref>
<ref id="B19"><label>19</label><mixed-citation publication-type="book"><string-name><surname>Derrida</surname>, <given-names>Jacques</given-names></string-name>. <chapter-title>&#34;Force of Law: The &#39;Mystical Foundation of Authority&#39;.&#34;</chapter-title> In <source>Deconstruction and the Possibility of Justice</source>, edited by <string-name><given-names>Drucilla</given-names> <surname>Cornell</surname></string-name>, <string-name><given-names>Michel</given-names> <surname>Rosenfeld</surname></string-name> and <string-name><given-names>David Gray</given-names> <surname>Carlson</surname></string-name>, <fpage>3</fpage>&#8211;<lpage>67</lpage>. <publisher-loc>London</publisher-loc>: <publisher-name>Routledge</publisher-name>, <year>1992</year>.</mixed-citation></ref>
<ref id="B20"><label>20</label><mixed-citation publication-type="book"><string-name><surname>Derrida</surname>, <given-names>Jacques</given-names></string-name>. <source>Specters of Marx. The State of the Debt, the Work of Mourning, &amp; the New International</source>. <publisher-loc>New York, London</publisher-loc>: <publisher-name>Routledge</publisher-name>, <year>1994</year>.</mixed-citation></ref>
<ref id="B21"><label>21</label><mixed-citation publication-type="book"><string-name><surname>Edwards</surname>, <given-names>Martin</given-names></string-name>, and <string-name><given-names>Kirstin</given-names> <surname>Edwards</surname></string-name>. <source>Predictive HR Analytics: Mastering the Hr Metric</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Kogan Page</publisher-name>, <year>2019</year>.</mixed-citation></ref>
<ref id="B22"><label>22</label><mixed-citation publication-type="journal"><string-name><surname>Ellerbrok</surname>, <given-names>Ariane</given-names></string-name>. <article-title>&#34;Empowerment: Analysing Technologies of Multiple Variable Visibility.&#34;</article-title> <source>Surveillance &amp; Society</source> <volume>8</volume>, <issue>no. 2</issue> (<year>2010</year>): <fpage>200</fpage>&#8211;<lpage>20</lpage>. DOI: <pub-id pub-id-type="doi">10.24908/ss.v8i2.3486</pub-id></mixed-citation></ref>
<ref id="B23"><label>23</label><mixed-citation publication-type="book"><string-name><surname>Elmer</surname>, <given-names>Greg</given-names></string-name>. <source>Profiling Machines: Mapping the Personal Information Economy</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>, <year>2003</year>. DOI: <pub-id pub-id-type="doi">10.7551/mitpress/5614.001.0001</pub-id></mixed-citation></ref>
<ref id="B24"><label>24</label><mixed-citation publication-type="book"><string-name><surname>Flyverbom</surname>, <given-names>Mikkel</given-names></string-name>. <source>The Digital Prism. Transparency and Managed Visibilies in a Datafied World</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>, <year>2019</year>. DOI: <pub-id pub-id-type="doi">10.1017/9781316442692</pub-id></mixed-citation></ref>
<ref id="B25"><label>25</label><mixed-citation publication-type="book"><string-name><surname>Flyverbom</surname>, <given-names>Mikkel</given-names></string-name>, and <string-name><given-names>Anders K.</given-names> <surname>Madsen</surname></string-name>. <chapter-title>&#34;Sorting Data Out: Unpacking Big Data Value Chains and Algorithmic Knowledge Production.&#34;</chapter-title> In <source>Die Gesellschaft der Daten. &#220;ber die digitale Transformation der sozialen Ordnung</source>, edited by <string-name><given-names>F.</given-names> <surname>S&#252;ssenguth</surname></string-name>. <publisher-loc>Bielefeld</publisher-loc>: <publisher-name>transcript</publisher-name>, <year>2015</year>.</mixed-citation></ref>
<ref id="B26"><label>26</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <chapter-title>&#34;Afterword by Michel Foucault: The Subject and Power.&#34;</chapter-title> In <source>Michel Foucault. Beyond Structuralism and Hermeneutics</source>, edited by <string-name><given-names>Hubert L.</given-names> <surname>Dreyfus</surname></string-name> and <string-name><given-names>Paul</given-names> <surname>Rabinow</surname></string-name>, <fpage>208</fpage>&#8211;<lpage>28</lpage>. <publisher-loc>Chicago</publisher-loc>: <publisher-name>University of Chicago Press</publisher-name>, <year>1983</year>.</mixed-citation></ref>
<ref id="B27"><label>27</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <source>Discipline and Punish. The Birth of the Prison</source>. Translated by <string-name><given-names>Alan</given-names> <surname>Sheridan</surname></string-name>. <publisher-loc>London</publisher-loc>: <publisher-name>Penguin Books</publisher-name>, <year>1977</year>.</mixed-citation></ref>
<ref id="B28"><label>28</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <source>On the Government of the Living. Lectures at the Coll&#232;ge the France 1979&#8211;1980 and Oedipal Knowledge</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Picador</publisher-name>, <year>2016</year>.</mixed-citation></ref>
<ref id="B29"><label>29</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <chapter-title>&#34;Questions of Method.&#34;</chapter-title> In <source>The Foucault Effect</source>, edited by <string-name><given-names>Graham</given-names> <surname>Burchell</surname></string-name>, <string-name><given-names>Colin</given-names> <surname>Gordon</surname></string-name> and <string-name><given-names>Peter</given-names> <surname>Miller</surname></string-name>, <fpage>73</fpage>&#8211;<lpage>86</lpage>. <publisher-loc>London</publisher-loc>: <publisher-name>Harvester Wheatsheaf</publisher-name>, <year>1991</year>.</mixed-citation></ref>
<ref id="B30"><label>30</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <source>Security, Territory, Population. Lectures as the Coll&#232;ge De France 1977&#8211;78</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Palgrave Macmillan</publisher-name>, <year>2007</year>.</mixed-citation></ref>
<ref id="B31"><label>31</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <source>The Courage of Truth. The Government of Self and Others II. Lectures at the Coll&#232;ge De France 1983&#8211;1984</source>. <publisher-name>Palgrave Macmillan</publisher-name>, <year>2011</year>. DOI: <pub-id pub-id-type="doi">10.1057/9780230274730</pub-id></mixed-citation></ref>
<ref id="B32"><label>32</label><mixed-citation publication-type="book"><string-name><surname>Foucault</surname>, <given-names>Michel</given-names></string-name>. <source>The History of Sexuality. Volume 1</source>. Translated by <string-name><given-names>Robert</given-names> <surname>Hurley</surname></string-name>. <publisher-loc>London</publisher-loc>: <publisher-name>Penguin Books</publisher-name>, <year>1981</year>.</mixed-citation></ref>
<ref id="B33"><label>33</label><mixed-citation publication-type="book"><string-name><surname>Gandy</surname>, <given-names>Oscar H</given-names></string-name>. <source>The Panoptic Sort: A Political Economy of Personal Information. Critical Studies in Communication and in the Cultural Industries</source>. <publisher-loc>Bolder</publisher-loc>: <publisher-name>Westview Press</publisher-name>, <year>1993</year>. DOI: <pub-id pub-id-type="doi">10.1080/15295039309366849</pub-id></mixed-citation></ref>
<ref id="B34"><label>34</label><mixed-citation publication-type="book"><string-name><surname>Gordon</surname>, <given-names>Avery F</given-names></string-name>. <source>Ghostly Matters: Haunting and the Sociological Imagination</source>. <publisher-loc>Minneapolis</publisher-loc>: <publisher-name>University of Minnesota Press</publisher-name>, <year>1997</year>.</mixed-citation></ref>
<ref id="B35"><label>35</label><mixed-citation publication-type="journal"><string-name><surname>Hacking</surname>, <given-names>Ian</given-names></string-name>. <article-title>&#34;Kinds of People: Moving Targets.&#34;</article-title> In <source>Proceedings of the British Academy, Volume 151, 2006 Lectures</source>, <fpage>285</fpage>&#8211;<lpage>318</lpage>, <year>2007</year>. DOI: <pub-id pub-id-type="doi">10.5871/bacad/9780197264249.003.0010</pub-id></mixed-citation></ref>
<ref id="B36"><label>36</label><mixed-citation publication-type="journal"><string-name><surname>Haggarty</surname>, <given-names>Kevin D</given-names></string-name>., and <string-name><given-names>Richard V.</given-names> <surname>Ericson</surname></string-name>. <article-title>&#34;The Surveillant Assemblage.&#34;</article-title> <source>British Journal of Sociology</source> <volume>51</volume>, <issue>no. 4</issue> (<year>2000</year>): <fpage>605</fpage>&#8211;<lpage>22</lpage>. DOI: <pub-id pub-id-type="doi">10.1080/00071310020015280</pub-id></mixed-citation></ref>
<ref id="B37"><label>37</label><mixed-citation publication-type="journal"><string-name><surname>Hansen</surname>, <given-names>Hans Krause</given-names></string-name>. <article-title>&#34;Numerical Operations, Transparency Illusions and the Datafication of Governance.&#34;</article-title> <source>European Journal of Social Theory</source> <volume>18</volume>, <issue>no. 2</issue> (<year>2015</year>): <fpage>203</fpage>&#8211;<lpage>20</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/1368431014555260</pub-id></mixed-citation></ref>
<ref id="B38"><label>38</label><mixed-citation publication-type="book"><string-name><surname>Harcourt</surname>, <given-names>Bernard E</given-names></string-name>. <source>Against Prediction. Profiling, Policing, and Punishing in an Actuarial Age</source>. <publisher-loc>Chicago</publisher-loc>: <publisher-name>University of Chicago Press</publisher-name>, <year>2007</year>. DOI: <pub-id pub-id-type="doi">10.7208/chicago/9780226315997.001.0001</pub-id></mixed-citation></ref>
<ref id="B39"><label>39</label><mixed-citation publication-type="book"><string-name><surname>Harcourt</surname>, <given-names>Bernard E</given-names></string-name>. <source>Exposed. Desire and Disobedience in the Digital Age</source>. <publisher-loc>Cambridge, Mass</publisher-loc>.: <publisher-name>Harvard University Press</publisher-name>, <year>2015</year>. DOI: <pub-id pub-id-type="doi">10.4159/9780674915077</pub-id></mixed-citation></ref>
<ref id="B40"><label>40</label><mixed-citation publication-type="book"><string-name><surname>Harding</surname>, <given-names>James M</given-names></string-name>. <source>Performance, Transparency and the Cultures of Surveillance</source>. <publisher-loc>Ann Arbor</publisher-loc>: <publisher-name>University of Michigan Press</publisher-name>, <year>2018</year>.</mixed-citation></ref>
<ref id="B41"><label>41</label><mixed-citation publication-type="book"><string-name><surname>Hildebrandt</surname>, <given-names>Mireille</given-names></string-name>. <chapter-title>&#34;Defining Profiling: A New Type of Knowledge.&#34; Chap. 2</chapter-title> In <source>Profing the European Citizen</source>, edited by <string-name><given-names>Marielle</given-names> <surname>Hildebrandt</surname></string-name> and <string-name><given-names>Serge</given-names> <surname>Gutwirth</surname></string-name>, <fpage>17</fpage>&#8211;<lpage>30</lpage>. <publisher-loc>Berlin</publisher-loc>: <publisher-name>Springer</publisher-name>, <year>2008</year>. DOI: <pub-id pub-id-type="doi">10.1007/978-1-4020-6914-7_2</pub-id></mixed-citation></ref>
<ref id="B42"><label>42</label><mixed-citation publication-type="book"><string-name><surname>Hildebrandt</surname>, <given-names>Marielle</given-names></string-name>. <chapter-title>&#34;Profile Transparency by Design? Re-Enabling Double Contingency.&#34;</chapter-title> In <source>Privacy, Due Process and the Computational Turn. The Philosophy of Law Meets the Philosophy of Technology</source>, edited by <string-name><given-names>Marielle</given-names> <surname>Hildebrandt</surname></string-name> and <string-name><given-names>Katja</given-names> <surname>de Vries</surname></string-name>, <fpage>221</fpage>&#8211;<lpage>46</lpage>. <publisher-loc>Milton Park</publisher-loc>: <publisher-name>Routledge</publisher-name>, <year>2013</year>. DOI: <pub-id pub-id-type="doi">10.4324/9780203427644</pub-id></mixed-citation></ref>
<ref id="B43"><label>43</label><mixed-citation publication-type="journal"><string-name><surname>Hildebrandt</surname>, <given-names>Mireille</given-names></string-name>. <article-title>&#34;Profiling: From Data to Knowledge.&#34;</article-title> <source>DuD. Datenschutz und Sicherhet</source> <volume>30</volume>, <issue>no. 9</issue> (<year>2006</year>): <fpage>548</fpage>&#8211;<lpage>52</lpage>. DOI: <pub-id pub-id-type="doi">10.1007/s11623-006-0140-3</pub-id></mixed-citation></ref>
<ref id="B44"><label>44</label><mixed-citation publication-type="book"><string-name><surname>Hildebrandt</surname>, <given-names>Mireille</given-names></string-name>. <chapter-title>&#34;Who Is Profiling Who? Invisible Visibility.&#34;</chapter-title> In <source>Reinventing Data Protection</source>, edited by <string-name><given-names>Serge</given-names> <surname>Gutwirth</surname></string-name>, <fpage>239</fpage>&#8211;<lpage>52</lpage>. <publisher-loc>Berlin</publisher-loc>: <publisher-name>Springer</publisher-name>, <year>2009</year>. DOI: <pub-id pub-id-type="doi">10.1007/978-1-4020-9498-9_14</pub-id></mixed-citation></ref>
<ref id="B45"><label>45</label><mixed-citation publication-type="book"><string-name><surname>Kitchin</surname>, <given-names>Rob</given-names></string-name>. <source>The Data Revolution. Big Data, Data Infrastructures &amp; Their Consequences</source>. <publisher-loc>Los Angeles</publisher-loc>: <publisher-name>Sage</publisher-name>, <year>2014</year>. DOI: <pub-id pub-id-type="doi">10.4135/9781473909472</pub-id></mixed-citation></ref>
<ref id="B46"><label>46</label><mixed-citation publication-type="journal"><string-name><surname>Kitchin</surname>, <given-names>Rob</given-names></string-name>. <article-title>&#34;Thinking Critically About and Researching Algorithms.&#34;</article-title> <source>Information, Communication and Society</source> <volume>20</volume>, <issue>no. 1</issue> (<year>2017</year>): <fpage>14</fpage>&#8211;<lpage>29</lpage>. DOI: <pub-id pub-id-type="doi">10.1080/1369118X.2016.1154087</pub-id></mixed-citation></ref>
<ref id="B47"><label>47</label><mixed-citation publication-type="book"><string-name><surname>Koopman</surname>, <given-names>Colin</given-names></string-name>. <source>How We Became Our Data. A Genealogy of the Informational Person</source>. <publisher-loc>Chicago</publisher-loc>: <publisher-name>The University of Chicago Press</publisher-name>, <year>2019</year>. DOI: <pub-id pub-id-type="doi">10.7208/chicago/9780226626611.001.0001</pub-id></mixed-citation></ref>
<ref id="B48"><label>48</label><mixed-citation publication-type="book"><string-name><surname>Latour</surname>, <given-names>Bruno</given-names></string-name>. <source>Science in Action</source>. <publisher-loc>Milton Keynes</publisher-loc>: <publisher-name>Open University Press</publisher-name>, <year>1987</year>.</mixed-citation></ref>
<ref id="B49"><label>49</label><mixed-citation publication-type="journal"><string-name><surname>Leese</surname>, <given-names>Mathias</given-names></string-name>. <article-title>&#34;The New Profiling: Algorithm, Black Boxes, and the Failure of Anti-Discriminatory Safeguards in the European Union.&#34;</article-title> <source>Security Dialogue</source> <volume>45</volume>, <issue>no. 5</issue> (<year>2014</year>): <fpage>49</fpage>&#8211;<lpage>511</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/0967010614544204</pub-id></mixed-citation></ref>
<ref id="B50"><label>50</label><mixed-citation publication-type="journal"><string-name><surname>Lindebaum</surname>, <given-names>Dirk</given-names></string-name>, <string-name><given-names>Mikko</given-names> <surname>Vesa</surname></string-name>, and <string-name><given-names>Frank</given-names> <surname>den Hond</surname></string-name>. <article-title>&#34;Insights from &#39;The Machine Stops&#39; to Better Understand Rational Assumptions in Algorithmic Decision Making and Its Implications for Organizations,&#34;</article-title> <source>Academy of Management Review</source>, no. <volume>1</volume> (<month>January</month> <year>2020</year>): <fpage>247</fpage>&#8211;<lpage>263</lpage>. DOI: <pub-id pub-id-type="doi">10.5465/amr.2018.0181</pub-id></mixed-citation></ref>
<ref id="B51"><label>51</label><mixed-citation publication-type="book"><string-name><surname>Lyon</surname>, <given-names>David</given-names></string-name>. <chapter-title>&#34;Surveillance as Social Sorting. Computer Codes and Mobile Bodies.&#34;</chapter-title> In <source>Surveillance as Social Sorting</source>, edited by <string-name><given-names>David</given-names> <surname>Lyon</surname></string-name>. <publisher-loc>London, New York</publisher-loc>: <publisher-name>Routledge</publisher-name>, <year>2003</year>.</mixed-citation></ref>
<ref id="B52"><label>52</label><mixed-citation publication-type="journal"><string-name><surname>Lyon</surname>, <given-names>David</given-names></string-name>. <article-title>&#34;Surveillance, Snowden, and Big Data: Capacities, Consequences, Critique.&#34;</article-title> <source>Big Data &amp; Society</source> <month>July&#8211;December</month> (<year>2014</year>): <fpage>1</fpage>&#8211;<lpage>13</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/2053951714541861</pub-id></mixed-citation></ref>
<ref id="B53"><label>53</label><mixed-citation publication-type="book"><string-name><surname>Mau</surname>, <given-names>Steffen</given-names></string-name>. <source>Das metrische Wir. &#220;ber die Quantifizierung des Sozialen</source>. <publisher-loc>Frankfurt a.M</publisher-loc>.: <publisher-name>Suhrkamp</publisher-name>, <year>2017</year>.</mixed-citation></ref>
<ref id="B54"><label>54</label><mixed-citation publication-type="book"><string-name><surname>Mayer-Sch&#246;nberger</surname>, <given-names>Viktor</given-names></string-name>, and <string-name><given-names>Kenneth</given-names> <surname>Cukier</surname></string-name>. <source>Big Data. A Revolution That Will Transform How We Live, Work and Think</source>. <publisher-loc>London</publisher-loc>: <publisher-name>John Murray</publisher-name>, <year>2013</year>.</mixed-citation></ref>
<ref id="B55"><label>55</label><mixed-citation publication-type="webpage"><string-name><surname>McAfee</surname>, <given-names>Andrew</given-names></string-name>, and <string-name><given-names>Erik</given-names> <surname>Brynjolfsson</surname></string-name>. <article-title>&#34;Big Data: The Management Revolution.&#34;</article-title> <source>Harvard Business Review</source> (<month>October</month> <year>2012</year>): <fpage>1</fpage>&#8211;<lpage>16</lpage>. <uri>https://hbr.org/2012/10/big-data-the-management-revolution</uri>.</mixed-citation></ref>
<ref id="B56"><label>56</label><mixed-citation publication-type="journal"><string-name><surname>McDonald</surname>, <given-names>Paula</given-names></string-name>, and <string-name><given-names>Paul</given-names> <surname>Thompson</surname></string-name>, <article-title>&#34;Social Media(tion) and the Reshaping fo the Public/Private Boundaries in Employment Relations.&#34;</article-title> <source>International Journal of Management Reviews</source>, <volume>18</volume> (<year>2016</year>): <fpage>69</fpage>&#8211;<lpage>84</lpage>. DOI: <pub-id pub-id-type="doi">10.1111/ijmr.12061</pub-id></mixed-citation></ref>
<ref id="B57"><label>57</label><mixed-citation publication-type="journal"><string-name><surname>Newell</surname>, <given-names>Sue</given-names></string-name>, and <string-name><given-names>Marco</given-names> <surname>Marabelli</surname></string-name>. <article-title>&#34;Strategic Opportunities (and Challenges) of Algorithmic Decision-Making: A Call for Action on the Long-Term Societal Effects of &#39;Datafication&#39;.&#34;</article-title> <source>Journal of Strategic Information Systems</source> <volume>24</volume> (<issue>2015</issue>): <fpage>3</fpage>&#8211;<lpage>14</lpage>. DOI: <pub-id pub-id-type="doi">10.1016/j.jsis.2015.02.001</pub-id></mixed-citation></ref>
<ref id="B58"><label>58</label><mixed-citation publication-type="book"><string-name><surname>Odiorne</surname>, <given-names>George S</given-names></string-name>. <source>Strategic Management of Human Resources</source>. <publisher-loc>San Francisco</publisher-loc>: <publisher-name>Jossey Bass</publisher-name>, <year>1984</year>.</mixed-citation></ref>
<ref id="B59"><label>59</label><mixed-citation publication-type="book"><string-name><surname>Orwell</surname>, <given-names>George</given-names></string-name>. <source>Nineteen Eighty Four</source>. <publisher-loc>Essex</publisher-loc>: <publisher-name>Heinemann</publisher-name>, <year>1949/1990</year>.</mixed-citation></ref>
<ref id="B60"><label>60</label><mixed-citation publication-type="book"><string-name><surname>Pasquale</surname>, <given-names>Frank</given-names></string-name>. <source>The Black Box Society. The Secret Algorithms That Control Money and Information</source>. <publisher-loc>Cambridge, Massachusetts</publisher-loc>: <publisher-name>Harvard University Press</publisher-name>, <year>2015</year>. DOI: <pub-id pub-id-type="doi">10.4159/harvard.9780674736061</pub-id></mixed-citation></ref>
<ref id="B61"><label>61</label><mixed-citation publication-type="journal"><string-name><surname>Pentland</surname>, <given-names>Alex</given-names></string-name>. <article-title>&#34;The Data-Driven Society.&#34;</article-title> <source>Scientific American</source> <volume>309</volume>, <issue>no. 3</issue> (<year>2013</year>): <fpage>78</fpage>&#8211;<lpage>83</lpage>. DOI: <pub-id pub-id-type="doi">10.1038/scientificamerican1013-78</pub-id></mixed-citation></ref>
<ref id="B62"><label>62</label><mixed-citation publication-type="journal"><string-name><surname>Pors</surname>, <given-names>Justine Gr&#248;nb&#230;k</given-names></string-name>, <string-name><given-names>Lena</given-names> <surname>Olaison</surname></string-name>, and <string-name><given-names>Birke</given-names> <surname>Otto</surname></string-name>. <article-title>&#34;Ghostly Matters in Organizing.&#34;</article-title> <source>Ephemera. Critical Dialogs on Organization</source> <volume>19</volume>, <issue>no. 1</issue> (<year>2019</year>): <fpage>1</fpage>&#8211;<lpage>29</lpage>.</mixed-citation></ref>
<ref id="B63"><label>63</label><mixed-citation publication-type="journal"><string-name><surname>Rieder</surname>, <given-names>Bernhard</given-names></string-name>. <article-title>&#34;Scrutinizing an Algorithmic Technique: The Bayes Classifier as Interested Reading of Reality, Information.&#34;</article-title> <source>Communication and Society</source> <volume>20</volume>, <issue>no. 1</issue> (<year>2017</year>): <fpage>100</fpage>&#8211;<lpage>17</lpage>. DOI: <pub-id pub-id-type="doi">10.1080/1369118X.2016.1181195</pub-id></mixed-citation></ref>
<ref id="B64"><label>64</label><mixed-citation publication-type="book"><string-name><surname>Rouvroy</surname>, <given-names>Antoinette</given-names></string-name>. <chapter-title>&#34;The End(s) of Critique. Data-Behaviourism Versus Due-Process.&#34;</chapter-title> In <source>Privacy, Due Process and the Computational Turn. Philosophers of Law Meet Philosophers of Technology</source>, edited by <string-name><given-names>Marielle</given-names> <surname>Hildebrandt</surname></string-name> and <string-name><given-names>Katja</given-names> <surname>de Vries</surname></string-name>, <fpage>143</fpage>&#8211;<lpage>68</lpage>. <publisher-loc>London</publisher-loc>: <publisher-name>Routledge</publisher-name>, <year>2013</year>.</mixed-citation></ref>
<ref id="B65"><label>65</label><mixed-citation publication-type="book"><string-name><surname>Sprinker</surname>, <given-names>Michael</given-names></string-name>, ed. <source>Ghostly Demarcations. A Symposium on Jacques Derrida&#39;s Specters of Marx</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Verso</publisher-name>, <year>1999</year>.</mixed-citation></ref>
<ref id="B66"><label>66</label><mixed-citation publication-type="book"><string-name><surname>Waber</surname>, <given-names>Benjamin Nathan</given-names></string-name>. <source>People Analytics: How Social Sensing Technology Will Transform Business and What It Tells Us About the Future of Work</source>. <publisher-loc>Upper Saddle River, N.J.</publisher-loc>: <publisher-name>FT Press</publisher-name>, <year>2013</year>.</mixed-citation></ref>
<ref id="B67"><label>67</label><mixed-citation publication-type="journal"><string-name><surname>Youyou</surname>, <given-names>Wu</given-names></string-name>, <string-name><given-names>Michal</given-names> <surname>Kosinski</surname></string-name>, and <string-name><given-names>David</given-names> <surname>Stillwell</surname></string-name>. <article-title>&#34;Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans.&#34;</article-title> <source>PNAS</source> <volume>112</volume>, <issue>no. 4</issue> (<year>2015</year>): <fpage>1036</fpage>&#8211;<lpage>40</lpage>. DOI: <pub-id pub-id-type="doi">10.1073/pnas.1418680112</pub-id></mixed-citation></ref>
<ref id="B68"><label>68</label><mixed-citation publication-type="book"><string-name><surname>&#381;i&#382;ek</surname>, <given-names>Slavoj</given-names></string-name>. <source>The Ticklish Subject: The Absent Centre of Political Ontology</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Verso</publisher-name>, <year>1999</year>.</mixed-citation></ref>
<ref id="B69"><label>69</label><mixed-citation publication-type="book"><string-name><surname>Zuboff</surname>, <given-names>Shoshana</given-names></string-name>. <source>The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Public Affairs</publisher-name>, <year>2019</year>.</mixed-citation></ref>
</ref-list>
</back>
</article>