Introduction
This was the introduction to the first issue of crowdwork-ethics, distributed by email to 12 subscribers, mostly researchers, on 14 Aug 2013.
This is the first issue of crowdwork-ethics, an informal and occasional newsletter for researchers interested in ethical issues in crowd work. Ethical issues in crowd work have not been discussed much in published research on crowd work in the human computation or human-centered computing (i.e., HCI and CSCW) literatures. Exceptions include Silberman, Irani, Ross, and Tomlinson (2010a, 2010b); Bederson and Quinn (2011); Kittur, Nickerson, Bernstein, Gerber, Shaw, Zimmerman, Lease, and Horton (2013); and Irani and Silberman (2013). If you know of more [that are not listed on the readings page], please forward them.
In our 2010 paper to the Human Computation Workshop (HCOMP), Lilly Irani, Joel Ross, Bill Tomlinson, and I summarized challenges faced by workers in Amazon’s Mechanical Turk platform (AMT) as follows:
(1) uncertainty about payment; (2) unaccountable and seemingly arbitrary rejections [i.e., non-payment]; (3) fraudulent tasks; (4) prohibitive time limits; (5) long pay delays; (6) uncommunicative requesters and administrators; (7) cost of requester and administrator errors borne by workers; and (8) low pay.
We also listed some broader open research and design questions, including:
How does database, interface, and interaction design influence individual outcomes and market equilibria? What are the economics of fraudulent tasks (scamming and spamming)? What decision logics are used by buyers and sellers in human computation markets? What’s fair in paid crowdsourcing?
We noted further that “gaps remain in our demographic understanding of AMT” and that “as new platforms and tools come online and mature, comparative studies will become possible, and longitudinal studies more feasible.”
Despite the papers listed above (and growing interest in the topic), these questions remain largely unaddressed. Our demographic and strategic understanding of workers and employers on AMT may be worse now than it was in 2010: demographics have very likely shifted but no new demographic studies have been published in human computation or human-centered computing — and workers have raised methodological issues with the earlier studies. New platforms have come online but are little-studied. I know of no comparative or longitudinal studies of crowd work platforms published in HCI, CSCW, or human computation. (If you do, please forward them.) And the question of fairness, in my view central to discussion of the ethics of crowd work, has been raised but not addressed directly — either empirically or theoretically.
Ethical issues in crowd work have been discussed more vigorously in the popular press than in computing research (e.g., Zittrain 2009; Cushing 2012; Brode 2013; Folbre 2013; Leonard 2013). The ethical discussion in the popular press can be summarized roughly as revolving around the question of whether AMT (few other platforms are discussed) is a “digital sweatshop.” Sweatshops are assumed to be bad, so if AMT is a sweatshop it must also be bad. But this portrayal, and the conclusions drawn by many commentators in favor of reform or regulation, are contested regularly by workers. The tension between persistent unaddressed worker complaints and persistent worker opposition to calls for reform has, to my knowledge, yet to be discussed in detail in any scholarly or popular analysis. My working hypothesis is that the worker opposition to calls for reform stems partly (perhaps mainly) from a fear that reform — and certainly regulation — will reduce the amount of work available through crowd work platforms, and the pay available for that work. Interested readers are encouraged to visit the subforum “mTurk & Amazon in the the Media” on the prominent AMT worker forum Turker Nation. Consider for example a response to a post by the economist Nancy Folbre (2013), who argued in the New York Times blog Economix that
a sustainable form of crowdsourcing will require forms of collective governance that mitigate the effects of market competition on those treated as mere links in a chain of algorithmic logic. In other words, it will require some assurance of human rights, including access to decent employment, living wages and high-quality public education.
In her post linking to Folbre’s piece, spamgirl, administrator of Turker Nation, wrote:
Aren’t any journalists focusing on something important? Why are they trying to take our work from us when jobs are so hard to find? If laws pass regulating hourly wages on mTurk, requesters will flee for the hills and we’ll be FUCKED. Journalists, LEAVE US ALONE! We don’t want your help.
Another worker on the same forum linked to a pre-publication article about “The future of crowd work” by Kittur et al. (2013) in which the first author was apparently interviewed. “Ultimately,” journalist Brode wrote, “[Kittur] said a national policy governing crowd work may be necessary to ensure that the country’s millions of crowd workers aren’t mistreated” (2013). Turker Nation user major40 responded:
I don’t like it. Another idiot professor who thinks he knows what’s best for the private market. This will only mean the government getting involved and regulating the requesters which in turn will end up in less pay for us. Someone please tell this idiot professor to stay in the classroom.
Yet when asked in a written interview, “In ten years, what do you think you’ll be doing in relation to Mechanical Turk?”, spamgirl (2010) wrote:
Praise God, may I not even remember the place by then! If I’m still kickin’ in 10 years, I’ll have won the lottery and moved on[.] I love mTurk, and I love the work I do, and I appreciate everything they have done for me, but I have to work so hard to make enough just to EAT… I can’t wait until the debt is paid off, hubby gets a job, and I can spend some time with my family. I’m a little burnt out.
In her 1962 book Economic Philosophy, the Cambridge economist Joan Robinson noted wryly that in some cases “the misery of being exploited by capitalists is nothing compared to the misery of not being exploited at all” (p. 46). This seems to characterize well the present situation of many workers in AMT. Current crowd work arrangements are vastly preferable to nothing, especially for workers who cannot work outside the home or cannot find other work. Yet current crowd work arrangements fall far short of what can be imagined — technologically, institutionally, economically, and ethically.
In my view, crowd work researchers now face three tasks, none trivial. First, we can develop a clear understanding of the present situation. What is the current situation in crowd work and what, if anything, is wrong with it? Though this question appears simple, we are not yet really able to answer it with rigor. The word “exploitation” is bandied about, and concerned and well-meaning noises about minimum or living wages are made. Yet we have no operational definition of exploitation and no understanding of implications for action if we were to identify it in present work arrangements. Establishment of wage floors in a global market would present unprecedented challenges — and some workers, even in countries with relatively high costs of living, are against it in the first place. Even the critique of AMT by analogy to sweatshops founders in the face of principled defenses of actual sweatshops by economists and journalists of a wide range of political inclinations (e.g., Krugman 1997, 2001; Kristof 2004, 2009; Powell and Skarbek 2006). Put bluntly, the easy critiques built mainly on the sense of unease many of us feel upon first encountering current crowd work arrangements do not hold up. All is not well, yet we are at a loss to explain exactly what is wrong. Empirical work and conceptual development — that is, theory — are needed to build a coherent understanding of present conditions and to communicate that understanding to other researchers, system administrators, managers, regulators, workers, employers, and other stakeholders.
Second, we can use this knowledge to develop models for possible alternative crowd work institutions and infrastructures. Such models can integrate technical knowledge, stakeholder concepts revealed by empirical research, and theory from other fields such as sociology, law, and political economy — and can offer significant contributions back to technical researchers, stakeholders, and researchers and practitioners in other fields. It is my strong belief that to “work”, such models will need more sophisticated views of human action and motivation than the “rational actor” models inherited from neoclassical economics that have undergirded much research and practice in human computation thus far. We will need, I suspect, not only to understand and represent the importance of intersubjective, largely qualitative phenomena like trust, fairness, justice, and power, but also to design technologies and institutions with them in mind.
Third, we can build and maintain actual working systems, and the institutions to support them, that embody this knowledge.
References
Bederson, B. B. and A. J. Quinn. 2011. Web workers unite! Addressing challenges of online laborers. CHI ’11 EA: 97-106.
Brode, N. 2013. CMU professor: online ‘crowd workers’ need protections, organization. 90.5 WESA: Pittsburgh’s NPR News Station, 17 Feb 2013.
Cushing, E. 2012. Dawn of the digital sweatshop. East Bay Express, 1 Aug 2012.
Folbre, N. 2013. The unregulated work of Mechanical Turk. New York Times: Economix, 18 Mar 2013.
Irani, L. C. and M. S. Silberman. 2013. Turkopticon: interrupting worker invisibility in Amazon Mechanical Turk. CHI ’13: 611-620.
Kittur, A., J. V. Nickerson, M. S. Bernstein, E. M. Gerber, A. Shaw, J. Zimmerman, M. Lease, J. J. Horton. 2013. The future of crowd work. CSCW ’13: 1301-1318.
Kristof, N. 2004. Inviting all Democrats. New York Times, 14 Jan 2004.
Kristof, N. 2009. Where sweatshops are a dream. New York Times, 14 Jan 2009.
Krugman, P. 1997. In praise of cheap labor. Slate, 21 Mar 1997.
Krugman, P. 2001. Reckonings; hearts and heads. New York Times, 22 Apr 2001.
Leonard, A. 2013. The internet’s destroying workâand turning the old middle class into the new proletariat. Salon.com, 12 Jul 2013.
Powell, B. and D. Skarbek. 2006. Sweatshops and Third World living standards: are the jobs worth the sweat? Journal of Labor Research 27(2): 263-274.
Robinson, J. 1962. Economic Philosophy. Penguin.
Silberman, M. S., L. C. Irani, and J. W. Ross. 2010. Ethics and tactics of professional crowdwork. XRDS 17(2): 39-43.
Silberman, M. S., L. C. Irani, J. W. Ross, and B. Tomlinson. 2010. Sellers’ problems in human computation markets. HCOMP ’10: 18-21.
spamgirl. 2010. Spamgirl speaks! An interview (part 1 of 3). Broken Turk, Nov 2010.
Zittrain, J. 2009. Work the new digital sweatshops. Newsweek, 7 Dec 2009.