ASQ-MISQ on Technology and Institutions

/

TECHNOLOGIES CHANGE, THE CHARGE REMAINS THE SAME

Curation Team:
Stephen R. Barley (University of California, Santa Barbara)
Wanda J. Orlikowski (Massachusetts Institute of Technology)

Download the ASQ-MISQ Research Curation on Technology and Institutions

Please cite this curation as follows: Barley, S. R., & Orlikowski, W. J. (2023). Technologies change, the charge remains the same. Administrative Science Quarterly & MIS Quarterly Research Curation, https://www.misqresearchcurations.org/asq-misq-curations.

In 2001 we published a paper in MIS Quarterly (MISQ) in which we argued that students of organizations and students of information systems (IS) could profit by learning from each other as they attempted to understand how technologies are changing the worlds we live in (Orlikowski & Barley, 2001). We argued that the organization studies literature tended to treat technologies too abstractly and deterministically, while the IS literature paid too little attention to institutional influences and dynamics. Consequently, both fields had difficulty dealing with how specific technologies were changing work, organizations, and society at large. We called for interaction across the boundaries of the two fields to address these shortfalls and stimulate new kinds of research that considered both technologies and institutions in empirical studies.

In 2021 Christine Beckman and Andrew Burton-Jones, the editors of the Administrative Science Quarterly (ASQ) and MISQ, respectively, asked us to curate a virtual collection of papers that had been published in their journals since 2001 and that addressed the themes we discussed in our paper. We accepted the invitation because the four of us believe that the need to grapple with the organizational and societal implications of technological change is even more important today than it was then. Society is now awash with computational capabilities generating unanticipated and profound changes in our lives. Algorithms now shape who gets elected, who gets a loan, who gets hired, who gets arrested, and even whom we date. Cars drive themselves, robots rescue people from burning buildings, drones monitor farmers’ fields, and watches do much more than tell time.

To curate this collection, we began by identifying all empirical papers published in ASQ and MISQ from 2002 through 2021 using a set of keywords designed to identify papers that dealt with technology and institutional or organizational change. Given that papers published in ASQ primarily deal with organizations, occupations, and institutions and that papers published in MISQ primarily deal with technologies and information systems, we used different keywords to select papers from the two journals¹ . With the help of members of the journals’ editorial boards, we then read through the abstracts of the 278 papers that our keywords flagged to determine which papers merited closer attention. Our reviews of the abstracts yielded an initial feasible set of 9 papers from ASQ and 26 papers from MISQ. We then read the 35 papers in their entirety and chose five papers from each journal for inclusion in the collection. Two criteria guided our final section. The first was that a paper needed to focus on how a specific technology (or a clear class of technologies) contributed directly or indirectly to substantial changes in work, organizations, occupations, or institutions. Second, we specifically sought papers that offered novel contributions to our understanding of technological and institutional dynamics.

We organized the papers into five groupings. The first two papers in the collection provide insight into the complex interplay between human agency and materiality. Leonardi (2011) draws on ideas of structuration, affordances, constraints, and flexible routines to develop a perspective that clarifies how technologies and human action shape each other over time to alter the institutions and practices of an organization and its occupational communities. Essén and Värlander (2019) also examine routines and affordances but move beyond a single organization to show how technological changes in multiple, distributed settings can accumulate gradually over a period of years to alter an entire institutional field.

The next three papers focus on the challenges that artificial intelligence and robotics pose for professionals and their knowledge. Lebovitz, Levina, and Lifshitz-Assaf (2021) examine the adoption of machine learning–based AI tools in radiology. Their analysis underscores how such tools can undermine the responsible exercise of professional judgment by failing to take into account how experts make and verify decisions in everyday practice. Van den Broek, Sergeeva, and Huysman (2021) document how the developers of an AI-based hiring tool that was intended to get “domain experts ‘out the loop’” was unable do so. Instead, after multiple cycles of development, the upshot was a hybrid practice in which the algorithm augmented rather than automated hiring, which, in turn, triggered a change in the organization’s institutionalized status quo. In the third paper, Beane (2019) shows how longstanding practices for training surgeons were challenged by robots that did not allow surgeons and their trainees to collaborate during surgery. He finds that, as a result, some newly minted surgeons graduated with limited competency in robotic surgery while those who eventually developed competency did so by resorting to institutionally unsanctioned ways of learning. Taken together, these three papers show that the integration of artificial intelligence and robotics in work and organizations is more complex and nuanced than public discourse admits.

The third set of papers addresses the rise of platforms as market intermediaries. Rahman (2021) describes how freelancers using an online labor platform experienced an invisible form of control over their work that made it difficult, if not impossible, to understand how their performance was evaluated and presented to clients. Curchod, Patriotta, Cohen, and Neysen (2020) explore power asymmetries that made it difficult for sellers and buyers on eBay to interact and negotiate with each other. In both papers, we see platforms shifting institutionalized power relationships by aligning with one party to the exchange (clients, buyers) to the detriment of the other (freelancers, sellers).

The next pair of papers investigates online communities as a locus of virtual self-organizing. Massa and O’Mahony (2021) explore how networked activists used online tools to organize protests without controlling participation. They also find, ironically, that these same tools enabled central members of the community to constrain the involvement of newcomers until they were adequately socialized into the ways of their community. Vaast and Pinsonneault (2021) document how data scientists defined and redefined their occupational identity in an online community over time as labor markets and technologies changed. These two papers highlight how social collectives can be mobilized online outside institutionalized paths for collective action and with little face-to-face involvement.

Finally, Bechky (2020) offers an account of how a new technology unexpectedly affected the institutional environment of occupations that did not use the technology. In 2009, the National Academy of Sciences published a report that argued all forensic sciences should seek to emulate the rigor of DNA analysis. In response to the report, other forensic occupational groups felt compelled to defend or change their technologies and practices, depending on how they were embedded in their institutional fields.

As we assembled the papers, we were struck by how important our original call for interaction across the two fields remains and how crucial it is to link specific changes in technical capabilities to changes in forms of organizing, the nature of work, and the larger institutions that shape our daily lives. Indeed, we hope scholars in both fields double down on what we were advocating and focus even more intently on the technological phenomena that we could not have envisioned more than 20 years ago. When we drafted our paper in 1999, Google was but a year old. YouTube, Facebook, and Twitter were years from founding. At the time, there were 3.17 million websites in the world; today there are almost 2 billion. There were no social media. Blogs had just emerged. Online commerce was still in its infancy, and Amazon only sold books. Gigs were something that musicians played, and platforms were what they performed on. Back then, the technologies that shaped organizations and institutions consisted of hardware that people could see and software that developers could understand. Moreover, these technologies were largely commissioned, developed, and implemented within single organizations.

We were pleased to find that most of the papers included in this collection deal with technological phenomena that were unimaginable at the time we wrote. Machine learning, online labor platforms, robotic surgery, and online activism have transformed ways of working and organizing more deeply than could have been anticipated. For example, who would have believed that online activities would have such far-reaching real-world effects? Who would have believed that from coffee shops, people would transact globally for labor and goods? Who would have believed that radiologists would use algorithms to identify pathology or that surgeons would operate without their hands touching a patient’s body? As the articles in this collection powerfully underscore, these phenomena are profoundly affecting institutions in ways that change the core values, practices, identities, and jurisdictions of occupations, organizations, and society at large.

The papers in this collection remind us that it is imperative to seek close encounters with what people do every day if we are to understand how technologies alter institutions of work. Immersion in social life remains indispensable because it is how we discover what technologies are doing to us and our society. For example, without fieldwork how would Beane (2019) have learned that young surgeons were being trained differently and potentially inadequately, and what they were doing offstage to compensate? Without participant observation how would Lebovitz, Levina, and Lifshitz-Assaf (2021) have come to understand why some AI tools complement radiologists’ practices and others do not? Without engaging in the field for almost two years how would van den Broek, Sergeeva, and Huysman (2021) have identified the mutual learning process that enabled developers and clients to co-produce a workable AI hiring tool?

Yet, at the same time, undertaking deep immersion and ethnographic work to understand a technology and its implications is becoming more difficult. In a world of platforms and distributed work, “being there” becomes infeasible because there is no place that can be called “there.” Online phenomena overflow specific sites, and their operations are obscured not only because the technologies are closely guarded but, more importantly, because the algorithms constituted through machine learning are opaque and often incomprehensible even to their developers. Further, as the content on media platforms is typically anonymous, it is difficult to identify, much less find and interview, people who participate in online communities.

Faced with these problems, those who study technology in the field must begin to develop new methods and tools consistent with the nature of evolving technical capabilities. The articles in this collection provide insight into the kinds of approaches that might be useful going forward. For example, Rahman (2021) found he needed to become both a worker and an employer on the labor platform he studied before he could gain access to clients and contractors. Massa and O’Mahony (2021) resorted to covertly observing websites and chatrooms to examine how online activists self-organize through open, digital networks, while Vaast and Pinsonneault (2021) lurked in online forums to chart data scientists’ changing conceptions of their field and their identity.

All of us have the sense that we stand on the cusp of futures far different from the pasts we have known. To navigate those futures more effectively, we believe that we need more in-depth empirical studies and less speculation and hype. We hope the papers in this collection stimulate further well-designed studies of how technologies are changing work, occupations, and organizations—for the better and for the worse. We need such scholarship now more than ever!

Endnote:

[1] For ASQ we used the keywords technolog (-y -ies -ical), platform (-s), algorithm (-s), computer (-s -ization -ize), information system (-s), digital (-ization), robot (-s -ization), digitization, virtual, material (-ity -ize), sociomaterial (-ity), information technolog (-y -ies). For MISQ the keywords were institution (-al -alize), organization (-al) theory (-ies), bureaucrac (-y -ies), heterarch (-y -ies), networked organization (-s), organizing, work, practice (-s), organization (-al) change, transformation (-s), sociomaterial (-ity), cultur (-e -al), hierarch (-y -ies), communit (-y -ies), profession (-s -al), occupation (-s -al), collaborat (-e -ation).

Administrative Science Quarterly & MIS Quarterly Research Curation

1. Human Agency and Materiality

Leonardi, P. M. (2011). When flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies, MIS Quarterly, 35(1), 147-167. Also available at https://aisel.aisnet.org/misq/vol35/iss1/10/. Interview: The Inside Story of Leonardi (MISQ, 2011)

Essén, A., & Värlander, S. W. (2019). How technology-afforded practices at the micro-level can generate change at the field level: Theorizing the recursive mechanism actualized in swedish rheumatology 2000–2014, MIS Quarterly, 43(4), 1155-1176. Also available at https://aisel.aisnet.org/misq/vol43/iss4/9/. Explainer video: “Values, norms, and technology”

2. Artificial Intelligence and Robotics and Professional Expertise

Lebovitz, S., Levina, N., & Lifshitz-Assaf, H. (2021). Is AI ground truth really ‘true’? The dangers of training and evaluating AI tools based on experts’ know-what, MIS Quarterly, 45(3), 1501-1525. Also available at https://aisel.aisnet.org/misq/vol45/iss3/19/. Video: “Is AI Ground Truth Really True?”

van den Broek, E., Sergeeva, A., & Huysman, M. (2021). When the machine meets the expert: An ethnography of developing AI for hiring, MIS Quarterly, 45(3), 1557-1580. Also available at https://aisel.aisnet.org/misq/vol45/iss3/21/ . Lecture: “When the Machine Meets the Expert: An Ethnography of Developing AI for Hiring”

Beane, M. (2019). Shadow learning: Building robotic surgical skill when approved means fail, Administrative Science Quarterly, 64(1), 87-123. TED talk: “How do we learn to work with intelligent machines?”

3. Platforms as Market Intermediaries

Rahman, H. A. (2021). The invisible cage: Workers’ reactivity to opaque algorithmic evaluations, Administrative Science Quarterly, 66(4), 945-988. Video: “The Invisible Cage”

Curchod C., Patriotta G., Cohen L., & Neysen N. (2020). Working for an algorithm: Power asymmetries and agency in online work settings, Administrative Science Quarterly, 65(3), 644-676. Interview: ASQ Blog

4. Online Communities

Massa, F. G., & O’Mahony, S. (2021). Order from chaos: How networked activists self-organize by creating a participation architecture, Administrative Science Quarterly, 66(4), 1037-1083. Video: Order from Chaos

Vaast, E., & Pinsonneault, A. (2021). When digital technologies enable and threaten occupational identity: The delicate balancing act of data scientists, MIS Quarterly, 45(3), 1087-1112. Also available at https://aisel.aisnet.org/misq/vol45/iss3/6/. Video: “Occupational Identity”

5. Technology and Professional Jurisdictions

Bechky, B. A. (2020). Evaluative spillovers from technological change: The effects of “DNA envy” on occupational practices in forensic science, Administrative Science Quarterly, 65(3), 606-643. Interview: ASQ Blog

Please cite this curation as follows: Barley, S. R., & Orlikowski, W. J. (2023). Technologies change, the charge remains the same. Administrative Science Quarterly & MIS Quarterly Research Curation, https://www.misqresearchcurations.org/asq-misq-curations.