Welcome back to Mixtape, the TechCrunch podcast that looks at the human element that powers technology.
For this episode we spoke with Meredith Whittaker, co-founder of the AI Now Institute and Minderoo Research Professor at NYU; Mara Mills, associate professor of Media, Culture and Communication at NYU and co-director of the NYU Center for Disability Studies; and Sara Hendren, professor at Olin College of Engineering and author of the recently published What Can a Body Do: How We Meet the Built World.
It was a wide-ranging discussion about artificial intelligence and disability. Hendren kicked us off by exploring the distinction between the medical and social models of disability:
So in a medical model of disability, as articulated in disability studies, the idea is just that disability is a kind of condition or an impairment or something that’s going on with your body that takes it out of the normative average state of the body says something in your sensory makeup or mobility or whatever is impaired, and therefore, the disability kind of lives on the body itself. But in a social model of disability, it’s just an invitation to widen the aperture a little bit and include, not just the body itself and what it what it does or doesn’t do biologically. But also the interaction between that body and the normative shapes of the world.
When it comes to technology, Mills says, some companies work squarely in the realm of the medical model with the goal being a total cure rather than just accommodation, while other companies or technologies – and even inventors – will work more in the social model with the goal of transforming the world and create an accommodation. But despite this, she says, they still tend to have “fundamentally normative or mainstream ideas of function and participation rather than disability forward ideas.”
“The question with AI, and also just with old mechanical things like Brailers I would say, would be are we aiming to perceive the world in different ways, in blind ways, in minoritarian ways? Or is the goal of the technology, even if it’s about making a social, infrastructural change still about something standard or normative or seemingly typical? And that’s — there are very few technologies, probably for financial reasons, that are really going for the disability forward design.”
As Whittaker notes, AI by its nature is fundamentally normative.
“It draws conclusions from large sets of data, and that’s the world it sees, right? And it looks at what’s most average in this data and what’s an outlier. So it’s something that is consistently replicating these norms, right? If it’s trained on the data, and then it gets an impression from the world that doesn’t match the data it’s already seen, that impression is going to be an outlier. It won’t recognize that it won’t know how to treat that. Right. And there are a lot of complexities here. But I think, I think that’s something we have to keep in mind as sort of a nucleus of this technology, when we talk about its potential applications in and out of these sorts of capitalist incentives, like what is it capable of doing? What does it do? What does it act like? And can we think about it, you know, ever possibly in company encompassing the multifarious, you know, huge amounts of ways that disability manifests or doesn’t manifest.”
We talked about this and much much more on the latest episode of Mixtape, so you click play above and dig right in. And then subscribe wherever you listen to podcasts.
