Q&A: A call for public debate and thoughtful regulation of AI systems
Joanna Redden says there needs to be more public debate and transparency around AI use
New research out of Western University is shining a light on the federal government's use of artificial intelligence.
From a database of over 300 automated tools, 95 per cent have been used by the feds for various functions, such as taxes, visa applications and diversity hiring, according to the research. It comes at a time when the Liberal government is proposing an Artificial Intelligence and Data Act, specifically aimed at AI.
Joanna Redden is an associate professor of information and media studies at Western University and co-director of Starling: Just Technologies. Just Societies. and Data Justice Lab. She spoke with CBC London Morning host Andrew Brown about compiling the data and what considerations need to be made as AI becomes more prevalent in our society.
The following has been edited for length and clarity.
Andrew Brown: So why did you want to take a look at the use of AI?
Joanna Redden: Well, I think that one of the reasons was to add some specificity. I think there's a lot of hype about AI quite generally, and one of the things that we were concerned about is how little we know, in fact, in a systematic way, about the different ways that AI is being used right now, both by government and by business.
AB: So then, what do we know now about how it is being used by the government?
JR: Well, I think one of the things that our research shows is how widespread uses of AI are. I think one of the other things that our research does is hopefully identify the need for there to be more information. So we think that there should be registries not just of government uses of AI, but also of business uses of AI.
AB: How widespread is it?
JR: Well, we identified 303 applications and that was mostly at the federal government level. I don't know how many applications we would find if we were also able or had the resources to look into provincial uses or uses across cities across Canada. We don't know how widespread uses are by businesses. And so I think that it's really important that we get a better handle on just how AI is being used right now across the board, particularly if we're regulating it or planning to regulate it. And of course, one of the problems with current approaches to regulation, you know, the first piece of legislation that's really focusing on AI regulation in Canada, the Artificial Intelligence and Data Act, is not going to apply to government uses.
AB: Why is that concerning to you?
JR: I think it's concerning because we think that we need far more protections than we have. We need to be having more widespread discussions about how AI is being used, but also about the impacts of AI on decision makers and people who are subjected to these systems, people who are being risk scored or assessed in different ways. These technologies are new. Their uses are new and we're still coming to terms in so many ways with the implications of their use.
AB: So I know that you looked at how widespread it is in this research. Did you look as well at the effect of it and how it's working?
JR: So in addition to this project, which involves mapping where and how government is using AI, I've also been a part of research teams that are looking at how people are being negatively affected by different systems as they're implemented around the world and what some of these negative implications are. There's quite a lot of research that's documented the harms that can occur as a result of AI practices. I think that research demonstrates how important it is that we get it right in terms of regulation. So just some brief examples, in the data on record, which is Open Access, you can find examples of people being wrongly flagged as having committed fraud, for example, which has led in some cases to bankruptcy, to people losing their homes. We've seen people who have been falsely identified through the use of facial recognition, which has led to wrongful arrest and detention. We've seen uses of risk assessment which has meant that people have been overcharged for certain things like insurance. The examples go on and on.
AB: Are we ready for all of this?
JR: I don't think we are, and this is one of the points that I hope that we're making with the register. I think that what the register shows is that we need to be having far more debate collectively about the kind of worlds we want to create through the uses of these technologies. And we need that debate to be informed by a contextually aware and specific understanding of what kinds of systems are already in use and what kinds of harms are already occurring.
AB: How urgent would you say this is?
JR: I would say it is incredibly urgent. I think that we're seeing examples in the news almost on a daily basis of how people are being negatively affected by different kinds of AI systems, including right here in London. If we look at the kinds of applications that were discussed I believe on your program just last week, with the images of the high school students. So I think that what we're seeing is that there's an urgent need that we do this carefully and intentionally and that we take the time to get it right.
