[MUSIC FADES IN]
Shumita Basu, Narrating: This is “In Conversation,” from Apple News. I'm Shumita Basu. Today, how facial recognition technology is upending privacy.
[MUSIC FADES OUT]
[SYNTHESIZED MUSIC]
Basu, Narrating: Last December Kelly Conlon was preparing a special outing for her daughter.
Kashmir Hill: She was taking her daughter, who is in the Girl Scouts, with the troop to see the Rockettes at Radio City Music Hall. It was a holiday show.
Basu, Narrating: This is Kashmir Hill, a tech reporter for “The New York Times.”
Hill: And as they try to enter Radio City Music Hall, Kelly Conlon gets pulled aside and asked for her ID. And she's informed that she's not allowed in the venue.
Basu, Narrating: Kelly had been put on a temporary blacklist, because she worked at a law firm that was suing the owner of the venue, who also owned a bunch of other big event spaces like Madison Square Garden. Kelly wasn't even working on that lawsuit. But when she entered the building, she was flagged by the venue's facial recognition system.
Hill: They had decided to use facial recognition technology to punish their enemies. And they weren't letting thousands of lawyers into their various venues until they dropped their lawsuits or resolved the litigation.
Basu, Narrating: The parent company that owns the venue called its facial recognition policy "straightforward," just one of its tools to provide a safe environment.
Hill: This idea of using it to really just ban your enemies was a pioneering use of the technology. And I could just imagine all the different ways it could go if more businesses adopted that practice. It could really usher in a new era of discrimination. This ability to judge us, track us, monitor us, ban us.
Basu, Narrating: Kashmir has been reporting on the ways facial recognition technology is being used in the real world, particularly by U.S. law enforcement agencies. In her new book, “Your Face Belongs to Us,” she looks at how one company in particular, Clearview AI, has ushered in the use of this new technology with a sort of move-fast-and-break-things ethos, and how other companies in the field, and regulators, are struggling to agree on what guardrails are needed. I was eager to talk with Kashmir about what this tech can and can't do right now, and what our world might look like if it keeps getting bigger and more powerful in the future. But first, I asked her to explain how she learned about Clearview AI.
[MUSIC FADES OUT]
Hill: I found out about this radical startup that claimed to have done something pretty astounding. They said that they had scraped billions of photos from the Internet, from social media sites. It was like Google for faces. You just take a picture of somebody, and you find all the places on the Internet where their faces appeared, from their social media profiles on Facebook and Instagram and Venmo and LinkedIn to them in the background of photos, in a crowd, walking by on a sidewalk. It was astounding to me because I'd never heard of Clearview AI before. The experts I talked to hadn't heard of it. And I was just thinking, how did some little, you know, no-name startup do this instead of a big technology company like Facebook or Google? And I knew about police use of facial recognition technology, and my understanding was that it was pretty clunky, that it didn't work that well and that they were limited to government databases, like mugshots. So, it was very surprising both what Clearview claimed to do and the accuracy it was claiming. It said that the app worked with something like 98.6% accuracy.
Basu: Yeah. So, can I ask you to maybe back up and explain a little bit how does Clearview AI's technology work and maybe how is it different from other facial recognition technology that's out there? What exactly does it do?
Hill: Yeah. So, I've had the founder now run a search of my face and the way it works is you can either upload a photo or you can take a photo of somebody. It analyzes the face, develops an identifier for your face, basically this mathematical formula, and then it will search through its database of known faces and look for other faces that are in the same ballpark as that formula. So, it'll return results of other places on the Internet where my photo appears, along with a link to the website where it is. And so, it's a way of figuring out, somebody's social media profiles, their name, who they know, where they've been, maybe where they live. It's this trail that lets you know anything about a person based on where their photo has appeared online. And this was different from what facial recognition vendors have been offering up until then, because before they were just selling algorithms and you, the customer, had to develop the database. And so, law enforcement would have these criminal mug shots, or driver's license photos, and they would just run it on those databases. This opened up to searching for people you didn't already have photos of people who might live in another state, or live in another country, or even identify children. This was one of the first big databases that had children's faces in it.
Basu: Hmm. So, you said that the company claimed that it had very high accuracy of finding a correct match. Have you been able to verify that in any way?
Hill: Yeah, so at the time - and this really surprised me - it started with the NYPD, and it spread like wildfire through other law enforcement departments, and you had all these officers that are just downloading this app and using it in active investigations, and it hadn't been tested. No independent party had said how accurate their algorithm was, but the police officers said it worked really well. It's trial by fire. It wasn't until about three years after law enforcement first started using Clearview AI that it had its algorithm tested by a federal lab called NIST, or the National Institute of Standards and Technology, which has been testing facial recognition algorithms for 20 years now. And it did turn out Clearview AI's algorithm was really powerful. It was one of the top ranked algorithms the first time that it submitted itself for testing.
Basu: So, what are the limitations of Clearview AI? What does it tend to get wrong when it does get things wrong?
Hill: So, you know, facial recognition technology has become very powerful, but it is not perfect, and it can still make mistakes. The way that these apps work is, when you run a facial recognition search, it will list people, it will list faces in order of confidence, where the algorithm is more or less likely to think it's the same person. And so, you'll get this list. And then there is a human facial recognition examiner who looks through those photos and says, “Okay, I think this is the right person.” And so, we have seen this go wrong. There's been a handful of mistaken arrests in the United States that we know of, and one of them has involved Clearview AI. And so, this was a case where there were people buying designer purses at consignment stores in and around New Orleans with a stolen credit card. They had surveillance footage. They ran the guys' faces through Clearview AI, and one of them was a hit for a man who lived in Atlanta, Randal Quran Reid, and they went to his Facebook profile, and they saw that he had a lot of friends in New Orleans. And based on that, they actually issued a warrant for his arrest and a request for extradition, because he lived in another state. And so, Mr. Reid, the day after Thanksgiving, he's driving to his mom's house and he gets pulled over by four police cars, asked to step out of his vehicle, and they start arresting him and telling him there's a warrant out for him for larceny in Jefferson Parish and he doesn't even know what that is. He says, “Where's Jefferson Parish?” They said, “It's in Louisiana.” He said, “I've never been to Louisiana in my life.” So, he ends up arrested. He has to hire lawyers in Georgia and Louisiana. He is sitting in jail for a week awaiting extradition and trying to clear this up, trying to figure out why he's been accused of this crime. And one of his lawyers basically gets a detective to admit that it was facial recognition and so they send the detectives more photos of Mr. Reid and a video of his face and the detectives realize he's got this kind of telltale mole on his face that doesn't match the person they're looking for, and so they drop the case.
Basu: Listening to your story and imagining how it can be heard in almost two ways, right? There's the one on the one hand concerns about wrongful arrests that could arise because of this and then I hear the other way, which is well, this actually just makes the case that this technology - if it got a little bit better - would be even more valuable, would be a great tool, would actually create even more accuracy to some degree. Have you heard both of these arguments laid out?
Hill: I mean, I think we're at that point right now. The technology is very powerful, and it can be very accurate. I mean, you can actually see it for yourself. So, Clearview AI, there is this whole story about how they did what they did, but in many ways, they were working with technologies that are open source now that are kind of off the shelf and they broke through this taboo. And now there's other companies out there that are offering very similar services where you can go online and upload your face, or someone else's face and it will show you other places that face has appeared on the Internet. You can go to a site called PimEyes.com, upload your photo and see how well this can work. I think we are at the point now where we need to acknowledge that this technology is very powerful, it is very accurate, and we need to grapple with that as a society. How widely do we want this deployed? How ubiquitous should it be?
Basu: Yeah.
Hill: And who should have access to it? And should we as individuals have the right to not be put in these databases or get out of them if we have already been put inside them, which most of us have been.
Basu: Yeah, and I should say, we've already seen plenty of scenarios where the issue isn't whether or not the technology is accurate but whether it's being used ethically. And maybe I can ask you to say a little bit about what we've seen happen in other countries, in places like Russia and China, where we know that facial recognition technology is being used to monitor and even penalize people. Right?
Hill: Yeah, I mean, I think China is a great example because it shows the many ways facial recognition is being used and how you can get this kind of what they call “surveillance creep” or “function creep,” where it starts out with the idea of, we're going to use it for security and then you start using it for other purposes. So, in China, there is pretty widespread use of facial recognition technology paired with surveillance cameras, which is something we've kind of been resistant to in the United States, that we don't like the idea of real time tracking of people, by face. And so, in China, it's been used to identify protesters in Hong Kong when they were resisting kind of mainland takeover. And so, protesters would scale these poles with the cameras on them and try to paint over them to try to protect their privacy. We've seen it used to monitor Uyghur Muslims. There are actually companies in China that have developed the capability, they say, of flagging the faces of Uyghur Muslims just generally, that you can get an alert when somebody walks in who appears to be part of that ethnic group. We've also seen facial recognition used in kind of silly and frivolous ways. It's been used to automatically give jaywalkers tickets when they cross the road. You get a letter in the mail, and it tells you, “You've been fined.” This actually happened to a Chinese executive, and she got a ticket from a city she'd never been to. And it turned out her face was on the side of a bus and the cameras had detected it, you know, crossing when it shouldn't.
Basu: Oh wow.
Hill: It's been used to name and publicly shame people who wear pajamas in public in a city where they were trying to get people to comply with a different dress code. And it's even, it was even deployed in a public restroom in Beijing at the Temple of Heaven because they were having a problem with toilet paper thieves. So, they installed this technology so that you would have to look at the camera to get in a certain amount of toilet paper. And if you needed more, you had to wait something like seven minutes to look into the camera again. And so, I mean, you can just see once you start installing this technology, rolling this out, the question is, when do you stop? Where do you stop? How much is too much?
Basu: Mmm. Yeah, I mean, to bring it back to the U.S., you have written about how Google and Facebook have also been working on facial recognition - AI powered - facial recognition tools. But sort of decided against releasing them publicly, thinking maybe it was going too far, maybe it was too radical. Again, not really knowing who might be the customer for that kind of tool. What were their concerns in going further with facial recognition tools? And why did Clearview AI not appear to share those concerns?
Hill: Yeah, so the big technology companies, Google, Facebook, Apple, they have been working on facial recognition technology for more than the last decade and they have rolled it out in ways that benefit us, right? I open my iPhone by looking at it and it unlocks for my face.
Basu: I was going to say, most people do it many times a day, looking at their iPhone.
Hill: And that's a different kind of facial recognition technology. That's what they call a “one-to-one search.” You know, it basically has your face print and it's reacting to your face print. And what Clearview AI did is a one-to-end search, which is you upload a face and it's finding it in this huge database of faces. So, I would say those are different, but Google and Facebook both did this with our photos more than a decade ago. They introduced this technology where you could upload photos from parties, and it would suggest the names of your friends so that you could tag them. And they were very careful at that time to say, “This is just to recognize people that you know. We're not identifying strangers. We're not putting a name to a face of somebody that you wouldn't otherwise know.” But meanwhile, internally, they were continuing to develop the technology, and it was getting very powerful. They, they could do [LAUGHS] what Clearview AI did. And both Google and Facebook held it back, I think, because they were worried about all the downsides of a technology like that. And I think that people are pretty sensitive about their face and their anonymity. And the idea that you would just put this tool out in the world that strips us of that, that makes us recognizable, Google and Facebook just did not want to be the first companies to put that out there, and so they sat on it. And now these others came and kind of broke it out of the box and the Pandora's box is open now and we all have to live with the consequences. [CHUCKLES]
Basu: Right, right. You know, it's interesting listening to you describe that dynamic. It reminds me a lot of what happened with Chat GPT. And I mean, this is how I've heard it described and tell me if you disagree with any part of this, but lots of big tech companies were working on AI-powered text tools, but they were more cautious about releasing them publicly. And then when OpenAI went public with ChatGPT, it unleashed this really big scramble and all these conversations and concerns about regulating and reining in generative AI… Does something feel like it's rhyming to you here?
Hill: Oh, it is so similar. Google did the same thing with this kind of generative Al tools. Again, they developed them, and they sat on them. They were worried about the power of these tools. I'm sure worried about misinformation and disinformation and just kind of all the unexpected ways that we might use this and in OpenAI, like Clearview AI, kind of forged forward and had that bravery or that gumption just to throw the doors open and let people have access to these really kind of powerful and paradigm shifting technologies.
Basu: Hmm. Yeah. As if I needed another reminder of how non-hypothetical all of this is, my husband and I were really recently looking into home security systems. And we were told that we could get cameras around our house that have facial recognition technology so that we could know exactly who, like the, the name of the person delivering packages and who's walking their dog down the street. Certainly, there must be people who hear about those available security packages and say, “Yeah, I definitely want that. That would make me feel better or safer somehow.” So, I mean, you write about these issues all the time. How have we, as a culture, evolved in our thinking on what privacy means and, like, what kind of information we expect to have and use?
Hill: Yeah, I mean, the Internet is certainly fundamentally changing our notions of privacy. I remember Julian Assange of WikiLeaks used to describe Facebook as kind of like intelligence agencies' dreams, this idea that we would all go online and create these dossiers with tons of photos of ourselves and who all of our friends are, and you know…
Basu: Just troves of information.
Hill: …what we've been doing.
Basu: Yeah.
Hill: Yeah, I mean, we just share more publicly because of the platforms that exist. And there are these ways of kind of putting cameras everywhere, you know, you can have a dashboard cam in your car. You can put surveillance cameras all over your house and outside of your house and observing people. I do think there is a push to kind of collect data all of the time and that we are very clear eyed when it comes to the benefits. Like, “Oh, this is a good way to make sure no one steals my Amazon package from the front door. This makes me feel safe.” But what are the downsides of all that information collection in terms of, the police show up at the door and say, “We want the footage from your cameras.” And maybe it's because a crime was committed in your neighborhood, or maybe there's something they're trying to gather about you. And so, I think the thing that is difficult about privacy is it's hard to predict how these things are going to come back to bite you. One thing I would note is that, because you said this was offered to you, that you don't live in Illinois, because Illinois…
Basu: [CHUCKLES] That's right.
Hill: Illinois is one of the few places where the law has moved faster than the [CHUCKLES] technology and the state passed a law in 2008 called the Biometric Information Privacy Act that says that companies can't use somebody's biometrics - their face print, their fingerprints, their voice print - without consent. And so, you wouldn't be allowed to, if you lived in Illinois, just to throw facial recognition up on all of the cameras and kind of track all the people that lived around you without their consent, or it would violate this law.
Basu: Oh, that's really interesting
Hill: Yeah, so it also speaks to it, how much easier it is to get laws passed at the local level, at the state level, than at the federal level, where there's just so much gridlock. So, this is one of those issues that it's worth thinking about locally. What is your city doing? What is your state doing about this?
Basu: Yeah, I mean, just to go back to the example that we started with in our conversation, the lawyer who was turned away from seeing the Rockettes with her kid, and many lawyers who were turned away from various shows at various venues related to MSG, was that found to be a legal use of the technology?
Hill: Well, if you ban a bunch of lawyers from your facilities because you're trying to prevent lawsuits, [LAUGHS] you know, it's going to backfire. So, [LAUGHS] MSG did get sued by quite a few lawyers over the ban. And they were mostly relying on this kind of old law that was on the books from way before phones, that was passed because theaters, at the time, were banning theater critics from coming inside. And so, this one theater critic said, “We need a law that says if you have a ticket to a show, you need to be let in.” And so, some of the lawyers sued MSG and said they were violating this very specific, you know, state law in New York.
Basu: Huh. Very creative.
Hill: Yeah, and so they had some success with that, but many of the law firms actually ended up dropping those suits because they resolved the underlying litigation. And their suit over the ban was the only thing keeping them from getting back into Madison Square Garden. And these are like big Knicks fans that they [CHUCKLES] really want to go to the basketball games and these lawyers just love going to concerts at MSG. And so, a lot of them ended up dropping it.
And so right…
Basu: Huh.
Hill: …now, as it stands, MSG is still using that technology.
Basu: Now, I understand that Clearview AI specifically has also gone through lots of legal scrutiny. What is the latest on the various lawsuits that it has faced, and has more scrutiny influenced the company to change anything?
Hill: So, Clearview AI has been sued in a few states in the United States, mostly ones with kind of relevant privacy laws, Illinois being one of the big ones. There was one suit that has been settled, a state suit in Illinois brought by the ACLU. And as part of that settlement, Clearview said, “We will continue to offer this database,” they have of now 30 billion faces, that they will only offer it to law enforcement, and they won't sell it to private companies or to individuals. And so, they essentially agreed to limit the market for the product. And Clearview also has been investigated by privacy regulators outside of the United States, in Canada, Australia, Europe. And they have stronger privacy laws in those countries. And the regulators said that Clearview AI had violated them, that you can't put people in a database like this, collect this sensitive information about them, their face prints, without their consent. That basically kicked Clearview out of those countries. They stopped working with law enforcement basically outside of the United States. And some of the regulators did fine them and said, “you need to delete our citizens from your database,” which Clearview has not done. And Clearview has been able to get at least one of those fines overturned because of jurisdiction issues. And with a court saying, the British privacy regulator can't determine how foreign governments use their citizens data.
Basu: Hmm. You mentioned earlier that Clearview was one of the first companies to include children's faces in their database. I'm just thinking of things people can do themselves if they’re against this kind of technology. And this is something you have written about: how to handle your kids' photos on the Internet.
Hill: Yeah, so I recently wrote about kids photos, and I personally try to keep my photos of my kids private, like I'll post it to Instagram, but I have a private account limited to friends and family, or I share them in private messaging apps. And talking to experts, they say it does seem like more people are doing that. At the same time, you have this influencer culture where a lot of people are putting things up very publicly. So, I think there's a tug of war there. But I did this story about parents who were using one of these public face search engines PimEyes to look and see if there are any photos of their children on the Internet that they didn't know about. And one of the mothers I talked to found a couple photos she didn't know about and got one taken down, got the other removed from PimEyes search results. But there was also concern about a stranger doing that, you know, taking a photo of your child on the playground. And I talked to PimEyes about it, the owner of the company, and he said they were worried enough about that, that they actually started blocking searches of children's faces. So now, if you go to PimEyes and try to upload a minor's face you'll get an alert saying you're not allowed to do this search.
Basu: Mm.
Hill: And so, we're seeing these companies kind of recognizing some of the harmful ways their technologies could be used and trying to fix it, maybe to ward off regulation. But you can imagine a law that says, “Hey, you can't put kids’ faces in this database. You don't have the right to do that.”
Basu: You know, you write about how there have been other people, other groups who have built their own facial recognition tech tools with kind of surprising ease, doing something very similar to what Clearview AI is doing, bringing publicly available code and images and data and just matching them all together. So, with this technology and its building blocks already out there, what kind of regulation is needed to govern what happens in the future?
Hill: Yeah, I mean, there's… And this is kind of the similar tale of generative AI, too. You have all this information on the Internet, and these companies are scraping it. You know, they are treating the public commons of the Internet as the place to gather all of this data that they can use to create a big face database, or that they can use to train their language learning model. And so that is one of the questions with regulation is: Should these companies have the right to just collect all of this data, all of our personal data, from our faces to things we've written? So that's one of the questions right now when it comes to what the rules should be. There are also these databases and there's this question in Europe. They said you can't put their citizens in the database without consent. Here in the U.S. in the few places where we have states with privacy laws, they say you need to let people out of these databases. So, if you live in California or Connecticut or Virginia or Colorado, you can go to Clearview AI and say, “Hey, I want access to the data you're holding on me and I want you to delete me.”
Basu: Which is a different… requires a different step. Opting out is very different than opting in.
Hill: Right. And opt out laws tend to be less effective. And I actually looked at this, California requires companies to say how many of these requests they get. And California is a place with 34 million people. In the last two years, only around 700 of them have asked to delete themselves from Clearview's database. So, it is definitely the case that opt in laws are more effective and they will protect more people, which is one of the reasons why we say the Europe privacy regime is stronger than here. And then I think Illinois is a great example of a very effective law that says you can't use somebody's biometrics without their consent - that is another way that we could go.
Basu: What is it about what we're doing in America, or what we're thinking about in America, where we are so, [SIGHS] I don't know… We're so averse to articulating what we need these laws to look like.
Hill: Oh my gosh, that's a big [CHUCKLES] question. I mean, I do think, having covered technology for a while now, that there is a concern about limiting what these companies can do. And these are very important companies to the U.S. economy, and we have traditionally relied on them to make the right decision. And I actually think facial recognition technology is a good example of a time that they did. You know, Google and Facebook sat on this, and they held it back, and they did not think it was something they should put out into the world. And so, in that case, you know, kind of self-regulation worked. But then eventually you're going to have these smaller actors come along who don't have anything to lose and only have something to gain by putting some new radical technology out there. You know, what Clearview AI did was not a technological breakthrough. It was really ethical arbitrage. They were willing to do what other companies won't. And I think that we're going to see that more and more in technology because there is so much information to be scraped from the Internet. And now we have these powerful AI models where you can do a lot building on that information. The barriers to entry are just much lower, and I don't know, maybe it's going to take really seeing the downsides of this for us to pass regulations.
Basu: I've seen you describe your own beat as writing about the looming tech dystopia and how we can avoid it. So, I imagine this takes up a lot of your brain space. [LAUGHS] I mean, what is the dystopian, logical endpoint for this kind of technology?
Hill: I mean, my concern is that this technology just becomes ubiquitous and that, yeah, I mean, you walk into a store and they just know who you are in the same kind of creepy tracking that we have online where you see ads that are just a little too relevant, where you feel like you just feel like everything's getting watched, everything you do online. All of that kind of tracking could move to the real world, because your face would become the token. And you could have basically cookies attached to your face that say whether you're a liberal or conservative, what your financial position is, like how much you might be able to spend, whether you're a gambling whale, you know, whether you've had trouble with the law in the past. It could all just be attached to your face and used to judge you in the real world. The government could install it in all the surveillance cameras and just watch us wherever we go and easily identify protesters, easily suppress dissent. And I even just find it chilling to think about it in the hands of other people. The idea that I would be at dinner and I'm having a sensitive conversation. I'm gossiping about friends or about work and somebody sitting next to me gets intrigued and takes my photo and now knows who I am and totally understands the context of the conversation. Or you're at the pharmacy and you're buying something sensitive or embarrassing, and some troll there just takes your photo and like identifies you, tweets it out that you're buying condoms or hemorrhoid cream or a pregnancy test. I think about the way it could be used against women, against people seeking out abortions, going to a Planned Parenthood and you're walking out, and somebody can just take your photo and know that you were there. There's just so many ways in which this could be really chilling. We probably don't appreciate right now how much we depend on being anonymous. That, you know, you kind of go out into the world and you assume that the people around you don't know who you are. And that gives you a kind of safety, a security, an ability to do things that aren't just going to follow you and haunt you for the rest of your life. I don't know, just like looking sloppy, you know, to run to the grocery store. And with this kind of technology, it would turn us all into celebrities whose faces are recognizable, who's kind of every decision, you know, movement, conversation in public could be linked back to us. I just think that would be very chilling.
[MUSIC FADES IN]
Basu: This was a really fascinating conversation, Kashmir. Thank you so much for your time.
Hill: Thank you for having me on. I really appreciate it.
Basu, Narrating: Kashmir Hill is a tech reporter for “The New York Times” and author of the new book, “Your Face Belongs to Us: A Secretive Startup's Quest to End Privacy as We Know It.” You can find it on Apple Books. And if you're enjoying this show, “Apple News In Conversation,” please don't forget to follow us, and leave us a rating and review, in Apple Podcasts.
[MUSIC FADES OUT]