According to a test by the ACLU, Amazon’s facial recognition system incorrectly identified one in five California lawmakers were incorrectly identified as criminals by the software.
Vice News reports that Amazon’s facial recognition software, Rekognition, incorrectly identified 26 California lawmakers as people arrested for crimes during a test of the software by the ACLU of Northern California. The software was used by the ACLU to scan 120 photos of lawmakers against a database of 25,000 photos of individuals who were arrested, according to ACLU lawyer Matt Cagle.
The test found that one in five California lawmakers were falsely associated with mugshots from the arrest database, exposing the issues with implementing facial recognition technology in law enforcement services. The Rekognition software reportedly matched the lawmakers to mugshot photos with a confidence rate of 80 percent.
Assemblymember Phil Ting was one of the individuals incorrectly identified as a criminal by the software. He is also a lawmaker who has been calling for the limiting of facial recognition technology for some time, introducing a bill in February which would ban the use of facial recognition and biometric surveillance system on police-worn body cameras.
Amazon stated that it encourages police to use 99 percent confidence ratings for public safety users stating: “First, you should use confidence thresholds of 99% or higher to reduce errors and false positives,” in a guide for law enforcement states. In a blog post in February, Amazon stated: “When using facial recognition to identify persons of interest in an investigation, law enforcement should use the recommended 99% confidence threshold, and only use those predictions as one element of the investigation (not the sole determinant).”
But Cagle stated in an email that Amazon’s 99 percent confidence threshold is not representative of how the technology is used in real-life situations. “In the real world, Amazon does not consider the failure to use a 99 confidence score to be irresponsible and a prominent law enforcement customer for Rekognition has acknowledged lowering and not using any score at all, according to news reports,”Cagle stated, citing a report from Gizmodo in which police stated that they do not set a confidence threshold.
At the press conference, Assemblymember Ting stated: “If you get falsely accused of an arrest, what happens?. It could impact your ability to get employment, it absolutely impacts your ability to get housing. There are real people who could have real impacts.”