Politicians Angry After Amazon Facial Recognition AI Falsely Matches 28 Congress Members to Criminals

Non-profit watchdog, the American Civil Liberties Union of Northern California (ACLU) revealed flaws in Amazon's facial recognition technology called Rekognition this week. Currently, the tool is in use by law-enforcement agencies in Oregon and Orlando, misidentified Rep. John Lewis (D-Ga.) and 27 other members of Congress as criminals during an ACLU commissioned test.

Here's what happened when we tested @Amazon’s facial recognition tool on members of Congress. (Spoiler alert: it didn't go well.) pic.twitter.com/OovMb6XfT9

— ACLU (@ACLU) July 27, 2018

Dangerous inaccuracies

ACLU is concerned that the artificial intelligence (AI) system has dangerous inaccuracies that disproportionately target people of color. The test prompted lawmakers and congress members to submit letters both to Amazon's founder and CEO Jeff Bezos and to law enforcement agencies.

Sen. Edward Markey and Reps. Luis Gutiérrez and Mark DeSaulnier wrote a letter adressed to Bezos citing the results of ACLU's recent test and demanding the halt of Rekognition as well as details on where the technology is currently in use. Sens. Ron Wyden, Cory Booker, and Ed Markey published another letter addressed to Chief of Police Jeffrey Smith requesting a review of the 39 law-enforcement agencies regarding their use of facial recognition and their policies for preventing abuse and misuse.

NEW - @SenBooker, @SenMarkey and I are seeking details from 39 federal law-enforcement agencies about their use facial recognition technology, and what policies, if any, they’ve put in place to prevent abuse and misuse. https://t.co/RmEk8QApUV

— Ron Wyden (@RonWyden) July 27, 2018

“As [facial recognition] technologies have increased in availability and accuracy, and decreased in cost, we understand that as a law enforcement tool this technology would appear attractive,” the senators wrote. "However, this technology comes with inherent risks, including the compromising of Americans’ right to privacy, as well as racial and gender bias."

"As such, in exercising its oversight role, it is important for Congress to better understand the degree to which federal law enforcement agencies may be employing these new technologies," explained the authors. The letter featured a request for the department to respond by September 12, 2018.

A test that uses Amazon's tech

ACLU conducted their test with the same Rekognition system available to the public on Amazon's site. The watchdog built a database and search tool with 25,000 publicly available arrest photos and then proceeded to scan every current member of the House and Senate for likely matches.

[see-also]

The system returned 28 incorrect matches. "The false matches were disproportionately of people of color, including six members of the Congressional Black Caucus," wrote ACLU in their statement.

This is not the first time concerns about Rekognition have surfaced. Just last month, several Amazon shareholders also demanded the system be put on hold.

“While Rekognition may be intended to enhance some law enforcement activities, we are deeply concerned it may ultimately violate civil and human rights”, the shareholders said in a letter. “We are concerned the technology would be used to unfairly and disproportionately target and surveil people of colour, immigrants, and civil society organisations”.