Chinese Scientists Created an AI 'Prosecutor' That Can Press Charges

In China, an AI could send you to jail. Researchers in the country have developed a machine that can charge people with crimes with the help of artificial intelligence.

This AI "prosecutor" can file a charge with more than 97 percent accuracy based on a verbal description of the case, as per the team. South China Morning Post reported that the machine was built and tested by the Shanghai Pudong People’s Procuratorate, the country’s largest and busiest district prosecution office.

According to Professor Shi Yong, director of the Chinese Academy of Sciences’ big data and knowledge management laboratory, and the project’s lead scientist, the technology could reduce prosecutors’ daily workload, allowing them to focus on more difficult tasks.

Shi and his colleagues said that “the system can replace prosecutors in the decision-making process to a certain extent,” in a paper published this month in the domestic peer-reviewed journal Management Review.

Better 'order' required

Though countries like Germany now use AI technology such as image recognition and digital forensics to increase case processing speed and accuracy, China’s prosecutors were early adopters when they began using AI back in 2016. Several of them now employ an AI tool known as System 206. 

The tool can evaluate the strength of evidence, conditions for an arrest, and how dangerous a suspect is considered to be to the public.

But all existing AI tools have a limited role since "they do not participate in the decision-making process of filing charges and [suggesting] sentences," Shi and colleagues told the SCMP.

Making such decisions would require a machine to perform more complicated tasks, such as identifying and removing any contents of a case file that are irrelevant to a crime, without extracting the useful information, and converting complex language into a format that a computer can fathom. 

The AI prosecutor developed by Shi’s team can run on a desktop computer. For each suspect, it would press a charge based on 1,000 “traits” obtained from the human-generated case description text, most of which are too small or abstract to make sense to humans. System 206 would then assess the evidence.

The machine was “trained” using more than 17,000 cases from 2015 to 2020. For now, it can identify and press charges for Shanghai’s eight most common crimes which include credit card fraud, running a gambling operation, dangerous driving, intentional injury, obstructing official duties, theft, fraud, and “picking quarrels and provoking trouble” – a catch-all charge often used to stifle dissent.

Shi and his team said that the AI prosecutor would soon become more powerful with upgrades. It will be able to recognize less common crimes and file multiple charges against one suspect. 

Concerns arise

The South China Morning Post reached out to a prosecutor in the city of Guangzhou who expressed some apprehensions about the use of AI in filing charges. “The accuracy of 97 percent may be high from a technological point of view, but there will always be a chance of a mistake,” said the prosecutor, who requested to remain anonymous.

Direct involvement of AI in decision-making could also affect a human prosecutor’s autonomy. Most prosecutors did not want computer scientists “meddling” in a legal judgment, the prosecutor said.

In the U.S., we're a long way off from the so-called idealized future promised by AI. We're still working on the bugs in forensic algorithms. A good example is the 2017 District of Columbia court case. The case involved an anonymous defendant who nearly experienced the fallout from faulty programming that was presented as evidence in court.  

To help address this and related concerns, Rep. Takano reintroduced the Justice in Forensic Algorithms Act, a bill aimed at ensuring the protection of civil rights for defendants in criminal cases and establishing best practices for the use of forensic AI software, earlier this year with co-sponsor Dwight Evans (D-Penn.). “We simply don’t allow the argument by software companies that their proprietary software interests or trade secrets are more sacrosanct than the due process rights of the defendants,” Takano had said in an interview with Interesting Engineering.

However, regardless of AI's imperfections, China continues to use AI in nearly every sector of the government to improve efficiency, reduce corruption, and strengthen control. Chinese courts have been using AI to help judges process case files and make decisions such as whether to accept or reject an appeal. Most Chinese prisons have also adopted AI technology to track prisoners’ physical and mental status, with the goal of reducing violence.