China using AI to 'improve' courts, flag 'perceived human errors'

Artificial intelligence will now take part in every case in China’s legal system to help streamline judges’ workload, but critics have raised concerns about privacy and discretion.

The Supreme People’s Court in Beijing announced this week that judges must consult the AI on every case by law, reports the Daily Mail.

The AI, which helps users recommend laws, draft documents and flag “perceived human errors” in rulings, is part of the “smart court” system the high court has been developing since at least 2016. It began as a simple database, but now analyzes almost 100,000 cases a day from across the country every day.

But the database is now connected to police databases and China’s social credit system, meaning any decision by judges can be immediately enforced. According to the South China Morning Post, the court’s AI can find and seize the property of a convict “almost instantly,” and can even list it for sale online.

The system can also make quick adjustments to a convict’s social credit rating which, for example, could bar a debtor from using a plane, high speed train, hotels or other services.

Any judge that goes against the AI recommendation must submit a written explanation for why.

Other improvements through the smart court system including robot receptionists in courthouses to offer online legal help, voice recorders that automatically transcribe speakers in the courtroom and “virtual courtrooms” to hold hearing online, according to the Mail.

In a report published Tuesday, Xu Jianfeng, director of the supreme court’s information center, said the AI had cut a judge’s average workload by over a third. He also claimed it saved Chinese citizens 1.7 billion working hours from 2019 to 2021, and that society had saved more than 300 billion yuan ($45 billion) over the same period – an amount equal to about half of total lawyers’ fees in China last year.

Legal scholars in China have warned of the drawbacks on overreliance on the supreme court’s AI.

Zhang Linghan, professor of law the China University of Political Science and Law in Beijing, warned that the rapid rise of AI could reduce judicial discretion – a human judge’s power to make a decision based on individual intuition, experience and training.

This could improve efficiency and fairness to a certain degree, but “humans will gradually lose free will with an increasing dependency on technology”, she said in a peer-reviewed paper published Sunday.

“We must be alert to the erosion of judicial power by technology companies and capital,” she added, referring to how power over the smart court system rested in the hands of the few experts who wrote the code and supervise the database.