A Chinese study warns that introducing artificial intelligence (AI) into the nation’s existing social credit system could be a mistake. Published in the Journal of Sichuan Normal University, the study warns that this could turn the rating system into something more akin to a “large-scale surveillance tool.”

Interestingly, the research was funded by a Chinese government-backed project examining social governance and institutional innovation. At present, China’s social credit score system is a collection of different program.

It combines things like a financial credit system similar to credit ratings or reports in the West. It also includes regulatory blacklists that list individuals or companies who have fallen foul of the law for whatever reason. The system also includes local behavioral experiments that can reward or punish people based on set criteria.

It also includes a set of government accountability systems to monitor compliance and conduct among public officials. These programs are not connected into a single national score for each citizen, but taken together can hamper a person’s access to things like loans through automated decisions.

Introducing AI into the mix, the study argues, could turn this into the Orwellian system, which many Western critics say it already is.

According to a report, the AI-driven system could analyze your activities, financial history, relationships, and other available data at an enormous scale. From that, it could determine that you resemble people who might cause problems.

Under this scenario, a person may not have necessarily done anything wrong, but the system could treat them differently anyway.

Source link