OpenAI, the company behind the ChatGPT artificial intelligence (AI) system, published a report on Wednesday that showed how a lone Chinese official accidentally exposed a worldwide network for intimidating dissidents by using ChatGPT to review and edit their reports.
The OpenAI report did not identify the Chinese official responsible, and it might have been several people using a single account, although the report described it as an “individual” user.
“The user’s activity revealed a well-resourced, meticulously-orchestrated strategy for covert IO (information operations) against domestic and foreign adversaries, termed ‘cyber special operations,’” the report said.
“As part of this strategy, they tried to use our model to plan a covert IO targeting the Japanese prime minister, but our model refused,” OpenAI said.
Japanese Prime Minister Takaichi Sanae has been a prime target of the Chinese Communist Party since shortly after she assumed office because she said a potential Chinese invasion of Taiwan would be a “survival-threatening situation” for Japan. That specific language would authorize Japanese military intervention, even under the current pacifist constitution. Japanese prime ministers had previously been cautious about unambiguously stating they would intervene to protect Taiwan.
According to the OpenAI report, the Chinese user tried to enlist ChatGPT’s help to conduct an information-war campaign against Takaichi even before she became prime minister, because she “publicly criticized the state of human rights in Inner Mongolia.” The Chinese user also wanted the A.I. to help with “stirring up anger against U.S. tariffs.”
OpenAI said the Chinese threat actor also used ChatGPT to “edit periodic status reports,” which revealed that China was already conducting information warfare against Takaichi before it approached ChatGPT, and was also running a “comprehensive effort to suppress dissent and silence critics both online and offline, at home and abroad.”
“This effort appears to be large-scale, resource-intensive, and sustained, engaging at least hundreds of staff, thousands of fake accounts across scores of platforms, and the use of locally-deployed AI models, especially Chinese ones,” the report said.
In other words, this Chinese official decided to use ChatGPT to cut some corners on editing reports to their superiors – and inadvertently revealed to the artificial intelligence the breathtaking scope of China’s program for transnational repression.
Those details are no great surprise to students of the Chinese Communist Party’s repression, but it is rather embarrassing to have a Chinese official describe them in detail, in the course of trying to get a robot to do their paperwork for them.
OpenAI said the details fed into ChatGPT included descriptions of “dozens of tactics, ranging from abusive reporting of dissidents’ social media accounts, through mass online posting, to forging documents and impersonating U.S. officials to intimidate critics.” OpenAI researchers were able to find “online activities that were consistent with some of the tactics this user described.”
One of the other targets of this campaign besides Takaichi was Chinese social media user “whyyoutouzhele,” also known as “Teacher Li Is Not Your Teacher,” the online handle of a prominent dissident named Li Ying.
If the reports fed by the Chinese official into ChatGPT were accurate, China’s information warfare campaign has generated “millions of posts on Chinese networks, and tens of thousands of posts on foreign ones, using thousands of accounts, many of which were fake or working under the direction of the operation.”
OpenAI found some evidence that these IO campaigns were effective, including reduced follower counts and lower online activity for targeted dissidents, and some dissident accounts were deleted entirely, although high-profile targets like Japan’s prime minister and Teacher Li Is Not Your Teacher were not damaged much.
“This is what Chinese modern transnational repression looks like. It’s not just digital. It’s not just about trolling. It’s industrialized. It’s about trying to hit critics of the CCP with everything, everywhere, all at once,” OpenAI principal investigator Ben Nimmo told reporters.
Read the full article here