Five covert influence operations (IO) that sought to use OpenAI’s models for dishonest online activity have been thwarted in the last three months. As of May 2024, using OpenAI’s services has not significantly increased audience engagement or reach for these ads.
According to OpenAI, the threat actors’ attempts to produce desired material have frequently been foiled by their dedication to building AI models with safety in mind. Furthermore, according to the business, OpenAI’s research has become more efficient thanks to AI tools.
The open-source community and distribution platforms have made substantial contributions to the fight against IO through detailed threat reporting. These results are being made public by OpenAI in an effort to encourage the larger community of stakeholders to exchange knowledge and best practices.
Upsetting hidden I/O
During the last three months, OpenAI has caused disruptions to multiple information operations by applying its models to a range of activities, including text translation, open-source research, social media profile creation, short comment generation, and simple code debugging.
Particular functions that were interfered with are:
Poor Grammar: A previously unreported Russian campaign aimed at the US, the Baltic States, Moldova, and Ukraine. This group created political remarks on Telegram in both Russian and English and utilized OpenAI’s models to debug code that ran a Telegram bot.
Doppelganger: Another Russian operation that creates headlines, edits and translates stories, makes comments in different languages on sites like X and 9GAG, and turns news pieces into Facebook posts.
Spamouflage: A Chinese network that generates texts in several languages, debugs code for database and website management, and uses OpenAI’s models for study on public social media activity.
The Iranian organization International Union of Virtual Media (IUVM) :creates and interprets long-form articles, headlines, and website tags that are posted on a connected website.
Zero Zeno :is a business enterprise based in Israel that produces content for various platforms such as Instagram, Facebook, X, and related websites.
These operations provided content centered on a variety of topics, such as criticisms of the Chinese government, the Gaza conflict, elections in India, European and American politics, and Russia’s invasion of Ukraine.
Despite these initiatives, none of these operations demonstrated a discernible rise in viewer engagement as a result of OpenAI’s models. None of the five operations received a score higher than a 2 on Brookings’ Breakout Scale, which evaluates the impact of covert IO. This indicates activity across several platforms but no breakout into genuine communities.
Trends of attackers
Several themes were found when these influence activities were investigated:
Content generation: Using OpenAI’s services, threat actors were able to produce a greater amount of content with fewer linguistic faults than they could have with just human operators.
Combining the old and the new: AI was applied with conventional media like copied memes or handwritten texts.
Fake involvement: While none of the networks were able to draw real engagement, a few produced answers to their own posts in an attempt to look engaged.
Productivity gains: By summarizing social media messages and fixing code, threat actors used AI to increase productivity.
Trends toward defense
Open-source research and industry sharing aided OpenAI’s findings. Among the defensive actions are:
Defensive design: Threat actors encountered friction from OpenAI’s safety mechanisms, which frequently stopped them from producing the intended material.
AI-enhanced investigation: By enhancing detection and analysis efficiency, AI-powered technologies cut down on investigation timeframes from weeks or months to days.
Distribution is important: In order for IO material to reach an audience, much like traditional content, it must be disseminated efficiently. None of the hampered operations succeeded in generating significant interaction, despite their best efforts.
Industry cooperation is crucial since it lessens the impact of OpenAI’s disruptions. Industry peers should share threat indicators with one another. Years of open-source study by the larger research community were beneficial to the company.
The human factor: Threat actors were prone to human error even with AI, as seen by the denial signals from OpenAI’s models that they posted on their websites and social media.
According to OpenAI, the company is still committed to creating ethical and safe AI. This entails taking preventative measures to stop harmful use and creating models with safety in mind.
While acknowledging that it can be difficult to identify and stop cross-platform abuses such as covert influence campaigns, OpenAI asserts that it is dedicated to reducing the risks.
