AIPasta uses AI to paraphrase and repeat disinformation

Brace yourself for a new source of online disinformation: AIPasta.

Research has demonstrated that generative AI can produce persuasive content. Meanwhile, so-called CopyPasta campaigns take advantage of the “repetitive truth” effect by repeating the exact same text over and over until it seems more likely to be true by those who encounter it many times.

In a study appearing in PNAS Nexus, researcher Saloni Dash and colleagues explore how these two strategies can be combined into what the authors term “AIPasta.”

In AIPasta campaigns, AI can be used to produce many slightly different versions of the same message, giving the public the impression that the message is widely held by many different people and likely to be true. The authors used both CopyPasta and AIPasta methods to produce messaging around the conspiracy theories that the 2020 presidential election was fraudulent or that the COVID-19 pandemic was intentional. In an online survey of 1,200 Americans recruited via Prolific, neither CopyPasta nor AIPasta were effective in convincing study participants that the studied conspiracy theories were true.

When examining just Republican participants, who might be predisposed to give credence to the specific conspiracies studied, AIPasta did increase belief in the false claim of the campaign more than CopyPasta. However, for participants of both parties, exposure to AIPasta—but not CopyPasta—increased the perception that there was broad consensus that the claim was true.

According to the authors, the AIPasta generated for the study was not detected by AI-text detectors, suggesting it will be harder to remove from social media platforms than CopyPasta, which is likely to amplify its effectiveness compared to CopyPasta.

Provided by
PNAS Nexus