Generative Persuasion
Algorithmic Propaganda
Generative Persuasion is an interactive installation that invites viewers to generate tailored propaganda and conspiracy theories on a military-style portable computing center using advanced artificial intelligence and personality-based microtargeting. Under the guise of a fictitious startup named Psybernetica, the generative persuasion application uses a Large Language Model (LLM) that has been conditioned and pre-prompted to generate polarizing and misleading content based on the characteristics a user chooses to target. The program is capable of producing social media posts, political advertisements, or conspiracy theories that target people based on the five-factor model of personality and whether the target is liberal or conservative. The goal of the project is to raise awareness to how easy and effective LLMs are at generating inauthentic content designed to influence politics or polarize the public. Generative Persuasion is a speculative present at the intersection of art, science, and ethics. It is based on research into the current capabilities of LLMs and past and present uses of psychometrics in political and disinformation campaigns and not a prediction of how this technology could develop.
It is a counterpoint to the current wave of technological exuberance and intended as a proof of concept without waiting for a whistleblower or evidence of a scandal.
Background
Microtargeting using the five-factor model of personality, sometimes referred to as psychographic profiling, has been used as a marketing tool for decades, but has become more effective with the explosion of behavioral data available through social media and other online sources. In 2018, it was revealed that the British consulting firm Cambridge Analytica had used a Facebook app called “This Is Your Digital Life” to harvest personal data from more than 87 million Facebook users. The app was really a veiled five-factor personality quiz that was used to find correlations between the users’ responses and their online behavior. These insights were then applied to Facebook users who did not take the quiz or agree to share their personal data. While technically a violation of its terms of service, Facebook’s API allowed the app to collect data from users who did not take the quiz themselves, but were friends with users who had consented. Cambridge Analytica is credited with the success of the 2016 Trump presidential campaign and the Leave.EU (Brexit) campaign. In the wake of the scandal, the SLC Group, Cambridge Analytica’s parent company, filed for insolvency as former employees formed the political consulting firm Emerdata Limited, which has been accused of using the same practices. Psybernetica is a thinly veiled reference to the legacy of Cambridge Analytica.
Large Language Models are a type of deep learning model using a transformer-based architecture that are capable of performing a variety of tasks, such as answering questions, summarizing documents, or generating very believable disinformation. Popular LLMs include Open AI’s ChatGPT or Google’s Gemini and are accessible through online interfaces. AI companies are aware of the potential “misuse” of their products. A report from an 2021 workshop that included participants from Georgetown University’s Center for Security and Emerging Technology, OpenAI, and the Stanford Internet Observatory concluded that LLMs will change influence operations, but have not yet been effective in campaigns. The report concluded that it is “all but impossible to ensure that large language models are never used to generate disinformation” (64). In October of 2024, Open AI revealed that it had blocked 20 global campaigns that were using AI for cybercrime and disinformation. It should be noted that these figures are coming from corporate research and only reflect disinformation that the company was able to detect–a rewording of AI generated content that keeps the basic message intact would make it undetectable.
References
- OpenAI Blocks 20 Global Malicious Campaigns Using AI for Cybercrime and Disinformation
- Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations
- Facebook Says Cambridge Analytica Harvested Data of Up to 87 Million Users
