Social networks flooded with Artificial Intelligence content: Increased risk of disinformation and manipulation

In a project titled “AI Slop & Algorithmic Virality,” researchers from AIF Forensics analyzed the spread of artificial intelligence (AI)-generated content on TikTok and Instagram in three European countries.

They found that up to 25 percent of the most popular searches on TikTok contain synthetic AI images, with more than 80 percent of AI content on TikTok coming from visuals created by “AI agent accounts,” similar to automated profiles that mass-share content, while on Instagram, only 15 percent of posts are made by AI agent accounts. What we refer to in this study as “artificial intelligence agent accounts” or AAA (Artificial Intelligence Agent Accounts) refers to accounts that post synthetic AI content using partially or fully automated processes.

"Such accounts use AI-generated tools to automate content creation, enabling rapid and iterative testing of platform algorithms and audience interests with the aim of finding content that will go viral," the researchers say.

They point out that such social media accounts use artificial intelligence tools to automatically generate and publish content. They publish a large number of posts created with artificial intelligence in order to “outdo” recommendation algorithms and increase the chances of the content going viral. They systematically test what attracts the audience’s attention and how the recommendation algorithms work.

What are artificial intelligence agent systems?

An account is considered an "agent" if its last 10 posts contain exclusively images created with artificial intelligence, even if it has previously posted different content (which was rare), they point out.

AI agent systems use artificial intelligence to: research trends, create videos/images, schedule posts, and tailor messages to the audience. All in an automated cycle, without the need for human intervention. Such accounts can spread content massively and have a huge impact on the public sphere.

The content is photorealistic, making it difficult to distinguish between fake and real. AI content labels are often hidden, unclear, or nonexistent – ​​especially on desktop versions. As part of a systematic analysis, researchers studied the labeling of AI-generated content on TikTok and Instagram and analyzed search results for 13 hashtags (#trump, #health, #history…) in Spain, Germany, and Poland in June 2025.

The most popular content on political topics and beyond was manually reviewed, identifying so-called “AI agent accounts” – specialized profiles that automatically generate and distribute AI content.

The development of artificial intelligence agents – which can independently perform complex tasks and communicate with their environment – ​​is expected to enable the complete automation of the content creation and publishing process without the need for human intervention.

"Artificial intelligence waste" as the new digital waste

Social media is increasingly saturated with content created by artificial intelligence – which “tricks algorithms” with visual credibility and viral potential. This phenomenon, called “AI waste”, refers to mass-produced, low-quality and potentially deceptive content created by artificial intelligence. Examples include AI videos advertising dubious dietary supplements, “resurrections” of dead children due to true crime stories, AI-generated religious and political scenes (e.g., Trump in Gaza), and viral but meaningless content. The goal of such content is to attract attention, garner views and generate revenue through algorithmic placement – ​​without much human effort. Regulation and legislation

Although platforms (in accordance with the Digital Services Act) have pledged to label content created by artificial intelligence, research shows that only around 50 percent of such content is labeled as created by artificial intelligence, despite European regulatory requirements, which increases the risk of disinformation and manipulation.

The existing Artificial Intelligence Act (EU, 2024) and the Digital Services Act (DSA) require that content created by artificial intelligence be clearly labeled. Platforms (TikTok, Instagram) promise technical means for detection, but labels are often invisible. Spain has announced fines of up to 35 million euros for unlabeled content created with artificial intelligence.

Current labeling measures are insufficient and inconsistent and do not protect users. If this trend is not curbed, the flood of artificial intelligence could seriously damage trust in the digital space.

/The Geopost