Social Rocket
Publishers in the USA and the UK are pushing for binding labelling obligations for disguised AI crawlers. In the USA, the Stealth Bot Prohibition Act (H.R. 9915) has been proposed. According to reports, British media companies are calling for a similar approach from 22 October 2026. These initiatives are not yet law. For German publishers, the question of how to distinguish legitimate search engines from unidentified automated access is becoming more urgent.
What the demand is about
The American News/Media Alliance supports a draft that would require bots to disclose their identity and purpose. The target is in particular crawlers that conceal their origin or imitate human visitors. The publishers' association represents the economic interests of rights holders and is not a neutral regulator.
According to Digiday, 22 October, British media companies are also campaigning for similar transparency rules. This political push does not establish either an already adopted British crawler obligation or a new EU rule.
Why bot figures need to be read carefully
The security provider DataDome reported in its Bot-and-Agent-Security-Report 2026 that 70,9% of the detected malicious bot requests were attributable to scraping. The scraping attempts classified this way rose by 185.2% from July 2025 to June 2026. The basis is access to more than 75.000 websites protected by DataDome.
This explicitly does not mean that 70,9% of all web requests are scrapers, or that every detected scraper comes from an AI provider. The data concerns a classified subset within one provider's network. Nor do they show whether a particular access occurred without a suitable usage permission.
Separating technical detection from rights clearance
The robots.txt file signals the desired access rules to cooperating crawlers. Deliberately disguised bots do not have to respect such rules. Publishers therefore need an inventory of known crawlers, CDN and server logs, suitable rate limits and identity checks, without blanket-blocking search engines or agreed partner access.
The significance of new contractual website access rules is shown in our report on Meta's Commercial Terms. The SPUR approach to content telemetry, by contrast, deals with the later proof of content use.
Real names alone do not pay for content
Even a correctly identified bot thereby does not automatically receive a right to reuse. Conversely, its identifiability does not create remuneration. Technical clearance, the scope of copyright licensing and economic value remain separate decisions. This is shown by Google's contractual news AI pilot projects as well as the considerations around registration and direct reader loyalty.
Assessment: The immediate practical step for German marketing and publisher teams is a documented approach to legitimate and unknown bots. The legislative process in the USA and the British demands should be monitored, but not presented as rules already in force.
Sources: Digiday, 22.10.2026; News/Media Alliance, H.R. 9915; DataDome, Bot Traffic & Threat Trends 2026.
Illustrative image: Taylor Vick / Unsplash. Does not depict a specific crawler.