AI crawler ads appear on Time pages served only to selected bots
Time and Mobian placed labeled brand content in crawler-facing pages, testing whether web ads can shape AI search results.
By Renata Fuchs · Policy Reporter
· 3 min read
AI crawler ads are now a documented publishing tactic: Time served selected AI bots a separate markdown version of some webpages containing labeled sponsored brand content, while ordinary visitors received the standard page. The effort, run with adtech firm Mobian for Ally Bank and the Project Management Institute, is aimed at influencing how AI assistants describe brands, though there is no evidence that it has changed any particular chatbot answer.
The Register reported on August 5 that the alternate Time pages consisted of text-heavy, crawler-friendly versions of articles. Their ads appeared as lengthy FAQ-style blocks containing brand information and marketing claims, designed to be readily ingested by automated systems. This is web content targeted at crawlers, rather than an ad displayed within a chatbot conversation.
The material was not visible in a typical human visit, but it was not inaccessible. Germany-based developer Vincent Schmalbach identified it by requesting the pages with different HTTP User-Agent headers, and The Register said it reproduced the result on several Time articles with a crawler simulator. The sponsored content was labeled on the markdown pages.
Which AI bots received Time's crawler ads?
In Schmalbach's tests reported by The Register, ClaudeBot, OAI-SearchBot and PerplexityBot received the alternate markdown pages. Google's standard search crawler received the same HTML as a human browser. GPTBot and ChatGPT-User returned HTTP 406 errors, while OAI-SearchBot, which supports ChatGPT Search, received the markdown version.
That distinction matters. The ads can be described as hidden from ordinary browsing, but calling them secret overstates the case: they were labeled as sponsored and available to someone who altered a request's crawler identity. The reporting also does not show that the same treatment was extended to every AI crawler, or that a model incorporated the content into an answer.
What are advertisers trying to achieve?
Mobian CEO Jonah Goodhart said the company wants to affect what AI assistants say about brands that pay for the service. He characterized the content as labeled, brand-verified facts backed by sources, while acknowledging that the weight an AI system gives it depends on the question, context and source credibility.
For marketers, the bet is that AI search referrals may carry more purchase intent than conventional search traffic. Ally Bank Chief Marketing Officer Andrea Brimmer told Business Insider that visitors arriving from AI platforms were 3.5 times more likely to open an account than traditional-search visitors, and that its AI-driven traffic had grown ninefold year over year. Those figures are executive statements, not independently verified performance data.
Business Insider reported that the pilot has sparked debate over brand safety, disclosure, scams and the prospect of competitors using crawler-facing content to distort recommendations. PMI CMO Menaka Gopinath said the sponsored material in the Time-Mobian pilot was clearly labeled.
Time is the only documented publisher example in the available reporting, and it remains unclear whether others have adopted the method. Public assurances from AI platforms that ads do not influence answers appear, according to Business Insider, to address advertising inside their own chatbots rather than content published on the web for their crawlers. For now, the relevant boundary is clear: the campaign demonstrates an attempt to influence AI-mediated discovery, not proof that any assistant's output has been bought or altered.
This story draws on original reporting from The Register.
