Cloudflare Adds Continuous Trust Monitoring for Verified Bots

From one-time verification to continuous monitoring

Cloudflare has introduced a system that evaluates whether Verified bots continue to behave responsibly after their identity has been confirmed. The company compares bot identity to a driver's license: verification establishes who operates a bot, but trust depends on what it does afterwards. A verified bot can still ignore website preferences, overwhelm origin servers, or attempt to evade controls, often through misconfiguration rather than malice.

To become a Cloudflare Verified Bot, operators must meet three criteria: identify themselves accurately and securely, do only what they say they will do, and avoid abusive behavior. Bots use Web Bot Auth to sign requests that establish their identity, and BotBase catalogues them in an open format. With identification addressed, Cloudflare says the focus now shifts to the second rule of verified bots: doing no harm.

Trust bands instead of permanent judgments

The system measures how often a crawler respects site preferences and translates those observations into an explainable trust band. This is a range showing how consistently a bot honors preferences and which observed behaviors informed the assessment, rather than a permanent judgment about a bot or its operator.

Cloudflare distinguishes between risk and trust: risk describes the likelihood that a particular request may be harmful and can change from one moment to the next, while trust is built over time through repeated and consistent behavior. The company cited the de-listing of Perplexity's bot from the Verified bot list in 2025, after it sought to access resources on websites that had opted out of scraping and subsequently attempted to circumvent controls when those requests were blocked.

Robots.txt compliance and rate limit responses

The initial signals evaluated are whether a bot respects instructions and whether it responds to direct feedback. A robots.txt file is the primary way websites communicate preferences, and the system considers whether observed traffic matches those published preferences over time, in line with RFC 9309. Cloudflare notes that robots.txt sets the minimum rules for what bots should not do, and that the absence of a statement does not imply permission. The company looks for repeated patterns of disregard rather than isolated violations.

The system also watches how bots react to HTTP 429 responses, which direct visitors to reduce their request rate. Consistently slowing down or pausing after receiving that signal increases trust, while repeatedly ignoring it lowers the bot's standing.

Availability plans

The system works alongside Bot Preference Sync, which helps site owners publish their preferences, and Bot Management, which lets owners decide how automated traffic is handled. Cloudflare is already using the system internally to evaluate how consistently Verified bots honor website preferences, assessing sustained patterns rather than isolated events. The company plans to make it available to all Cloudflare customers, who will be able to choose the trust threshold used to decide how Verified bot traffic is handled. Trust scores are also expected to be shared publicly on Radar in the future.

Sources: Cloudflare