Agent readiness of the Forbes Cloud 100

Method

• Companies: the Forbes Cloud 100 of September 2025, the latest list on October 24, 2026.
• Scanner: isitagentready.com, Cloudflare’s agent readiness scanner, through its public API with the default set of checks. Each site was scanned at the address that its home page redirects to.
• Scores: the scanner gives a level from 0 to 5. We also count how many of its 17 site checks pass. Web Bot Auth is informational, and the 5 payment checks apply only to stores, so neither is in the count.
• Errors: when a check timed out or hit a rate limit, we scanned the site again and kept the result with fewer errors. When a check got a 403 or 401, we requested the same address in Chrome. Chrome got no agent file either, so the result stands.
• Refusals: Cohesity (403), Personio (429), and Axonius (429) refuse the scanner. For these three sites we requested the same files in Chrome, so they have no scanner level.

The full results are a CSV file with one row per company and one column per check.

Results by level

Level 1 needs a robots.txt file and a sitemap. 53 sites pass exactly 3 of the 17 checks: robots.txt, the sitemap, and the AI crawler rules. Checkout.com is at level 0 because its robots.txt has no User-agent line, and AppsFlyer because it has no sitemap.

Results by check

For two checks, Cloudflare publishes a figure for the web at large: in its April 2026 scan of the most visited domains, 4% had Content Signals and 3.9% passed Markdown negotiation. The Cloud 100 does better on both: 13% and 23%.

Two sites publish DNS-AID records without DNSSEC, which the check requires. One more site, Axonius, has a /.well-known/mcp.json file, but in the format that MCP clients use for their own settings. The scanner needs a server name in it.

The most prepared sites

Why Vercel and Hugging Face are at level 1

The levels build on each other, and Content Signals come before the rest. In this data, every site above level 1 has a Content-Signal line in robots.txt, and every site with one is above level 1. Vercel passes 10 checks and Hugging Face passes 8, but neither has the line, so both stay at level 1. One line in robots.txt would move them up:

Content-Signal: ai-train=yes, search=yes, ai-input=yes

Set each value to yes or no to match your own policy.

AI companies

The AI model and tool makers on the list are at level 1, except ElevenLabs. OpenAI’s home page gives agents one WebMCP tool, a site search.

AI crawler rules in robots.txt

21 of the 97 sites name at least one AI crawler in robots.txt. 14 of the 21 allow every crawler that they name. 7 sites block at least one AI crawler from the home page:

• Midjourney blocks 12, among them GPTBot, Google-Extended, and anthropic-ai.
• Canva blocks 10, among them GPTBot and ClaudeBot.
• Forter blocks 6, among them CCBot and Bytespider.
• Postman blocks 3, Notion 2, and Calendly and Together AI 1 each.

Sites that block /.well-known/

Miro, OneTrust, AppsFlyer, and ClickUp answer 403 to every agent file under /.well-known/, in Chrome too. A firewall rule that blocks the whole folder also blocks the files that agents look for there. If you add an agent file, check that your firewall lets it through.

Where to start

1. Add a Content-Signal line to robots.txt. It is one line, and the scanner’s levels depend on it.
2. Answer Accept: text/markdown with Markdown. On Cloudflare, Pro and Business zones can turn on Markdown for Agents. On other hosts, convert the page in your server or edge function.
3. Add Link headers that point to your docs and your Markdown copy.
4. If you have an API, publish an API catalog and your OAuth metadata. If you have an MCP server, publish its server card.

This site, flared.page, is at level 5 on the same scanner and passes 12 of the 17 checks. It is the site of Flared, a URL shortener with a remote MCP server for AI assistants. Its robots.txt and llms.txt are public, as an example.

All 100 sites

Rank is the 2025 Forbes Cloud 100 rank. For the three sites that refuse the scanner, the count is from Chrome and leaves out WebMCP.

Limits

• Each site was scanned once, on one day. Sites change, and so do the scanner’s checks.
• The scanner checks the home page and the files at the root of the site. An agent file on another host, such as a docs or API subdomain, does not count.
• Five WebMCP checks did not finish in either scan, after a time-out or a page navigation (Databricks, Automation Anywhere, NinjaOne, Algolia, and MaintainX). Four checks on Papaya Global did not finish either; in Chrome, its agent files return 404. They count as not passed, as the scanner reports them.