Why Websites Block AI Agents (and What to Do)

Todd Brooks, Founderupdated October 31, 202611 min read

A wine merchant keeps his shop door on a crimson chain while he reads a courier's order slip through the gap.
Jump to a section11

TL;DR

  • General information, not legal, security or payment advice. RedHub is not affiliated with or endorsed by the companies and organizations named in this post. Product, protocol and availability descriptions are based on the cited public materials as of October 31, 2026 and may change. Confirm current documentation and obtain qualified advice for your implementation.
  • The case: In September 2026, SiliconANGLE and Retail Gazette reported that Amazon blocked Meta’s Muse agent from shopping on Amazon for users. The outlets attributed to Amazon concerns about agent identification, account access and credentials. They reported Meta’s position disputing the credential concern. This post summarizes that reporting, takes no position on the underlying facts or merits, and does not determine whether either party’s terms or conduct were lawful.
  • What a site objects to: not agents. Unidentified agents, agents handling credentials, and actions nobody on the site's side approved. Those are three separate objections with three separate fixes.
  • What to do: decide per agent and per action, not "agents yes or no." Allow a named operator to read the catalog. Require a human at checkout. Refuse anonymous traffic. Then make sure you can enforce what you decided.
  • The honest part: you are probably an agent operator somewhere else. The policy you write for your own site is the policy you will be judged by at someone else's.

Why do websites block AI agents?

Websites block AI agents for three reasons that get lumped together. The agent does not identify itself, so the site cannot tell it from a human or from a scraper. The agent handles customer login credentials or moves through account data the site did not agree to expose. And the agent takes actions, like purchases, that nobody on the site's side authorized. SiliconANGLE and Retail Gazette reported that Amazon blocked Meta’s Muse agent in September 2026 and attributed to Amazon concerns about agent identification, account access and credentials. They reported Meta’s position disputing the credential concern. This post takes no position on that dispute. The useful response for a site owner is a policy per agent and per action, backed by something that can enforce it.

Best for: founders, ops leads, and platform owners deciding how to treat the agents already hitting their site. This is cluster two of the agentic internet: who decides what AI agents do.


On a Sunday in late September 2026, shoppers who asked Meta's new Muse agent to buy something on Amazon started seeing a message instead of a cart. Retail Gazette reported the message said that continued access by an "unauthorised AI agent" breached Amazon's conditions of use. Muse had been public for under two weeks. By SiliconANGLE's account it had passed ChatGPT as the most popular free app on the App Store the Friday before. Amazon blocked it.

RedHub is not affiliated with the companies named here. This post summarizes cited public documentation and reporting and is not legal, security or payment advice.

This is the second cluster in our guide to the agentic internet, and the case is here because it is the clearest public example of the pillar's thesis: a site can refuse an agent no matter how capable or popular it is. What follows is the dispute with both sides attributed, then what sites that block AI agents are objecting to, then the policy that replaces a blanket yes or no.

The case, both sides

Here is what the two outlets reported, kept separate from any view of ours.

The statements below are allegations, positions and descriptions attributed to the cited outlets and parties. RedHub did not independently verify the agents’ technical behavior, the parties’ compliance with applicable terms or the legal merits of the dispute.

SiliconANGLE (opens in a new tab), September 21, 2026, on Amazon's position: "According to Amazon, one of the motivations behind the move is that Muse doesn’t identify itself as an agent when making purchases." The piece adds: "The company’s terms of service require agents to identify themselves by embedding a text snippet in HTTP requests." SiliconANGLE, citing GeekWire’s reporting of Amazon’s position, said Amazon viewed Muse’s prompted access to account pages and purchase history as undisclosed third-party activity within customer accounts. Amazon's statement to GeekWire, as SiliconANGLE quoted it, was that "we think it’s fairly straightforward that third-party applications that offer to make purchases on behalf of customers from other businesses should operate openly and respect service provider decisions about whether or not to participate."

Retail Gazette (opens in a new tab), September 22, 2026, adds the notice question: "Amazon said Meta had not told it that Muse would access its store and claimed the AI assistant did not identify itself while browsing the site." It reports Amazon also raised privacy and security concerns about how customer login credentials were handled.

Meta's side, from the same two sources. Retail Gazette: "Meta has disputed those concerns, maintaining that Muse cannot see customers’ passwords or payment information and that credentials shared with the service are kept in secure storage." SiliconANGLE describes a second, cybersecurity-focused agent that reviews sensitive actions such as purchases and prevents Muse "from directly accessing credit card numbers and account credentials," and reports that Meta has said it is working to expand the agent's data-protection guardrails. The two reports reviewed for this post did not quote a Meta response addressing the identification allegation as of their publication dates.

The cited reports also place the Muse dispute in a broader series of disagreements between Amazon and third-party shopping-agent providers. This post does not summarize or draw conclusions from those separate proceedings. Who is right in the Muse dispute, and whether Amazon's terms are enforceable against an agent acting for a logged-in customer, are questions for the parties and their counsel. This post takes no position on either.

What a site is objecting to

Set the dispute aside and read the objections as a list, because they are the same three a site owner of any size would raise. Each has a different fix.

ObjectionWhat it means for the siteWhat resolves it
The agent is unidentifiedThe site cannot tell an agent from a customer, or from a scraper. It cannot apply a policy to something it cannot name.Self-identification the site can verify: a signed request, a published IP list with a stable user-agent, or a declared text marker, depending on what the site accepts.
The agent handles credentialsIn general, a logged-in session is the customer's. A program driving it with the customer's password is, from the site's side, an unknown party inside an account.Scoped grants instead of shared logins. The payment rails in this set are built on that idea, and so is the Visa pilot's "Tokenized payment credentials rather than shared supplier-portal logins."
The action was not approvedNobody on the site's side agreed that this operator may purchase, cancel, post, or extract. The user's approval is not the site's.A per-action permission the site declares and can enforce, with the actions that need a human marked as such.

Notice what is not on the list. The site is not objecting to the agent being capable. It is objecting to not knowing who the agent is, what it holds, and whether it was allowed to do this. Those are the first three of the pillar's four checks, seen from the door.

Identification has the most mature answer

The first objection is the one with the most mature answer. Cloudflare's verified bots documentation (opens in a new tab) sets two bars for a bot or agent to be verified. The first is honest self-identification, meaning "it declares who it is deterministically, through a cryptographic Web Bot Auth signature, a published IP list with a stable user-agent, or reverse DNS." The second is non-abusive behavior: obeying robots.txt, keeping request rates reasonable, not evading site owner preferences.

The same documentation classifies each bot by what it does, and "Transact" is its own behavior, defined as "Checkout or other transaction actions on behalf of users." It also labels how a bot is operated, Direct or Intermediary, and explains why the label matters: "Because an intermediary acts on behalf of many different end users, the operator and the end user are not the same party." A consumer shopping agent used by many people is an intermediary in that sense. The site may trust the operator and still want to know which end user is driving, which is why Cloudflare describes an experimental Forwarded header for carrying that through. Experimental is their word, and we keep it.

Operators that choose to identify themselves have documented ways to do it, and your policy can treat identified and unidentified traffic as two different cases. That is a general point about the mechanisms, not a statement about any party to the dispute above.

Per agent, per action, and a human where it counts

"Do we allow AI agents" is the wrong question because it has no good answer. Allow all of them and you have handed your checkout to anonymous software. Block all of them and you have blocked the answer engines that send you customers, plus every operator who would have identified itself and behaved. The policy that works is a grid: operators down the side, actions across the top, and a human-verification mark on the cells that need one.

The Agent-to-Web Framework (opens in a new tab) proposal makes that grid literal. Its siteai.json file would let a site declare what an agent may do, "submit forms, complete transactions, book appointments, or extract data," alongside what it may read, with per-action rate limits and a flag that requires human confirmation. Its own worked example allows search and add-to-cart, requires a human at checkout, denies account creation and review posting, and refuses anonymous agents. It is a work in progress from a community group launched in March 2026, and the specification itself says that, like robots.txt, it relies primarily on voluntary compliance by reputable agents. It is a proposal, not an adopted standard. We mention it because the grid it describes is the right shape for the decision, and you can make that decision in a spreadsheet today without waiting for the file format to settle.

A worked grid for a mid-size retailer might read: named answer-engine crawlers may read the catalog, pricing, and availability at a declared rate. Named shopping agents from operators you have terms with may add to cart. Checkout requires a human to confirm on your page, for every operator, until you decide otherwise. Anonymous automated traffic to pricing and search is throttled, and login and account pages are protected. Review submission and account creation are closed to agents. That is a policy. "We welcome AI" is a slogan.

A stance you cannot enforce is a request

The last step is the one that separates a policy from a wish. Robots.txt and a published AI terms page are the polite layer, and compliant bots read them. Traffic that ignores them can be stopped only by something that can act on traffic: a WAF rule, a rate limit, a bot-management layer. And a "good bot" allow-list is only as good as your ability to verify that the bot presenting a name is the bot that owns it, since user-agent strings are text anyone can send.

A site can technically restrict or refuse automated traffic through access controls, rate limits and bot-management rules. Whether a particular restriction is lawful, enforceable or appropriate may depend on contracts and terms, consumer-protection obligations, accessibility requirements, competition law, platform commitments and other applicable law. Site owners should obtain advice for their specific implementation.

The complication to hold onto is that your company is likely an agent operator somewhere else. Your AP team's payment agent wants into a supplier's portal. Your research assistant reads competitor pricing. The grid you write for your own door is the grid you will be judged by at someone else's, and an operator that would not pass its own policy should fix that first. Zero trust for AI agents is the version of this argument for the agents inside your own walls.

Pairs well with

Authentication vs authorization is the distinction the first and third objections turn on. Can AI agents buy from you is the five-step path that runs once you have decided who may act. Zero trust for AI agents applies the same per-action logic to the agents you run.

RedHub self-assessment tool

The following is a paid RedHub product and is not connected with Amazon, Meta or the reporting described above.

The AI Bot and Scraper Traffic Control Readiness Gate grades exactly this posture on six controls, one row per web property, and returns READY, GAPS, or EXPOSED. Its gate is dispositive: if you cannot enforce on non-compliant traffic and cannot verify real bots, the property reads EXPOSED even when the weighted score is in the sixties, because a stance with no teeth is not a stance. It grades the setup, never people. It is not a bot detector, a scanner, a WAF, or a block list, and no tool guarantees a crawler obeys you.

Declare a stance you can back up

The AI Bot and Scraper Traffic Control Readiness Gate grades six controls per property, with a dispositive enforceability gate, and returns READY, GAPS, or EXPOSED with a portfolio dashboard and fix-first. One workbook, a worked five-property example, deterministic and offline. A readiness aid, not a bot detector or a block list. Not legal advice.

Get the Traffic Control Readiness Gate →

Keep going: the full map is in the agentic internet: who decides what AI agents do. The next cluster separates the three layers that identification, approval, and payment keep getting confused with: Web Bot Auth vs. AP2 vs. payment tokens.


Decision Guide

Write the grid now if: you have a public catalog, a login, or a checkout. Agents are already arriving, and the absence of a policy is a policy of allowing whatever shows up.

Block broadly only if: you have decided you do not want answer-engine traffic either, because a blanket block takes the crawlers that send customers along with the agents you meant to stop.

Best first step: pull a week of access logs and list the automated user-agents you can name and the ones you cannot. The second list is the one your grid has to have a row for.

FAQ

Why did Amazon block Meta's Muse agent?

SiliconANGLE and Retail Gazette reported that Amazon attributed the block to concerns including identification, account access and credentials. The outlets also reported Meta’s position that Muse could not see passwords or payment information and that supplied credentials were placed in secure storage. These are reported party positions; RedHub did not independently verify them and takes no position on the merits.

Can a website legally block an AI agent acting for a logged-in customer?

That is a legal question this post does not answer. What is observable is that sites do block agents, through terms, edge rules, and declared policies, and that the current public dispute between Amazon and Meta has not been resolved in a way either outlet reported. Ask counsel about your own terms before relying on them.

Should I block all AI agents from my site?

A blanket block also stops the answer-engine crawlers that send you customers and the operators who would have identified themselves. The approach this post recommends is per agent and per action: named operators may read, some may add to cart, checkout requires a human, anonymous traffic is throttled, and sensitive actions are closed.

How does a site tell a legitimate agent from a scraper?

By verification the site can check, not by the user-agent string. Cloudflare's verified-bot documentation lists a cryptographic Web Bot Auth signature, a published IP list with a stable user-agent, or reverse DNS as the ways a bot declares who it is. An operator that offers none of those has not given you a way, under Cloudflare's scheme, to allow it specifically.

Does the Traffic Control Readiness Gate block bots for me?

No. It grades whether your setup could allow, throttle, or block on six controls from marks you enter, and returns READY, GAPS, or EXPOSED per property. Your CDN or WAF does any blocking. It detects nothing, scores no people, and no tool guarantees a crawler obeys you.

How it decides
Diagram of the AI Bot & Scraper Traffic Control Readiness Gate: six weighted defenses scored to 0–100 and an enforceability gate forcing EXPOSED when controls can't be enforced.

The gate this post refers to, drawn from the tool’s own logic. See the tool.