Skip to content
Botscent

From the server to the page

How the server half sends the request's own verdict to the page in a Server-Timing entry, and when.

The transport sends the request's own verdict from the server half to the page half. An adapter adds one Server-Timing entry to the response, and the page half reads the entry when it starts.

Response headers
Server-Timing: botscent;desc="1;chatgpt;1759300000123;signer.web-bot-auth.verified"
Cache-Control: no-store

Why the page needs the entry

Some agents show their evidence only in the request. For example, a hosted agent can sign the navigation and nothing else. The page half cannot see that signature. With the entry, the page verdict holds the evidence of both halves, so a single page sees both.

When there is no entry, the page half has no server evidence. It does not count the absence as evidence of a person.

When an adapter writes the entry

An adapter writes the entry only when both of these are true:

  • The request's own verdict is an agent.
  • The request is a document navigation. That is Sec-Fetch-Dest: document, or, without fetch metadata, a GET that accepts text/html from a user agent that begins with Mozilla/.

Link previewers, crawlers that name themselves and HTTP clients never run the page half. So they get no entry and no no-store. People's responses never carry an entry.

The adapter also does 3 things to keep the entry away from people:

  • It writes the entry after the shared-cache lookup, at most once for each response.
  • It adds Cache-Control: no-store to each response with an entry, so that no cache stores the agent's entry.
  • On every response it passes, it removes each botscent entry that the response already has. It keeps all other Server-Timing entries.

The format of the entry

The entry has 4 fields, separated by ;:

FieldExampleMeaning
Version1The major version of the format, and an optional minor version.
NamechatgptThe agent name. It is empty when the verdict has no name.
Time1759300000123When the server received the request, in milliseconds since the epoch.
Reasonssigner.web-bot-auth.verifiedThe reasons, separated by commas.

An entry is at most 256 bytes. A reader discards an entry that does not match the format, that is longer, or that has an unknown major version. A reader keeps reasons that it does not know.

What the page accepts

The page half accepts one entry whose time is between 10 s before and 120 s after the page's own navigation start. The page half ignores the entry in these cases:

  • The response has more than one botscent entry.
  • The entry does not match the format.
  • The entry's time is outside the window.

An accepted entry adds its reasons to the page verdict. Its name counts as a declaration. The browser shows Server-Timing to a page only in a secure context, that is on https or on localhost.

diagnostics().transport tells you what happened to the entry: pending, received, absent, unsupported or rejected.

The transport option

Every TypeScript adapter has a transport option, with these values:

  • 'auto': the adapter sends the entry only where it runs for each request in front of the cache.
  • 'always': you state that no shared cache stores your HTML.
  • 'never': the adapter never sends the entry.

In Python, the adapters take transport=True. In Django, set BOTSCENT_TRANSPORT = True in the settings. The Express and Astro adapters take 'always' or 'never'.

The default depends on where the adapter runs:

AdapterTransport by default
botscent/nextOn when the app runs on Vercel. Off when it is self-hosted.
botscent/vercelOn.
botscent/workers, for Cloudflare Workers and Netlify Edge FunctionsOn. Behind Workers Cache, cached pages carry no entry.
botscent/honoOn when the app runs on Cloudflare Workers. Off on Node.js.
botscent/expressOff.
botscent/astroOff.
botscent.asgi, botscent.django, botscent.flaskOff.

Why the transport is off at an origin

An origin server, such as Express or Django, cannot see whether a CDN in front of it stores HTML. A cache can ignore Cache-Control: no-store. Then the cache serves an agent's entry to a person, and the person's page reports an agent.

The page half's time window limits how long a stored entry can be replayed. It cannot show that the entry was written for this response. In a test, nginx ignored Cache-Control and replayed an agent's entry to a person within the same minute.

So origin adapters leave the transport off. With the transport off, your server code still gets the request's own verdict. Only the page half does not get the server's evidence.

Warning: Set transport: 'always' only when no shared cache stores your HTML. A cache that ignores no-store serves an agent's entry to people, and their pages then report an agent. After the change, run npx botscent check and read the person line.

Limits

  • The Next.js proxy cannot see headers that the route sets later. On the agent navigations it changes, its Server-Timing replaces one that the route set. People's responses do not change.
  • Vercel Routing Middleware runs before the response exists. It cannot remove an entry that the response itself carries, and Vercel appends the middleware's Server-Timing to the response's own.
  • CloudFront with a minimum TTL above zero served cache hits without the origin's Server-Timing in a test.
  • The Cache API in a service worker does not follow Cache-Control. If a service worker caches pages, keep navigations with a botscent entry out of its cache. As an alternative, leave the transport off.