AI provider status monitoring: vendor signals vs independent probes

Three signals, not one

WebmasterID Models keeps three distinct signals separate at every layer of the platform. They answer different questions and they are not interchangeable; conflating them is the fastest way to publish a misleading reliability claim.

1. Vendor-reported status. The provider tells us about themselves via their public status feed.
2. Independent HTTP probe. WebmasterID issues an HTTP request from our own infrastructure against a public, non-inference vendor endpoint.
3. Computed uptime window. A derived metric over a meaningful window of stored observations.

Every observation we store records its source as one of these three values. UI surfaces, the dynamic status endpoints, and the integrity guards all key off that field so a vendor signal cannot be silently relabelled as an independent measurement.

Vendor-reported status

A vendor-reported observation is the provider's own characterisation of their service health. It is the cheapest, fastest, and most-widely-available signal — but it is the provider reporting on themselves. We surface it because it is useful as a colour signal, not because it is an availability measurement. Every UI surface that renders one is labelled "Vendor-reported status observed by WebmasterID".

The Anthropic observer maps the Statuspage status.indicator enum (none / minor / major / critical / maintenance) into our canonical ObservedStatus vocabulary. The Google observer filters the Cloud incidents feed to entries whose affected_products mention Gemini / Vertex AI / AI Studio. Both observers run hourly via Vercel Cron at /api/cron/status.

Independent HTTP probes

An independent probe is a request issued by WebmasterID itself against a public, non-inference, non-billing vendor endpoint. The Anthropic probe targets https://api.anthropic.com/ (host root with no path) — an unauthenticated GET returns HTTP 404, which is exactly the signal we want: DNS resolves, TLS negotiates, the socket connects, and the API gateway processes the request. No API key is sent. No inference is triggered. No prompt or completion is produced. No billing is invoked.

A 5xx from the same endpoint means the API gateway itself is failing — that is the genuine "vendor degraded" signal a reachability probe gives us. A network error or timeout maps to unknown rather than to a guess at the cause.

The probe's wall-clock fetch time is recorded as latencyMs. The codebase documents this as fetch wall-clock time only and forbids it being relabelled as the provider's request latency. An integrity guard refuses to ship a build where any status pipeline file contains the literal phrase "API latency".

Computed uptime windows

Uptime is a derived metric over a window of durable observations. The platform does not publish an uptime percentage today. When one is published, it will be the share of stored observations whose observedStatus was operational over the requested window — labelled precisely as a "vendor-reported operational-sample rate" (or "probe-reachability sample rate" for probe-only windows). It will not be labelled "uptime" without qualification.

The sample threshold

The minimum number of observations required before any uptime-shaped number can be exposed is 24 samples in the requested window — declared as a single constant in lib/status-store.ts and verified by an integrity guard. Below the threshold, the uptimePercentage field on the window endpoint is null and the response's policyNote explains the gating decision explicitly.

Signal taxonomy + live observers

The two tables below document the source vocabulary and the currently-registered observers. The observer table is derived from lib/observers/index.ts at render time, so it cannot drift from the production pipeline.

Currently observed providers

Two providers have observers wired: Anthropic (vendor + probe) and Google (vendor only). The live status hub at /status renders the observer matrix; the per-provider live and windowed endpoints are at /api/status/<slug>, /api/status/<slug>/latest, and /api/status/<slug>/window?hours=24.

What we do not claim

The platform does not assert availability, does not make SLA commitments, does not produce a continuously-updated availability number, and does not relabel probe wall-clock time as request latency. Vendor-reported status is published with that framing on every surface it appears on; the moment it is rendered as anything else, an integrity guard refuses the build. See /docs/status-observations for the schema and policy in reference form.