How Notch rates AI agents

01What we rate

We rate one agent at a time: a defined system with its own configuration, data, integrations and paying clients, whose revenue and costs can be separated from the rest of its builder's business.

The rated object is the agent's Net Distributable Earnings: its recurring revenue from external clients, minus the direct costs of running it (model fees, compute, tools and platforms) and the cost of maintaining it.

02What every rating contains

Every Notch rating has four parts, always published together:

• Grade, N1 to N9: how strong and reliable the agent's earnings are today. N1 is the strongest; N9 the weakest.
• Half-life: the expected number of months until the agent's earnings fall to half their current level. This is Notch's signature measure, built for the risk that matters most for agents: being overtaken by better models or competitors.
• Confidence (High, Medium or Low): how much verified evidence sits behind the rating.
• Outlook (Improving, Stable, Weakening or Under review): the likely direction of the grade over the next six months.

03The Notch Scale

04Who can be rated

An agent can be rated when it has:

• at least three months of verified revenue from paying clients outside the builder's own group
• revenue and costs that can be separated from the builder's other activities
• read-only access to its payment, accounting and model billing data
• lawful activity that complies with the terms of the providers it depends on

05How we verify

Every figure behind a rating is checked against at least two independent sources. Self-reported numbers are never used on their own.

We connect read-only to the agent's payment processor, accounting records and model and cloud billing, and review task logs and redacted client contracts. We reconcile revenue across sources, exclude one-off and related-party income, allocate shared costs by measured usage, and sample task logs to confirm the work is done by the agent rather than by people.

06The six factors

Each agent is assessed on six factors.

07From factors to grade

The six factor scores combine into a weighted score that indicates the grade. Some weaknesses limit the grade however strong the rest of the agent is: for example, an agent that depends on a single client for half its revenue, or that relies on people for most of its work, cannot receive a strong grade. A rating committee may adjust the result by at most one notch, with every adjustment recorded and explained.

08Half-life

Half-life blends what we observe in the agent's own earnings with what we know about agents of the same kind. The less history an agent has, the more weight its category carries. Agents protected by proprietary data, deep integrations or specialist workflows are expected to last longer; thin layers on general-purpose models are expected to fade faster. No agent is assumed to last forever, however fast it is growing today.

09Live ratings

Ratings are not annual snapshots. Verified data refreshes every month, and grades, half-lives and outlooks are recalculated each time. Significant events, such as losing the largest client, changing the main model provider or a sharp fall in revenue, trigger an immediate review. To avoid noise, a grade changes only on a sustained move, not a single month's data.

10Independence

• Fees never depend on the grade awarded.
• People who negotiate fees take no part in rating decisions.
• Every grade is approved by a committee, including a reviewer who did not prepare the analysis.
• Builders may appeal once, with new information.
• Our methodology and every change to it are published.
• Any conflict of interest is disclosed, and the person concerned stands aside.

11An evolving standard

This is version 0.1, written before our first ratings. Factor thresholds, weights and half-life assumptions will be calibrated on the first rated agents and revised openly, with each change and its reason published. The full technical methodology is available to pilot participants and institutional partners on request.