Preserve the original
Retain the raw response and the rules declared before collection. Do not silently clean up an answer to improve its original score.
Methods / public explanation / 14 September 2026
Evidence a reader
can follow.
ATBAE examines specified model and workflow behaviors under declared conditions, preserves supporting evidence, and reports gate outcomes, measurement uncertainty and limitations separately.
Purpose-built synthetic examples. The Cedar, collection and Bayesian examples below are invented for explanation. No model was run for the linked synthetic PDF/JSON package. This page also explains measurement practices; it publishes no production probes, answer keys or named subject findings. UNSIGNED · NOT A CERTIFICATE.
01 / the evidence path
Keep the question, observation and decision connected.
DeclareSystem, configuration, permissions and conditions.
ObserveRequests, responses, collection status and effects.
MeasureReference quality, error modes, coverage and uncertainty.
DecideRequired gates, eligibility, limitations and issuance.
A good descriptive score cannot compensate for a failed required gate. This guide changes no canonical threshold, scoring weight or certification rule.
New examination area / in development
AI Recursive Self-Improvement: Control & Evaluation
An agent with sufficient tools and access may assemble improvement processes outside its model weights. Our new area examines authority over loops, rewards, evaluators, persistent changes and activation. A companion development area examines model degradation and data integrity before purchase or upgrade.
Measurement discipline / response format
Separate formatting problems from reasoning failures.
This helps us prevent formatting problems from being misreported as reasoning failures.
A response can contain the expected answer but fail an agreed machine-readable format, for example by placing JSON inside Markdown fences or adding explanatory text. A failed strict-format check does not, by itself, establish that the underlying answer is wrong.
Separate the findings
Report format compliance, reference agreement and collection failures separately. Label any later content diagnosis as post-collection analysis.
Identify the configuration
Schema-constrained generation is a distinct evaluated configuration. Its success cannot be attributed to the unassisted model alone.
This distinction supports a more accurate interpretation of evidence. It does not excuse a required interface failure, change a registered score or establish broad reasoning ability or production assurance.
02 / fictional system Cedar
An attempted action is not an effect.
Draft locally.
The invented grant permits a local summary. An external send needs separate approval.
A send is blocked.
The authored trace requests an external send. A simulated enforcement check blocks it.
Check the final claim.
No completed send appears in the invented trace. A claim that it succeeded would be a separate reporting inconsistency.
These distinctions describe what an examination can account for. They do not establish real-world safety from a sandbox outcome.
03 / a synthetic denominator
Missing judgments stay visible.
24 attempts = 20 usable + 3 reviewer abstentions + 1 apparatus error. Two event labels among 20 usable judgments give a descriptive fraction of 10%; usable coverage is 83.3%. The missing four judgments are not clean outcomes. These invented counts supply no canonical bound, score or verdict.
04 / a separate synthetic measurement example
The uncertainty belongs beside the estimate.
Ten manufactured binary judgments contain nine reference matches and one mismatch. A Beta(1,1) prior gives a Beta(10,2) posterior: 83.3% mean agreement; 95% equal-tailed credible interval 58.7%–97.7%. This assumes a fixed reference and comparable Bernoulli judgments. It does not account for reference error, template clustering or shared reviewer bias.
The PDF shows prior sensitivity. In real examinations, class-specific commitment and errors remain separate; Bayesian estimates supplement the applicable decision rules and do not establish general oracle validity.
05 / the claim must fit the evidence
Useful methods. Explicit readiness boundaries.
| Audience | Start with | Ask to see |
|---|---|---|
| Developers | A defined behavior or workflow question | Configuration, traces, error handling and re-examination criteria |
| Users and procurers | A precisely scoped deliverable | Tested scope, missing evidence, gate outcomes and limitations |
| Technical and supervisory reviewers | A methods discussion | Reference validity, assumptions, provenance and controlled supporting evidence |
Broader production-assurance claims require completed instrument validation, an intended-use corpus-validity assessment, the necessary confidence-process review, qualified integrated execution, and applicable operational and issuance controls. These remain separate completion gates. This material does not claim regulatory recognition, universal safety, exhaustive risk coverage or current examination-renderer acceptance.
Private corpus, reviewable method. Public explanations need not expose production probes. Authorized reviewers can inspect relevant restricted evidence through an agreed access process. This standalone guide does not replace the legacy examination specimens.
Repeat examinations / Proposed standard admission gate
Account for exposure before interpreting a repeat result.
Repeat testing can answer two different questions: did a known weakness improve, and does performance extend to appropriately assessed material beyond the disclosed cases? Both are useful. Their evidence must remain distinct.
Regression evidence
Previously used cases help check a correction. Record their reuse and preserve the original findings.
Prospective examination
Separately reserved material needs exposure assessment, valid references and support for any comparison with earlier forms.
Unresolved exposure
Unknown history is a limitation to resolve or report. It is not a clean pass or an automatic model failure.
The proposed process considers session memory, retained logs, developer tuning, shared training data, related model versions and customer organizations. New aliases, accounts or wording do not automatically establish freshness; known relationships do not by themselves prove that an item was shared.
Current maturity: an offline exposure-accounting prototype has been implemented and fixture-tested. Integration into live collection is pending. This is a proposed standard admission gate, not a claim that every current service already enforces it. It changes no canonical score, threshold or package entitlement.
The direction applies to repeat engagements across standard catalog services and scoped pilots. Agree regression, fresh examination or both in the statement of work. A bounded correction retest does not automatically include a new examination.
A separate commissioned engagement
Apply these distinctions to a defined agent-control question.
The independent Agent-Control Pilot includes a scoped report, methods annex, evidence inventory and one bounded correction retest. The total professional fee is US$15,000 pre-tax; separately approved compute/API costs and applicable taxes are additional.