Chapter 11. AI Assurance: S6
Five assurance types, five different owners, and why treating them as one activity produces both over-testing and blind spots.
11.1 Assurance is not one thing
The Core Model holds a single Assurance entity, and that single entity bundles five logically separable activities with different owners, different evidence, and different escalation urgency. Separating them is what allows the framework to escalate a safety finding quickly while routing a value shortfall through ordinary change.
Table 33. Assurance is not one thing
| Assurance type | Question | Owner | Evidence | Feeds |
|---|---|---|---|---|
| Technical AI Evaluation | Does it perform correctly and safely at the output level? | ML engineering, with architecture visibility only | Evaluation results against defined thresholds | G3, G4 |
| Architecture Assurance | Does deployed configuration still match approved architecture? | Architecture, increasingly automated | Conformance output, drift alerts | S8, G4 trigger |
| Risk Assurance | Is the tier still right, and are mapped controls effective? | Risk and Compliance | Control-effectiveness testing, re-scoring | G4 trigger, escalation |
| Compliance Assurance | Does it still conform to policy and regulation? | Risk and Compliance | Compliance checks, external audit evidence | G3, G4, reporting |
| Business Outcome Assurance | Is it delivering the value it was authorized to deliver? | Business or Capability Owner | Outcome metrics against the S1 baseline | Benefits realization, retirement |
11.2 Differential escalation
The five types deliberately do not escalate identically, and this is one of the more useful design choices to explain to stakeholders.
A Technical AI Evaluation failure — a hallucination rate breaching threshold — triggers Risk Assurance re-scoring, which may trigger escalation, which may re-open G4. That path is fast.
A Business Outcome Assurance failure — the system works correctly, complies fully, and simply is not delivering the promised value — does not trigger risk escalation. It routes through ordinary Change or Retirement. Treating a value shortfall with the same urgency as a safety finding misallocates governance attention and, over time, trains people to ignore escalations.
Reserve the fast path for safety and compliance. Route value failures through the normal cycle.
11.3 What architecture governance owns and does not
Technical AI Evaluation belongs to ML engineering. IRGF governs that evaluation gates exist and bind to deployment, not how they are conducted. The framework does not specify benchmark suites, evaluation datasets, or thresholds for hallucination or bias, and should not: those are engineering and domain judgments that change faster than a governance framework can track.
What architecture governance requires is narrower and enforceable:
- An evaluation was run before G3, appropriate to the build type
- Its results are referenced from the AI Assurance Summary
- Its pass criteria were defined before the run, not after
- The evidence chain is deep enough for the tier
Tier depth differs materially here. At Tier 1 the AI Assurance Summary may be a pass or fail reference. At Tier 4 it must carry the full underlying evaluation reference chain, sufficient for someone to reconstruct what was tested and against what threshold.
11.4 Assurance scope by build type
Table 34.
| Build type | Assurance emphasis |
|---|---|
| Traditional ML | Accuracy, bias, explainability, stability over time |
| Foundation model API | Output quality, data leakage, vendor dependency, behavior under model update |
| Generative AI | Hallucination, toxicity, bias, adversarial resilience, prompt injection |
| Retrieval-augmented | Faithfulness to grounding source, stale-knowledge detection, retrieval failure behavior |
| Fine-tuned | Regression against base-model behavior, training-data governance |
| Copilot | Usability of the oversight point, not only technical accuracy |
| Single agent | Adversarial resilience, containment, boundary enforcement under pressure |
| Multi-agent | Emergent behavior, interaction effects, delegation-chain integrity |
The copilot row is easy to skip and important. For a human-paired system, an assurance activity that tests only model accuracy has tested half the system. The oversight interaction is an architectural element with its own failure mode — an approve button that is always pressed provides no assurance regardless of how accurate the model is (Chapter 6, §6.3).