Chapter 7. Risk Classification: Tiering, Calibration, and Gaming

Turning five scores into a tier, and defending that tier against the pressure it will come under.

7.1 Two axes, not a sum

Averaging five dimensions would let one catastrophic factor be diluted by four benign ones. A system with D1 = 4 and low scores elsewhere would average into apparent moderation, which is precisely the failure the tiering model must avoid.

The model uses two axes adapted from risk-matrix practice, with Impact and Control Deficit replacing probability and impact. For AI systems the decision-relevant question is not how likely failure is, but how much human control exists over whether failure becomes consequential.

Impact = average(D1, D4, D5), rounded up.

Control Deficit = average(D2, D3), rounded up.

D2 and D3 are paired deliberately. High autonomy combined with low reversibility is the combination that most sharply removes the enterprise’s ability to intervene before harm lands.

Table 23. Two axes, not a sum

7.2 The safety-override floor

Regardless of what the matrix returns, a system is floored at Tier 3 minimum if either condition holds:

• D1 = 4 — catastrophic consequence if wrong; or
• D2 = 4 and D3 = 4 — fully autonomous and effectively irreversible.

The floor exists to stop averaging from masking a single severe factor. It is a blunt instrument by design, and its bluntness is the point: it does not require judgment to apply.

What the floor does not do. It catches extremes only. It does nothing about the mid-range, where a system scored D1 = 3 that arguably warrants 4 lands two tiers lower with no automatic backstop. Three independent reviewers identified this during the framework’s stress-testing as the live gaming vector, and it remains unresolved. Sections 7.6 to 7.8 are the practical response.

7.3 Worked examples

Example A — Internal document summarizer

An internal tool that summarizes long policy documents for staff. No decisions, no external exposure, internal-confidential inputs.

D1 = 1 (a poor summary wastes minutes), D2 = 1 (advisory), D3 = 1 (nothing leaves), D4 = 2 (departmental), D5 = 2 (internal confidential; generative but low stakes, uncertainty sub-score 3, sensitivity 2 → composite 3).

Impact = avg(1, 2, 3) = 2.0 → 2. Control Deficit = avg(1, 1) = 1.

Matrix: Impact 2, Control Deficit 1 → Tier 1. No override triggered.

Note that D5 was scored 3 on the higher-of-two rule rather than averaged to 2.5. It did not change the tier here, but it would have if D1 or D4 had been one point higher, which is the kind of case the rule exists for.

Example B — Claims triage assistant

The worked example from Chapter 5, now scored fully.

D1 = 3 (misrouting delays a claim and may disadvantage a claimant), D2 = 2 (proposes a routing category, adjuster approves each), D3 = 2 (routing is reversible, though delay is not recoverable), D4 = 3 (all claims, internal users, feeds the claims system of record), D5 = 3 (restricted personal data; generative).

Impact = avg(3, 3, 3) = 3. Control Deficit = avg(2, 2) = 2.

Matrix: Impact 3, Control Deficit 2 → Tier 2. No override.

This result deserves scrutiny, and it illustrates why review matters. Tier 2 means architect self-certification at G2 and owner approval at G3. Is that right for a system touching every claim and handling restricted personal data? The scores are defensible individually. The outcome sits one point of D1 away from Tier 3. This is precisely the boundary case where an independent countersigner earns their place, and where the override-rate instrumentation from §6.3 should be mandatory: if adjusters accept 99% of routings, effective D2 is 3 and the tier moves.

Example C — Autonomous reconciliation agent

An agent that identifies and posts correcting journal entries in a financial ledger without human review below a value threshold.

D1 = 3 (misposting is a financial control issue, correctable but reportable), D2 = 4 (acts without review below threshold), D3 = 3 (a posted entry can be reversed, but the audit trail and any downstream reporting cannot be un-run), D4 = 3 (enterprise financial records, feeds reporting), D5 = 2 (internal financial data, deterministic-ish logic).

Impact = avg(3, 3, 2) = 2.67 → 3. Control Deficit = avg(4, 3) = 3.5 → 4.

Matrix: Impact 3, Control Deficit 4 → Tier 4. Override not triggered, but not needed.

The matrix does the work here. The combination of autonomy and partial irreversibility pushes Control Deficit to 4, and Tier 4 brings AI Governance Body plus Enterprise Sponsor sign-off, mandatory preventive controls, and a pre-approved kill-switch condition. That is a proportionate outcome for an agent posting to a ledger.

Example D — A scoring dispute

The same reconciliation agent, as the delivery team scored it.

D1 = 2 (“entries are reversible, so the consequence is minor”), D2 = 4, D3 = 1 (“a journal reversal takes one transaction”), D4 = 2 (“only the finance team uses it”), D5 = 2.

Impact = avg(2, 2, 2) = 2. Control Deficit = avg(4, 1) = 2.5 → 3.

Matrix: Impact 2, Control Deficit 3 → Tier 2.

Two tiers lower, and every score is arguable in isolation. The errors are instructive. D3 was scored on mechanism reversibility, not consequence: the ledger entry reverses, the audit trail and any interim reporting do not. D4 counted direct users rather than the reach of the financial records the agent writes into. D1 followed from the D3 error.

None of this requires bad faith. It is what happens when anchors are applied loosely by people who also own the delivery date. §7.6 is the control.

7.4 Re-scoring: when the tier must be revisited

Classification is not a one-time event. Six triggers require a re-score.

Table 24. Re-scoring: when the tier must be revisited

Preserve score history instead of overwriting it. A Risk Classification Record that shows a system moved from Tier 3 to Tier 2 six weeks before an incident is a governance artifact of considerable value, and it is destroyed by editing in place.

7.5 Tier semantics: what each tier actually buys

Practitioners find tiers easier to apply when they know what each one costs.

Table 25. Tier semantics: what each tier actually buys

A common and reasonable question is whether four tiers are the right number. The answer in the research is a reasoned judgment instead of a tested one: four is thought to be roughly the finest granularity assessors can apply consistently without significant inter-rater disagreement. That claim is untested. If your own experience shows assessors cannot reliably separate Tier 2 from Tier 3, the honest response is to sharpen the anchors, not to add a tier.

7.6 Independent countersignature

This is the highest-leverage control in IRGF, and the one most likely to be quietly dropped during adoption because it costs a person’s time.

The rule. A D1–D5 score produced by a delivery team is not final until countersigned by the Risk and Compliance Lead, who is independent of delivery. The countersigner may change any score and must record a reason when they do.

Why it matters more than anything else. Unverified self-scoring was the single most confirmed weakness across the framework’s entire adversarial review, converging from multiple independent reviewers. Every tier-scaled control downstream inherits the accuracy of this one number. A framework with excellent gates and wrong tiers governs the wrong things rigorously.

What countersignature is not. It is not a second full assessment. The countersigner reviews the justifications, tests the two or three scores nearest a threshold, and either accepts or adjusts. A thorough countersignature on a routine system takes fifteen minutes.

Where to concentrate it. Not uniformly. Concentrate on boundary cases: any score of 3 where 4 is arguable, any D3 justified on mechanism, not consequence, any D2 of 1 or 2 on a system with D1 ≥ 3, and any case where the matrix result sits one point from a tier change.

7.7 Audit sampling for Tier 1–2

Countersignature covers the initial score. Sampling covers the systems that were scored low and then never looked at again.

[Practice recommendation] Sample 10% of Tier 1–2 self-certifications per quarter, selected with deliberate bias, not at random: weight selection toward systems whose D-scores sit one point below a tier boundary, systems whose owner has multiple low-scored systems, and systems whose function has changed since scoring. Random sampling of a population where most cases are correctly scored wastes the sample on easy cases.

Publish the outcome as a rate, not as individual findings. “12% of sampled Tier 1 systems were re-tiered upward” is a governance signal the organization can act on. Naming teams turns sampling into an enforcement activity and drives scoring underground.

7.8 Recognizing and countering under-tiering

Under-tiering is rarely dishonest. It is the predictable result of asking people with a delivery commitment to assess their own governance burden. Treat it as a systems problem.

Table 26. Recognizing and countering under-tiering

The last row is the most useful portfolio-level diagnostic available. [Practice recommendation] Publish the tier distribution quarterly alongside the incident and near-miss record. If almost everything is Tier 1 and incidents keep arising in Tier 1 systems, the classification model is not describing your estate, and no amount of gate rigor will compensate.