A toolbox is not mature because it is large. It is mature when each tool that the organisation relies on is defined, used, verified by someone independent of its operator, and renewed on evidence. This appendix gives the method by which an organisation can assess its toolbox against the tools of this book, find the weakest practice and choose the next one to adopt. The method is the book's own, drawn from the conclusions of the series (Chapters 2, 3, 24 and 25); it is not an external standard. It has six levels, 7 areas and 35 practices. The web tool Toolbox Maturity (webtools/toolbox-maturity.html) implements it directly in the browser, with no installation, and exports the assessment as JSON.
Relation to the earlier maturity models
Architecture as Code defines a maturity model with levels 0 to 5 and a radar tool (1, Nordqvist, 2025); AI Governance defines a maturity model with levels 1 to 5 (2, Nordqvist, 2026). The two scales differ by the Level 0 that the first adds. This method uses 0 to 5 so that a result can be set beside the radar of Architecture as Code; read Level 1 here as Level 1 in AI Governance, and a practice at Level 0 as outside the scale of that model. The models assess a discipline; this method assesses the tools that make the discipline checkable. They are complementary and are not combined into one score.
The six levels
| Level |
Name |
What is true |
| 0 |
Absent |
No tool of this kind is in use. |
| 1 |
Ad hoc |
Individuals use something of the kind. Nothing is written down and nobody owns it. |
| 2 |
Defined |
A template or record from the series is adopted. A named owner exists and the artefact is in version control. |
| 3 |
Operated |
The tool is used on real work, in the pipeline or the process. Its review date is set and kept. |
| 4 |
Verified |
Someone independent of the author and the operator has tested that it works (a seeded fault, a restore test, an exercise), and the evidence is younger than its window. |
| 5 |
Continuously assured |
The checks run automatically, the evidence renews on defined triggers, and adoption, renewal and retirement are decided on measured evidence. |
The evidence-age rule
A claim of Level 4 or 5 needs evidence that is not older than its window, by default 12 months. When the evidence is older, or has no date, the practice counts as Level 3 until it is verified again. This applies the series' rule that nothing is said to be covered, recoverable or competent without saying what was tested, when and with what result. The window is set per practice by the assessor; shorter windows suit stop paths, restore tests and detections.
Scoring
- Rate each practice 0 to 5 and, for Levels 4 and 5, record what was tested, the date and the result.
- Apply the evidence-age rule. The result is the counted level of the practice.
- The score of an area is the mean of the counted levels of its practices, shown to one decimal. The reported level of an area is the whole number of that mean, but never more than the weakest practice of the area plus one, so that strength in one practice does not hide a gap in another.
- Do not add the areas into one number. Show the seven reported levels side by side, as a radar or a table, with the weakest practice of each area named.
- Set a target level (Level 3 is a sensible first target) and list the practices below it, lowest first. Each carries the chapter to read and a search term for the template library.
How to run the assessment
- Scope. Name the organisation or unit, the date and two assessors, one who operates the tools and one who does not.
- Rate with evidence. For each practice the assessor who operates the tool proposes a level and cites the artefact (a file in version control, a record identifier). The second assessor challenges any level that has no artefact.
- Verify a sample. Open at least one artefact per area and check that it is what the rating says.
- Score as above, in the web tool or on paper.
- Plan. Choose at most three practices for the next period, the lowest first, and give each an owner and a date.
- Repeat every six to twelve months, and after any incident that exposed a missing tool. Keep the exported JSON files in version control so that the trend is visible.
The areas and practices
Area A: Architecture and delivery (Chapters 4 to 6)
| Practice |
Statement |
Principal tools |
Chapter |
| A1 |
Architecture decisions are recorded as decision records in version control, with the reasons, the alternatives and a review date. |
Architecture Decision Record (ADR) |
Chapter 4 |
| A2 |
Architecture is described as code (models, diagrams, documentation) and reviewed in the same change as the system it describes. |
Architecture model as code; documentation as code |
Chapter 4 |
| A3 |
Policy is written as code, tested, and enforced as a gate in the delivery pipeline. |
Policy as code; Open Policy Agent and Rego |
Chapter 5 |
| A4 |
Each release produces an evidence manifest (provenance, signatures, bill of materials) that a third party can check. |
Evidence manifest; provenance and signing |
Chapter 5 |
| A5 |
The platform and its AI components are defined as code from a baseline, and drift is detected. |
Infrastructure as code; platform baseline |
Chapter 6 |
Area B: Governing AI (Chapters 7 to 9)
| Practice |
Statement |
Principal tools |
Chapter |
| B1 |
Every AI use is in an inventory with an owner, an intake record and a risk classification. |
AI inventory; intake record; risk tiering |
Chapter 7 |
| B2 |
Risks and exceptions are recorded with an owner, an expiry and a review date; high-risk uses have an impact assessment. |
Risk record; exception record; impact assessment |
Chapter 7 |
| B3 |
Decisions about AI are recorded as AI Decision Records that name who decided. |
AI Decision Record (AIDR) |
Chapter 7 |
| B4 |
Obligations are mapped to controls and evidence once and reused across frameworks (assure once, comply many). |
Control mapping matrix |
Chapter 8 |
| B5 |
Suppliers of AI are assessed for sovereignty and contract terms, and the exit plan has been tested. |
Sovereignty assessment; exit plan |
Chapter 9 |
Area C: Securing and testing (Chapters 10 to 14)
| Practice |
Statement |
Principal tools |
Chapter |
| C1 |
Each AI system has a threat model that uses a named framework and produces tests. |
Threat model; ATT&CK and ATLAS |
Chapter 10 |
| C2 |
Adversarial testing runs only under a signed rules-of-engagement manifest that is enforced outside the tester. |
Rules-of-engagement manifest (RoE) |
Chapter 11 |
| C3 |
Detections are records with an owner, test cases and a review or retirement date. |
Detection record (DET) |
Chapter 12 |
| C4 |
Incidents are recorded to a standard, reporting clocks are configured, and recovery is proved by restore tests whose evidence age is known. |
Incident record; restore-test record |
Chapter 13 |
| C5 |
Coverage is claimed only with evidence (what was tested, when, with what result), and the tester is independent of the tested. |
Coverage record; regression case; purple session record |
Chapter 14 |
Area D: Operating (Chapters 15 to 17)
| Practice |
Statement |
Principal tools |
Chapter |
| D1 |
Services have records with dependencies and blast radius, and objectives with burn-rate alerts that have been tested. |
Service record (SVC); service level objective record (SLO) |
Chapter 15 |
| D2 |
Alert correlation and AI assistance in observation operate within a written policy and a noise budget. |
Correlation policy and noise budget |
Chapter 15 |
| D3 |
Changes are classified, windows and freezes are enforced, and every change has a way back. |
Change class; change record (CHG) |
Chapter 16 |
| D4 |
Runbooks are manifests with defined authority, a tested rollback and a stop path. |
Runbook manifest (RBK) |
Chapter 16 |
| D5 |
Capacity, cost and energy are recorded and steered against budgets, including token and GPU cost. |
Capacity and cost record (CAP); FinOps |
Chapter 17 |
Area E: Agents and autonomy (Chapters 18 to 20)
| Practice |
Statement |
Principal tools |
Chapter |
| E1 |
Every agent has a mandate manifest that names its purpose, actions, data and the accountable person. |
Mandate manifest (MND) |
Chapter 18 |
| E2 |
Delegation is set per action class and level, the autonomy budget is tracked, and changes are decided by a named authority. |
Delegation profile (DLG); autonomy budget |
Chapter 18 |
| E3 |
What verifies an agent is independent of it: an observer manifest and seeded faults. |
Observer manifest (OBS); seeded-fault record |
Chapter 19 |
| E4 |
A stop path outside the agent platform exists, is held by named people and is drilled. |
Stop path; stop drill |
Chapter 19 |
| E5 |
Agents have their own identity, scoped credentials and a permission manifest, and the agent control plane is treated as Tier 0. |
Workload identity; permission manifest |
Chapter 20 |
Area F: People, organisation and society (Chapters 21 to 23)
| Practice |
Statement |
Principal tools |
Chapter |
| F1 |
Roles that use or supervise AI have charters with decision rights. |
Role charter (ROLE) |
Chapter 21 |
| F2 |
Competence is recorded without scoring individuals, and the craft is kept up through practice. |
Competence record (CMP) |
Chapter 21 |
| F3 |
Where AI directs people, a named human decides, decisions are recorded and they can be contested. |
Management decision record (MDR); contestation record (CNT) |
Chapter 22 |
| F4 |
Employee representatives are involved before monitoring or AI-directed work starts, and impact tests are run with them. |
Impact test with representatives |
Chapter 22 |
| F5 |
Claims about threats and the future are graded on an evidence ladder and labelled observable, expected or speculation. |
Evidence record; threat register |
Chapter 23 |
| Practice |
Statement |
Principal tools |
Chapter |
| G1 |
There is an inventory of the tools in use, with kind, owner and the book and chapter that defines each. |
Tool inventory |
Chapter 3 |
| G2 |
Identifier families are used consistently, and one identifier means one thing. |
Identifier families |
Chapter 2 |
| G3 |
Adoption passes a gate with evidence: a pilot, an owner and a review date. |
Adoption gate |
Chapter 3 |
| G4 |
Tools are reviewed on a cadence, and tools that are unused or cannot be verified are retired. |
Retirement criteria; toolbox review |
Chapter 3 |
| G5 |
The toolbox is measured (evidence age, adoption, retirements) and reported to a named forum. |
Toolbox measures |
Chapter 24 |
What to do at each level
| From level |
Next step |
| 0 |
Pick the principal template for this practice in the chapter named, copy it into version control and name an owner. |
| 1 |
Write the practice down: adopt the template, name an owner and put the artefact in version control. |
| 2 |
Use it on real work: attach it to the pipeline or process and set a review date that is kept. |
| 3 |
Have someone independent of the author and the operator test it (seeded fault, restore test or exercise) and record the date and result. |
| 4 |
Automate the check, define the triggers that expire the evidence, and measure it over time. |
| 5 |
Keep the evidence current; review whether the practice is still needed. |
Limits of the method
- It is a self-assessment. The second assessor and the evidence requirement reduce the bias; they do not remove it, and an external review is a separate instrument.
- It measures whether the tools of this book are in place and checked, not whether they are sufficient for a given regulation. Legal obligations are attributed to the instrument and article elsewhere in the book and are framed as architectural implications, not legal advice.
- The practices are a first set drawn from the tools that recur across the ten earlier volumes. An organisation may add practices; it should not delete a practice to raise a score.
- The levels are conclusions of this book. They have not been calibrated against a population of organisations, and no benchmark is claimed.
Sources
- Nordqvist, G. Architecture as Code. (1, Nordqvist, 2025)
- Nordqvist, G. AI Governance. (2, Nordqvist, 2026)