AI risk register

The training set under-represents the population the system serves (regions, languages, age groups, minority classes), so the model performs worse for them and the gap is invisible in headline accuracy.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

People in an under-represented group receive materially worse decisions or service

Key controls

A.7.4 data quality requirements; A.6.2.4 validation by sub-group; A.7.3 acquisition records

Owner and indicator

Data owner. Share of production inputs outside the training distribution; error gap between best and worst group

Regulatory hooks

AI Act Art 10(2)-(4), Art 15

Arises at

design and development; role exposed: both; harmed: individuals; organisation

Sources

MIT 1.3; ISO 42001 C.2.3; NIST AI RMF Map 2.3

Labels used to train or evaluate the model are wrong, inconsistent between annotators, or encode past decisions that were themselves flawed; the model learns the error and the evaluation certifies it.

Example scores: likelihood 4, impact 3 (customer and operational), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 3 × 2 Moderate 6.

Harm to score (per decision)

Decisions built on flawed labels go wrong and have to be reworked or reversed

Key controls

A.7.4; A.7.6 data preparation criteria; A.6.2.4 independent test set

Owner and indicator

Data owner. Inter-annotator agreement; error rate on an audited label sample

Regulatory hooks

AI Act Art 10(2)(c),(g)

Arises at

design and development; role exposed: provider; harmed: organisation; individuals

Sources

ISO 42001 C.3.4; NIST AI RMF Measure 2.1

Data scraped, bought or re-used for training without a lawful basis, licence or the purpose it was collected for; the model cannot lawfully be used and may have to be retrained or withdrawn.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A regulator or rights-holder challenge forces the model to be retrained or withdrawn

Key controls

A.7.3 acquisition records; A.7.5 provenance; DPIA; A.2.3 alignment with the privacy policy

Owner and indicator

Data owner (DPO advises). Share of training sources with a documented basis and licence

Regulatory hooks

GDPR Art 5(1)(b), 6, 14; AI Act Art 10(2)(b), 53(1)(c); EDPB Opinion 28/2024

Arises at

inception; design and development; role exposed: provider; harmed: organisation; individuals

Sources

EDPB Opinion 28/2024; Garante decision on ChatGPT, December 2024

The deployer cannot say what the vendor's model was trained on; it cannot answer a regulator, a court or a customer about bias, copyright or personal data in the training set.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Low; target residual 3 × 2 Moderate 6.

Harm to score (per year)

The organisation cannot answer a regulator, court or customer about what the bought model was trained on

Key controls

A.10.3 supplier process; A.6.2.7 technical documentation required by contract; A.7.5

Owner and indicator

Procurement. Share of AI suppliers providing the training-data summary and model documentation

Regulatory hooks

AI Act Art 13 (high-risk systems); Art 25(4) by written agreement for high-risk components; Art 53(1)(d) for general-purpose models; otherwise contractual

Arises at

inception; deployment; role exposed: deployer; harmed: organisation

Sources

ISO 42001 C.3.2

The data the system sees in production moves away from the data it was built on (new products, demographics, pricing, language), so outputs degrade while monitoring shows green.

Example scores: likelihood 3, impact 3 (customer and operational), inherent Moderate 9; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

Drift degrades outputs enough to disrupt decisions or service before monitoring catches it

Key controls

A.6.2.6 operation and monitoring; A.7.4; drift indicators with thresholds

Owner and indicator

System owner. Population stability index; feature-distribution alerts per month

Regulatory hooks

AI Act Art 26(4), 26(5), 72

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

NIST AI RMF Measure 2.6; ISO 42001 C.3.4

Upstream fields arrive empty, mis-coded or deliberately altered by users gaming a scoring model, and the model produces confident outputs on bad inputs.

Example scores: likelihood 4, impact 3 (customer and operational), inherent High 12; vulnerability 3, speed of onset 4, data quality Medium; target residual 3 × 2 Moderate 6.

Harm to score (per decision)

Bad or gamed inputs produce confident outputs that are acted on

Key controls

A.7.4 input validation; A.6.2.6; A.9.4 intended use

Owner and indicator

System owner. Input validation rejection rate; anomaly alerts

Regulatory hooks

AI Act Art 26(4), 15(4)

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation; individuals

Sources

OWASP AI Exchange; ISO 42001 C.3.4

Prompts, documents, transcripts and features are kept longer than needed or copied into vendor systems, widening the blast radius of any breach and breaching minimisation.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

Over-retained or copied AI data comes to light through a complaint, audit or breach

Key controls

A.7.2 data management processes; A.7.6; retention schedule; vendor retention terms

Owner and indicator

Data owner (DPO advises). Volume of AI-pipeline data beyond its retention date; vendor retention terms reviewed

Regulatory hooks

GDPR Art 5(1)(c),(e), 32

Arises at

design and development; operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

EDPB LLM privacy report 2025

Engineers, contractors or the model itself can reach data sets holding special-category or confidential data without need; a quiet route to leakage.

Example scores: likelihood 2, impact 4 (regulatory and legal), inherent Moderate 8; vulnerability 3, speed of onset 4, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

Unneeded access to a data store leads to leakage of special-category or confidential data

Key controls

A.7.2; A.4.3 data resources documented; ISO/IEC 27001 access control

Owner and indicator

CISO. Privileged-access reviews completed on AI data stores

Regulatory hooks

GDPR Art 32; NIS2 Art 21

Arises at

design and development; operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

ISO/IEC 27001

A generative model states things that are false but fluent: invented case law, policy terms, product facts, numbers; staff or customers act on them. False statements about named people can also defame them.

Example scores: likelihood 3, impact 4 (reputational), inherent High 12; vulnerability 4, speed of onset 5, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per output)

A fabricated fact, citation or figure reaches a customer, court or regulator and becomes public

Key controls

A.6.2.4 factuality evaluation; retrieval grounding; A.9.3 human review for consequential outputs; A.8.2 user information on limits

Owner and indicator

Product owner. Fabrication rate on a fixed test set; customer correction rate

Regulatory hooks

AI Act Art 50(1), 26(2); consumer protection law

Arises at

operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

Mata v Avianca (US, 2023); Air Canada tribunal (Canada, 2024); Deloitte Australia report refund (2025); NIST AI 600-1 confabulation

A classifier that is accurate on average is wrong for the cases near its threshold, which are exactly the cases where a decision matters (borderline credit, triage, fraud flags). Without a reliable confidence estimate on each output, nobody can tell which cases those are.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 3, speed of onset 3, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

Wrong decisions on borderline cases cause serious harm to the people decided about

Key controls

A.6.2.4 threshold setting with the cost of each error type; calibration; A.9.3 human review band around the threshold

Owner and indicator

Model owner. Error rate inside the decision band; override rate

Regulatory hooks

AI Act Art 9(8), 14

Arises at

operation and monitoring; role exposed: both; harmed: individuals

Sources

Epic sepsis model external validation (JAMA Internal Medicine, 2021)

Performance collapses on inputs unlike the training set (rare events, new formats, adversarial phrasing); the failure is total rather than graceful. Customers can also be left without service when the system refuses legitimate requests or answers with nonsense.

Example scores: likelihood 3, impact 4 (customer and operational), inherent High 12; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

An unfamiliar input causes a total failure that stops or corrupts an operation

Key controls

A.6.2.4 stress and edge-case testing; A.6.2.6; fallback to a human path

Owner and indicator

Validator. Out-of-distribution detection rate; fallback invocations

Regulatory hooks

AI Act Art 15(1),(4)

Arises at

verification and validation; operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

ISO 42001 C.3.1; ISO/IEC TR 24029-1

The same input gives different outputs (non-determinism, sampling temperature, model routing), so decisions are not reproducible and cannot be audited or defended.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 3, speed of onset 3, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per decision)

A challenged decision cannot be reproduced or defended

Key controls

A.6.2.4 reproducibility tests; A.6.2.8 logging of model version and parameters

Owner and indicator

Model owner. Output variance on a fixed test set between runs

Regulatory hooks

AI Act Art 12, 13

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

NIST AI 600-1

The provider replaces or retrains the model behind the API; behaviour changes overnight; validations, prompts and thresholds set for the old model no longer hold.

Example scores: likelihood 4, impact 4 (customer and operational), inherent Critical 16; vulnerability 5, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6. Material by the method's rule.

Harm to score (per year)

A vendor model change degrades outputs at volume before anyone notices

Key controls

A.10.3 contractual notice of model changes; A.6.2.6 regression suite run on every version; version pinning; A.6.2.8

Owner and indicator

System owner. Vendor version changes per quarter; regression pass rate per version

Regulatory hooks

AI Act Art 25(1)(b), 26(5); DORA Art 30

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation; individuals

Sources

ISO 42001 C.3.7

The relationship the model learned stops holding: what predicted default, churn or fraud last year no longer does; accuracy decays while the model keeps scoring.

Example scores: likelihood 3, impact 3 (financial), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

Decayed predictions cause losses, such as mispriced risk or missed fraud, before the model is recalibrated

Key controls

A.6.2.6 performance monitoring against ground truth; scheduled re-validation (clause 8.2)

Owner and indicator

Model owner. Accuracy on a recent labelled sample against the validation baseline

Regulatory hooks

AI Act Art 9(2)(c), 72

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

Zillow Offers write-down (2021)

The system goes live without agreed measures of acceptable performance, so nobody can say it is failing, and the Act's testing duty is unmet.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 5, speed of onset 2, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A failing system goes undetected and an auditor or regulator finds no thresholds were defined

Key controls

A.6.2.4 verification and validation measures with criteria; A.6.2.5 deployment gate

Owner and indicator

Validator. Share of live systems with signed-off metrics and thresholds

Regulatory hooks

AI Act Art 9(6)-(8), 15(2)

Arises at

verification and validation; deployment; role exposed: provider; harmed: organisation

Sources

ISO 42001 A.6.2.4

Tested on a public benchmark or the vendor's demonstration data, not on the organisation's own cases, so test results do not predict production. Benchmarks the model saw in training, and models that behave differently when they detect a test, overstate performance in the same way.

Example scores: likelihood 4, impact 3 (customer and operational), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

Production performance falls short of what the tests showed and the shortfall disrupts the service

Key controls

A.6.2.4 test on representative held-out data of the organisation's own; A.7.4

Owner and indicator

Validator. Share of evaluations run on the organisation's own data

Regulatory hooks

AI Act Art 15

Arises at

verification and validation; role exposed: both; harmed: organisation

Sources

NIST AI RMF Measure 2

A pilot on easy cases persuades the business the system can do more than it can; it is rolled out to harder cases and fails.

Example scores: likelihood 3, impact 3 (financial), inherent Moderate 9; vulnerability 4, speed of onset 3, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

A rollout to harder cases fails and the investment is written off

Key controls

A.6.2.2 requirements and specification; staged deployment (A.6.2.5); A.10.4

Owner and indicator

Business owner. Gap between pilot and production performance

Regulatory hooks

AI Act Art 26(1)

Arises at

inception; deployment; role exposed: deployer; harmed: organisation; individuals

Sources

Amazon recruiting tool (2018); McDonald's drive-through order AI withdrawn (2024)

A system allowed to call tools, send messages, move money or change records acts on a misread instruction or a planted one; the damage is done before anyone looks.

Example scores: likelihood 2, impact 5 (financial), inherent High 10; vulnerability 5, speed of onset 5, data quality Low; target residual 1 × 4 Low 4. Material by the method's rule.

Harm to score (per decision)

An agent takes an action nobody intended that moves money or changes records, with severe loss

Key controls

A.9.3 human approval for consequential actions; least-privilege tool access; A.6.2.8 action logs; a kill switch

Owner and indicator

System owner. Actions executed without approval; blocked-action rate

Regulatory hooks

AI Act Art 14, 26(2)

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

Singapore Model AI Governance Framework for Agentic AI (2026); OWASP LLM06

Two or more automated systems, the organisation's own or other parties', react to each other: pricing or trading algorithms align or spiral, agents miscoordinate or loop, and one failure cascades through connected systems faster than people can intervene.

Example scores: likelihood 2, impact 4 (financial), inherent Moderate 8; vulnerability 4, speed of onset 5, data quality Low; target residual 1 × 3 Low 3.

Harm to score (per year)

Interaction between automated systems causes a market, pricing or operational incident with material loss

Key controls

A.6.2.6 monitoring for interaction effects; A.9.3 circuit breakers, kill switches and human override; A.6.2.4 testing against other agents in simulation; A.10.3 interfaces with suppliers' agents documented

Owner and indicator

Business owner of the automated process. Circuit-breaker trips; unexplained correlated actions between systems

Regulatory hooks

MiFID II Art 17 (algorithmic trading); TFEU Art 101; DORA Art 6

Arises at

operation and monitoring; role exposed: both; harmed: organisation; customers

Sources

MIT 7.6 (V4 rows 63.01-63.10, 24.09.05, 56.11.00); the Flash Crash of 6 May 2010; OWASP Top 10 for Agentic Applications ASI07, ASI08

The objective the system was given (clicks, handling time, approval rate, a reward score) diverges from what the organisation wants; the system games it, and the metric improves while the outcome gets worse.

Example scores: likelihood 3, impact 3 (customer and operational), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

Optimising the proxy causes customer detriment or loss before the divergence is noticed

Key controls

A.6.1.2 objectives defined against outcomes, not proxies; A.6.2.4 testing against an independent outcome measure; A.9.3 periodic review of what the metric rewards

Owner and indicator

Business owner. Gap between the optimised metric and an independent outcome measure

Regulatory hooks

AI Act Art 9(2)(a); Consumer Protection Code

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation; customers

Sources

MIT 7.1 (V4 rows 34.01, 35.04, 53.01, 54.03.01); Wells Fargo sales-target accounts (2016) as the human analogue

A bug in feature code, a wrong join in the data pipeline, a mis-set parameter or an unsuitable model choice makes the system wrong in a way testing did not catch; the error is systematic rather than random.

Example scores: likelihood 3, impact 4 (customer and operational), inherent High 12; vulnerability 3, speed of onset 3, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A defect produces systematically wrong outputs that are acted on before it is found

Key controls

A.6.1.3 processes for responsible design and development; A.6.2.3 design documentation and review; A.6.2.4 independent validation including code and data lineage; A.6.2.5 release controls

Owner and indicator

Model owner; engineering lead. Defects found after release per quarter; failed reconciliations of inputs to source

Regulatory hooks

AI Act Art 15(1), 17(1)

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation; customers

Sources

MIT 7.3 (V4 rows 15.01.03, 15.01.06, 15.01.07, 19.01.02, 40.03.00); Knight Capital trading loss (August 2012); Post Office Horizon

A system that moves, controls or advises on something physical (a vehicle, robot, medical device, building system or production line) fails or is misused and hurts a person or damages property.

Example scores: likelihood 2, impact 5 (harm to individuals), inherent High 10; vulnerability 3, speed of onset 5, data quality Medium; target residual 1 × 4 Low 4. Material by the method's rule.

Harm to score (per year)

An AI-controlled physical system injures someone or causes serious damage

Key controls

Safety case and hazard analysis under the sector standard; A.6.2.4 testing in the operating environment; A.9.3 human override and safe stop; A.8.4 incident communication

Owner and indicator

Product safety owner. Safety incidents and near misses per operating hour

Regulatory hooks

AI Act Art 6(1), Annex I (from 2 August 2028); Machinery Regulation (EU) 2023/1230; Medical Device Regulation (EU) 2017/745; Directive (EU) 2024/2853

Arises at

all stages; role exposed: both; harmed: individuals; organisation

Sources

MIT V4 rows 58.02, 66.11; Uber test vehicle fatality, Tempe (March 2018); Cruise (October 2023)

Screening, ranking or monitoring tools penalise applicants or staff by sex, age, disability, ethnicity or proxies of them; claims, regulator action and reputational damage follow.

Example scores: likelihood 3, impact 5 (regulatory and legal), inherent Critical 15; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 4 Moderate 8. Material by the method's rule.

Harm to score (per decision)

A claim or regulator action alleges that the hiring or worker tool discriminated

Key controls

A.5.4 impact assessment on individuals; A.6.2.4 bias testing by group before and after deployment; A.9.3

Owner and indicator

HR and fairness lead. Selection-rate ratios by group; complaints alleging bias

Regulatory hooks

AI Act Annex III(4), Art 10(2)(f), 26(7); Employment Equality Acts; NYC Local Law 144; Illinois HB 3773

Arises at

all stages; role exposed: both; harmed: individuals

Sources

Amazon recruiting tool (2018); iTutorGroup EEOC settlement (2023); Mobley v Workday (US, ongoing)

Creditworthiness or pricing models disadvantage protected groups directly or through proxies (postcode, device, name).

Example scores: likelihood 3, impact 5 (regulatory and legal), inherent Critical 15; vulnerability 3, speed of onset 2, data quality Medium; target residual 2 × 4 Moderate 8. Material by the method's rule.

Harm to score (per decision)

A regulator finding or claims that a credit or pricing model disadvantages a protected group

Key controls

A.5.4; fairness metrics with thresholds; adverse-action explanations; A.9.3

Owner and indicator

Model owner and fairness lead. Approval-rate and price gaps by group after controlling for risk

Regulatory hooks

AI Act Annex III(5)(b),(c), Art 27; Equal Status Acts; Consumer Protection Code

Arises at

all stages; role exposed: both; harmed: individuals

Sources

Apple Card inquiry by New York DFS (2019-2021)

Eligibility, fraud-risk or triage models in public bodies single out groups (nationality, ethnicity, low income) for scrutiny or denial.

Example scores: likelihood 3, impact 5 (harm to individuals), inherent Critical 15; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 4 Moderate 8. Material by the method's rule.

Harm to score (per decision)

A group is wrongly singled out for scrutiny or denied a public service

Key controls

A.5.4, A.5.5; fundamental rights impact assessment (Art 27) before use; human decision on adverse outcomes; A.8.3 complaints channel

Owner and indicator

Service owner. Scrutiny rates by group; overturn rate on appeal

Regulatory hooks

AI Act Annex III(5)(a), Art 27; GDPR Art 22

Arises at

all stages; role exposed: deployer; harmed: individuals; groups

Sources

Dutch childcare benefits scandal (2021); Robodebt Royal Commission (2023); SyRI judgment (The Hague, 2020)

Face, voice, language or medical models perform worse for darker skin, accents, dialects, women or older people; the errors fall on those groups.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

A group suffers serious harm from the system's higher error rate for them

Key controls

A.6.2.4 disaggregated evaluation; A.7.4 representative data; A.8.2 limits disclosed to users

Owner and indicator

Validator. Accuracy gap between the best and worst performing group

Regulatory hooks

AI Act Art 10, 15, 5(1)(g)

Arises at

verification and validation; operation and monitoring; role exposed: both; harmed: individuals

Sources

FTC order against Rite Aid (2023); Gender Shades study (2018)

Innocuous features stand in for protected ones, and the model's own decisions generate the next training data (more scrutiny, fewer loans), entrenching the pattern.

Example scores: likelihood 2, impact 4 (harm to individuals), inherent Moderate 8; vulnerability 4, speed of onset 1, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

A proxy-driven pattern entrenches and causes serious harm to a group

Key controls

Feature review for proxies (A.7.6); A.6.2.4; periodic re-identification (8.2); A.5.5 societal impact

Owner and indicator

Fairness lead. Proxy-correlation audit results; drift in group outcomes across cycles

Regulatory hooks

AI Act Art 10(2)(f), 5(1)(c)

Arises at

design and development; operation and monitoring; role exposed: both; harmed: individuals; groups

Sources

ISO/IEC TR 24027

A chatbot or content tool produces abusive, sexual, extremist or demeaning material, or is manipulated into it, in the organisation's name.

Example scores: likelihood 4, impact 3 (reputational), inherent High 12; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per output)

A toxic or demeaning output in the organisation's name is shared publicly

Key controls

Content filters and red-teaming (A.6.2.4); A.9.4 intended use; monitoring and takedown (A.6.2.6); A.8.4 incident communication

Owner and indicator

Product owner. Flagged-output rate; time to takedown

Regulatory hooks

AI Act Art 50; Digital Services Act; UK Online Safety Act

Arises at

operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

Microsoft Tay (2016); DPD chatbot (2024)

Text, images or voice produced by the organisation's system repeat stereotypes, leave groups out or portray them falsely (in marketing, recruitment material or customer answers), although no single decision is discriminatory.

Example scores: likelihood 3, impact 3 (reputational), inherent Moderate 9; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per output)

Stereotyped or exclusionary content published under the organisation's name draws complaints or public criticism

Key controls

A.6.2.4 testing of generated content for representation by group; A.9.3 review of content before publication; A.5.4 impact assessment on affected groups

Owner and indicator

Content owner; fairness lead. Share of sampled outputs failing the representation checklist; complaints about stereotyping

Regulatory hooks

Equal Status Acts 2000-2018; Employment Equality Acts (recruitment material); AI Act Art 50

Arises at

operation and monitoring; role exposed: both; harmed: groups; organisation

Sources

MIT 1.1 (V4 rows 04.02, 11.01, 16.01); Google suspended Gemini image generation of people (February 2024)

Customers, applicants or citizens are not informed that a decision was made or assisted by AI, so they cannot question it; where the Act's or the GDPR's notice duties apply, that is a breach.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 2, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per person)

A complaint or regulator finding that people were not told AI was used on them

Key controls

A.8.2 user information; A.8.5 obligations to interested parties; privacy notices; Art 26(11) notice

Owner and indicator

Service owner. Share of AI-assisted decisions carrying a notice; complaints citing no notice

Regulatory hooks

AI Act Art 26(11), 50; GDPR Art 13(2)(f), 22

Arises at

deployment; operation and monitoring; role exposed: deployer; harmed: individuals

Sources

ISO 42001 A.8

A person refused credit, a job, housing or a benefit cannot get a reason a human can act on, nor a human review; the organisation cannot defend the decision either.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per person)

A refused person's challenge or complaint succeeds because no explanation or review was available

Key controls

Explanation capability specified (A.6.2.2); A.9.3 human review with authority to override; A.8.3 reporting channel

Owner and indicator

Service owner. Explanation requests; outcomes of human review

Regulatory hooks

AI Act Art 86, 13; GDPR Art 22(3)

Arises at

operation and monitoring; role exposed: deployer; harmed: individuals

Sources

Apple Card inquiry (2019-2021)

Nobody inside can say which features drive the output, so errors cannot be diagnosed, bias cannot be traced and the regulator cannot be answered.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

The organisation cannot answer a regulator or diagnose an error when asked

Key controls

A.6.2.3 design documentation; interpretability methods in validation (A.6.2.4); A.6.2.7 technical documentation

Owner and indicator

Model owner. Documentation completeness score per system

Regulatory hooks

AI Act Art 11, 13

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation

Sources

ISO 42001 C.3.2

The supplier refuses or cannot provide model cards, evaluation results, training-data summaries or incident history; the deployer carries the risk blind.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality High; target residual 2 × 2 Low 4.

Harm to score (per year)

The deployer cannot answer a regulator, auditor or customer about the vendor's model

Key controls

A.10.3 supplier requirements; A.6.2.7 documentation as a contract condition; Art 13 and 25 information rights

Owner and indicator

Procurement. Share of AI contracts with documentation and notification clauses

Regulatory hooks

AI Act Art 13 (high-risk systems); Art 25(4) by written agreement for high-risk components; Art 53 for general-purpose models; otherwise contractual

Arises at

inception; operation and monitoring; role exposed: deployer; harmed: organisation

Sources

ISO 42001 A.10

The model belongs to IT or to the vendor; when it fails nobody is accountable, nobody re-assesses it, and the inventory line goes stale.

Example scores: likelihood 4, impact 3 (customer and operational), inherent High 12; vulnerability 5, speed of onset 2, data quality High; target residual 2 × 2 Low 4.

Harm to score (per year)

A failure goes unaddressed because nobody owns the system

Key controls

A.3.2 roles and responsibilities; A.10.2 allocation with third parties; inventory with a named owner

Owner and indicator

AI officer. Share of inventory lines with a named, current owner

Regulatory hooks

AI Act Art 26(2), 25; GDPR Art 5(2) accountability

Arises at

all stages; role exposed: both; harmed: organisation

Sources

ISO 42001 C.2.1

Staff attribute their own decisions to the system, or the system's output is treated as a decision no human took; accountability dissolves.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 4, speed of onset 2, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

A harmful decision is not corrected because nobody accepts it as theirs

Key controls

A.9.3 oversight objectives with decision rights; A.3.3 reporting of concerns; training (7.2)

Owner and indicator

Business owner. Override and challenge rates; complaint analysis

Regulatory hooks

GDPR Art 22; AI Act Art 14, 26(2)

Arises at

operation and monitoring; role exposed: deployer; harmed: individuals; organisation

Sources

Post Office Horizon scandal (UK); Robodebt (Australia)

The system does not record inputs, version, parameters and outputs in a way that lets anyone reconstruct what happened; logging duties are unmet and incidents cannot be investigated.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 2, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per year)

An incident or complaint cannot be investigated, or a decision reconstructed, when required

Key controls

A.6.2.8 event logs; A.5.3 retention of assessments; log retention policy (six months minimum for deployers)

Owner and indicator

Records manager. Share of systems with reconstructable logs; log-retention compliance

Regulatory hooks

AI Act Art 12, 19, 26(6)

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

ISO 42001 A.6.2.8

Technical files, risk assessments and user instructions describe a version that no longer runs; the record is a liability rather than a defence.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

An auditor or regulator relies on documentation that describes a superseded version

Key controls

7.5 control of documented information; change-triggered re-documentation (6.3); A.6.2.7

Owner and indicator

System owner. Documentation age against the last model change

Regulatory hooks

AI Act Art 11, 72

Arises at

operation and monitoring; re-evaluation; role exposed: both; harmed: organisation

Sources

ISO 42001 7.5

Reviewers approve the machine's recommendation at a rate that shows no review is taking place; errors flow through unchecked while the oversight box is ticked.

Example scores: likelihood 4, impact 4 (harm to individuals), inherent Critical 16; vulnerability 4, speed of onset 2, data quality Medium; target residual 3 × 3 Moderate 9. Material by the method's rule.

Harm to score (per decision)

Errors approved by reviewers reach people and cause them serious harm

Key controls

A.9.3 oversight design with sampling and blind re-checks; training on failure modes; override-rate monitoring; Art 14(4)(b) awareness

Owner and indicator

Business owner. Override rate; agreement with the model on a blind sample

Regulatory hooks

AI Act Art 14(4)(b), 26(2)

Arises at

operation and monitoring; role exposed: deployer; harmed: individuals; organisation

Sources

Epic sepsis model (2021); Post Office Horizon (UK)

Lawyers, clinicians, advisers and engineers submit AI-produced work without checking; fabrications and errors reach clients, courts and regulators.

Example scores: likelihood 3, impact 4 (reputational), inherent High 12; vulnerability 4, speed of onset 4, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per output)

An unchecked AI-produced error reaches a client, court or regulator and becomes public

Key controls

Acceptable-use policy (A.2.2, A.9.2); mandatory verification of external outputs; training (7.2, Art 4)

Owner and indicator

Business owner. Sampled error rate in AI-assisted deliverables

Regulatory hooks

AI Act Art 4; professional codes of conduct

Arises at

operation and monitoring; role exposed: deployer; harmed: individuals; organisation

Sources

Mata v Avianca (2023); Deloitte Australia (2025)

A tool procured for drafting is used for decisions; a model built for one population is run on another; a triage aid becomes the decider.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

A tool used beyond its purpose causes serious harm to the people it is used on

Key controls

A.9.4 intended use enforced; A.9.2 processes for use; A.6.2.5 deployment plan; use-case inventory

Owner and indicator

Business owner. Uses in production not in the inventory; mismatches found in review

Regulatory hooks

AI Act Art 26(1), 25(1)(c)

Arises at

operation and monitoring; role exposed: deployer; harmed: individuals; organisation

Sources

ISO 42001 A.9.4

Customers or staff use the system for ends it was not designed for (medical advice from a sales bot, harassment through an image tool); the organisation answers for what it should have foreseen.

Example scores: likelihood 3, impact 4 (reputational), inherent High 12; vulnerability 4, speed of onset 4, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

Users misuse the system in a way that becomes public and is blamed on the organisation

Key controls

A.5.2 impact assessment covering misuse (ISO/IEC 42005 6.8.3); guardrails; A.8.2 user information; monitoring

Owner and indicator

Product owner. Blocked misuse attempts; incident reports

Regulatory hooks

AI Act Art 9(2)(b), 13(3)(b)

Arises at

design and development; operation and monitoring; role exposed: provider; harmed: individuals; organisation

Sources

New York City MyCity chatbot (2024)

The people assigned to oversee the system do not understand its limits, its failure modes or their authority; oversight exists on paper only.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

An overseer fails to catch or stop an error that seriously harms someone

Key controls

7.2 competence; A.4.6 human resources documented; Art 4 literacy programme; role-specific training

Owner and indicator

HR and business owner. Share of overseers trained and assessed; literacy coverage

Regulatory hooks

AI Act Art 4, 14(4), 26(2)

Arises at

deployment; operation and monitoring; role exposed: deployer; harmed: individuals; organisation

Sources

ISO 42001 7.2

After years of the machine doing the task, nobody can do it by hand, check the output or run the service when the system fails.

Example scores: likelihood 2, impact 4 (customer and operational), inherent Moderate 8; vulnerability 4, speed of onset 1, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per year)

When the system fails, nobody can run the service or check the output by hand

Key controls

A.4.6; rotation and manual-run exercises; continuity plans (A.6.2.6)

Owner and indicator

HR. Manual-capability drills passed; key-person dependence

Regulatory hooks

None specific

Arises at

operation and monitoring; retirement; role exposed: deployer; harmed: organisation

Sources

ISO 42001 C.2.2

The interface gives the reviewer no way to stop or change the output, or the volume makes review impossible (alert fatigue); the human in the loop is decoration.

Example scores: likelihood 3, impact 4 (harm to individuals), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

Reviewers cannot stop an error at volume and it seriously harms people

Key controls

A.9.3 override authority and tooling; workload design; alert thresholds tuned; Art 14(3) measures built in by the provider

Owner and indicator

Business owner. Alerts per reviewer per day; actions taken on alerts

Regulatory hooks

AI Act Art 14(3),(4)

Arises at

design and development; operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

Epic sepsis model alert burden (2021)

Conversational systems nudge, deceive or form attachments with users (children, vulnerable adults, lonely customers) in ways that damage them or breach the Act's prohibitions. Sycophancy, telling users what they want to hear, is one form.

Example scores: likelihood 2, impact 5 (harm to individuals), inherent High 10; vulnerability 4, speed of onset 2, data quality Low; target residual 1 × 4 Low 4. Material by the method's rule.

Harm to score (per person)

A conversational system damages a vulnerable user or breaches a prohibition

Key controls

A.5.4 impact on vulnerable groups; design reviews; A.9.3; age assurance; Art 5 screening at classification

Owner and indicator

Product owner. Vulnerable-user interaction metrics; complaints

Regulatory hooks

AI Act Art 5(1)(a),(b), 50(1); California SB 243

Arises at

design and development; operation and monitoring; role exposed: provider; harmed: individuals

Sources

Garante fine on Replika (2025)

A chatbot or assistant gives a user workable instructions or encouragement for crime, violence, weapons, fraud or self-harm, in the organisation's name or through its product.

Example scores: likelihood 2, impact 5 (harm to individuals), inherent High 10; vulnerability 4, speed of onset 5, data quality Medium; target residual 1 × 4 Low 4. Material by the method's rule.

Harm to score (per output)

Help given by the system contributes to a crime, an injury or self-harm, and it becomes known

Key controls

A.6.2.4 red-team testing for harmful assistance before release; A.9.4 intended-use limits enforced by content filters; A.6.2.6 monitoring of flagged conversations; A.8.2 user information and crisis signposting

Owner and indicator

Product owner (trust and safety). Share of red-team prompts for harmful assistance that are answered; flagged conversations per 10,000

Regulatory hooks

AI Act Art 9(2)(b), 55(1)(b); Directive (EU) 2024/2853

Arises at

operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

MIT 1.2, 4.3, 7.2 (V4 rows 23.01-23.06, 57.01-57.02); NEDA "Tessa" chatbot withdrawn (June 2023); Garcia v Character Technologies (US, filed October 2024)

A user crafts inputs that override system instructions, extract the system prompt, disable safety behaviour or make the model act against policy.

Example scores: likelihood 4, impact 3 (reputational), inherent High 12; vulnerability 4, speed of onset 5, data quality High; target residual 3 × 2 Moderate 6.

Harm to score (per output)

A jailbreak makes a public system produce embarrassing output or reveal its instructions, and it becomes public

Key controls

Input and output filtering; privilege separation between the model and its tools; red-teaming (A.6.2.4); monitoring (A.6.2.6)

Owner and indicator

CISO. Jailbreak success rate in testing; detected injection attempts

Regulatory hooks

AI Act Art 15(5)

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

OWASP LLM01; Chevrolet dealership chatbot (2023)

Instructions hidden in a web page, PDF, email or ticket are read by the model and executed (exfiltrate data, send a message, approve a request) without any user intending it.

Example scores: likelihood 3, impact 4 (financial), inherent High 12; vulnerability 5, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

An injected instruction makes an assistant leak data or take an action that causes a major loss

Key controls

Treat all retrieved content as untrusted; no tool actions on unverified instructions; A.9.3 approval for consequential actions; allow-lists

Owner and indicator

CISO. Injection detections in retrieved content; blocked actions

Regulatory hooks

AI Act Art 15(5)

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

OWASP LLM01; NIST AI 600-1

An attacker or a careless pipeline inserts data that plants a backdoor, a bias or a trigger phrase; the model behaves normally until the trigger appears.

Example scores: likelihood 2, impact 4 (customer and operational), inherent Moderate 8; vulnerability 4, speed of onset 2, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A planted backdoor or trigger corrupts the model's behaviour in production

Key controls

A.7.3 acquisition controls; A.7.5 provenance; integrity checks; A.6.2.4 testing for triggers; no uncontrolled continuous learning

Owner and indicator

CISO and model owner. Provenance coverage of training sources

Regulatory hooks

AI Act Art 15(5)

Arises at

design and development; operation and monitoring; role exposed: provider; harmed: organisation; individuals

Sources

OWASP LLM04

Documents injected into a knowledge base or vector store change what the assistant tells everyone (wrong policy, malicious link) with no model change at all.

Example scores: likelihood 3, impact 3 (customer and operational), inherent Moderate 9; vulnerability 4, speed of onset 4, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

Poisoned documents change what the assistant tells users

Key controls

Write controls on knowledge bases; content review; provenance tags; monitoring

Owner and indicator

CISO. Unauthorised changes to the corpus; share of stale sources

Regulatory hooks

AI Act Art 15(5)

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation; individuals

Sources

OWASP LLM08

Repeated queries reconstruct a proprietary model or its training data; a competitor or attacker gets the asset, or the stolen copy is used to find weaknesses.

Example scores: likelihood 2, impact 3 (financial), inherent Moderate 6; vulnerability 3, speed of onset 3, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

A competitor or attacker obtains a copy of the model or its training data

Key controls

Rate limiting; query monitoring; watermarking; access control

Owner and indicator

CISO. Anomalous query volumes per client

Regulatory hooks

AI Act Art 15(5); trade secrets law

Arises at

operation and monitoring; role exposed: provider; harmed: organisation

Sources

OWASP Top 10 for LLM Applications (model theft, 2023 edition)

An attacker determines that a specific person was in the training set, or infers sensitive attributes from the model's responses.

Example scores: likelihood 2, impact 4 (regulatory and legal), inherent Moderate 8; vulnerability 4, speed of onset 3, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per person)

An attacker shows that a person was in the training data or infers their sensitive attributes

Key controls

Differential privacy or de-identification where feasible; output filtering; A.7.6

Owner and indicator

CISO (DPO advises). Inference test results at validation

Regulatory hooks

GDPR Art 32

Arises at

operation and monitoring; role exposed: provider; harmed: individuals

Sources

EDPB LLM privacy report 2025

Model output is passed to a browser, a database, a shell or an email system without sanitisation; the model becomes a route for injection into the organisation's own systems.

Example scores: likelihood 2, impact 4 (customer and operational), inherent Moderate 8; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per output)

Model output injected into a downstream system causes a major incident

Key controls

Treat output as untrusted input; encoding and validation; least privilege for downstream systems; security testing

Owner and indicator

CISO. Output-handling test coverage

Regulatory hooks

AI Act Art 15(5); NIS2 Art 21

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation

Sources

OWASP LLM05

The system has more tools, data access or autonomy than its task needs, so a single error or injection has a large reach.

Example scores: likelihood 3, impact 4 (financial), inherent High 12; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

An error or injection uses excess permissions to cause a major loss

Key controls

Least privilege; scoped tool permissions; human approval thresholds (A.9.3); action logging (A.6.2.8)

Owner and indicator

System owner. Permissions per agent against task need; high-value actions auto-executed

Regulatory hooks

AI Act Art 14

Arises at

design and development; operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

OWASP LLM06; Singapore agentic AI framework (2026)

A downloaded model, package or extension contains malicious code or an unsafe serialisation; it runs inside the organisation's environment.

Example scores: likelihood 2, impact 4 (customer and operational), inherent Moderate 8; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A malicious model, package or plugin runs inside the environment and causes a major incident

Key controls

A.10.3 supplier process; signed artefacts; software and AI bill of materials; sandboxing; vulnerability management

Owner and indicator

CISO. Share of AI components from approved, verified sources

Regulatory hooks

NIS2 Art 21; DORA; AI Act Art 15

Arises at

design and development; deployment; role exposed: both; harmed: organisation

Sources

OWASP LLM03

API keys in code, open notebooks, unsecured endpoints, logs holding prompts with personal data; the AI stack is run outside the security baseline.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A breach through the AI stack exposes personal data

Key controls

ISO/IEC 27001 controls extended to AI assets (A.4.2-A.4.5 inventory); secrets management; penetration testing

Owner and indicator

CISO. AI assets inside the security baseline; findings per test

Regulatory hooks

GDPR Art 32; NIS2

Arises at

all stages; role exposed: both; harmed: organisation; individuals

Sources

ISO/IEC 27001

An attacker alters an input in ways a person would not notice (a perturbed image, a reworded claim, a padded document) so that a classifier or detector gives the answer the attacker wants: fraud passes, filters miss, identity checks accept.

Example scores: likelihood 3, impact 4 (financial), inherent High 12; vulnerability 4, speed of onset 4, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per decision)

Crafted inputs get fraud or prohibited content through a model-based control, causing a major loss

Key controls

A.6.2.4 adversarial robustness testing; A.6.2.6 monitoring of score distributions; layered controls that do not rely on one model

Owner and indicator

CISO; model owner. Detection rate on an adversarial test set; losses through model-cleared cases

Regulatory hooks

AI Act Art 15(5); DORA Art 9

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

MIT 2.2 (V4 rows 02.10.06, 21.01.04, 64.04.02); NIST AI 100-2 E2025 (adversarial machine learning)

Memorised or in-context personal data (names, addresses, health, cases) surfaces in outputs to other users or staff.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 4, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per output)

Personal data surfaces to another user and is a reportable breach

Key controls

Personal-data filtering in and out; no training on raw personal data without a basis; tenancy isolation; testing (A.6.2.4)

Owner and indicator

System owner (DPO advises). Personal-data detections in outputs; isolation test results

Regulatory hooks

GDPR Art 5(1)(f), 32, 33

Arises at

operation and monitoring; role exposed: both; harmed: individuals

Sources

EDPB LLM privacy report 2025; Samsung source-code leak to ChatGPT (2023)

The system infers health, sexuality, religion, pregnancy or financial distress from ordinary data and acts on it.

Example scores: likelihood 2, impact 4 (regulatory and legal), inherent Moderate 8; vulnerability 4, speed of onset 2, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per person)

The organisation acts on an inferred sensitive attribute and it comes to light

Key controls

A.5.4 impact assessment; feature review; purpose limitation; prohibition screening (Art 5(1)(g))

Owner and indicator

Product owner (DPO advises). Sensitive-inference review outcomes

Regulatory hooks

GDPR Art 9; AI Act Art 5(1)(g), 26

Arises at

design and development; operation and monitoring; role exposed: both; harmed: individuals

Sources

EDPB LLM privacy report 2025

Personal data is pushed through an AI tool for a new purpose with no basis, no DPIA and no record; the use is unlawful from day one.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 2, data quality High; target residual 2 × 4 Moderate 8.

Harm to score (per year)

The Data Protection Commission or a complainant finds the AI processing unlawful

Key controls

Lawful basis identified and recorded before use; purpose specified; DPIA where the processing is likely to be high risk (GDPR Art 35; AI Act 26(9)); A.2.3 alignment with the privacy policy; A.5.2 impact assessment process; records of processing

Owner and indicator

Business owner of the use (DPO advises). AI uses with a recorded lawful basis and, where required, a completed DPIA; uses found without one

Regulatory hooks

GDPR Art 6, 35; AI Act Art 26(9), 27(4)

Arises at

inception; deployment; role exposed: deployer; harmed: individuals; organisation

Sources

Garante decisions on ChatGPT (2023, 2024); Dutch DPA fine on Clearview (2024)

Health, biometric or other special-category data processed by AI without an Article 9 condition; criminal-offence data without the Article 10 safeguards; children's data without age-appropriate design and the care the DPC's Fundamentals expect.

Example scores: likelihood 2, impact 5 (regulatory and legal), inherent High 10; vulnerability 4, speed of onset 2, data quality Medium; target residual 1 × 4 Low 4. Material by the method's rule.

Harm to score (per person)

Unlawful processing of special-category or children's data leads to enforcement

Key controls

A.5.4; Article 9 condition or Article 10 safeguard documented; DPIA; children's data handled under the DPC Fundamentals; biometric and emotion-recognition prohibitions checked (Art 5)

Owner and indicator

Data owner (DPO advises). Inventory lines flagged special-category with a documented condition

Regulatory hooks

GDPR Art 8, 9, 10; AI Act Art 5(1)(f),(g), Annex III(1)

Arises at

all stages; role exposed: both; harmed: individuals

Sources

EDPB; Commission guidelines on prohibited practices (2025)

Staff paste contracts, code, client data or board papers into public tools; the data leaves the organisation and may train someone else's model.

Example scores: likelihood 3, impact 4 (reputational), inherent High 12; vulnerability 4, speed of onset 5, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per year)

Confidential client or company data pasted into an outside tool is exposed or must be disclosed

Key controls

A.2.2 acceptable-use policy; approved enterprise tools with no-training terms; data-loss-prevention controls; training (Art 4)

Owner and indicator

CISO and legal. Data-loss-prevention blocks on AI endpoints; use of unapproved tools

Regulatory hooks

GDPR Art 32; confidentiality and trade-secrets law

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation; individuals

Sources

Samsung (2023)

Generated text, code or images reproduce protected works or licensed code; the organisation publishes or ships it. The reverse exposure sits here too: the organisation may not own, or be able to protect, what its AI produces.

Example scores: likelihood 2, impact 3 (regulatory and legal), inherent Moderate 6; vulnerability 3, speed of onset 3, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per output)

A rights-holder claims against published AI output

Key controls

Provider terms and indemnities (A.10.3); similarity checks for published material; A.9.2

Owner and indicator

Legal. Similarity-check hits on published outputs

Regulatory hooks

Copyright and Related Rights Act 2000; AI Act Art 53(1)(c)

Arises at

operation and monitoring; role exposed: both; harmed: organisation

Sources

US Copyright Office reports on AI (2024-2025)

A system in an Annex III area is treated as not high-risk without the documented assessment, or a deployer becomes a provider by substantial modification without noticing; the full obligations arrive unprepared.

Example scores: likelihood 2, impact 4 (regulatory and legal), inherent Moderate 8; vulnerability 4, speed of onset 2, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A regulator or auditor finds a high-risk system treated as not high-risk

Key controls

Classification record per system (Art 6(4)); legal review at inception (A.6.2.2); change triggers (6.3)

Owner and indicator

Compliance. Share of systems with a documented classification

Regulatory hooks

AI Act Art 6, 25, 49, 99

Arises at

inception; deployment; role exposed: both; harmed: organisation

Sources

Commission draft guidelines on high-risk classification (May 2026)

Emotion recognition at work, social-scoring-like profiling, manipulative design or untargeted face scraping enters through a vendor feature; penalties are the Act's highest tier.

Example scores: likelihood 2, impact 5 (regulatory and legal), inherent High 10; vulnerability 4, speed of onset 2, data quality Medium; target residual 1 × 5 Moderate 5. Material by the method's rule.

Harm to score (per year)

A prohibited practice is found in a live feature

Key controls

Art 5 screening checklist at procurement and at change; A.10.3; A.5.4

Owner and indicator

Compliance. Features screened against Art 5

Regulatory hooks

AI Act Art 5, 99(3)

Arises at

inception; operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

Commission guidelines on prohibited practices (February 2025)

Chatbots and synthetic content unlabelled; public-body systems unregistered; no literacy programme; high-risk duties unmet by 2 December 2027.

Example scores: likelihood 3, impact 3 (regulatory and legal), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality High; target residual 2 × 2 Low 4.

Harm to score (per year)

A regulator or complainant finds a missed transparency, registration or literacy duty

Key controls

Compliance calendar; A.8.2, A.8.5; registration process (Art 49, 26(8)); Art 4 programme; a guidance watcher

Owner and indicator

Compliance. Open obligations against the calendar

Regulatory hooks

AI Act Art 4, 26(8), 49, 50, 113; Regulation (EU) 2026/1744

Arises at

deployment; operation and monitoring; role exposed: both; harmed: organisation

Sources

AI Act summary in the regulations folder

AI use breaches model-risk, outsourcing, device, consumer or employment rules that apply regardless of the AI Act (DORA, MDR, Consumer Protection Code, supervisory guidance). Pricing algorithms that align prices without any agreement can also breach competition law.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 3, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A supervisory finding that AI use breaches sector rules

Key controls

Regulatory mapping per use case (Chapter 15); sector sign-off; A.2.3

Owner and indicator

Compliance. Use cases with a completed sector mapping

Regulatory hooks

DORA; Medical Devices Regulation; EBA and EIOPA opinions; Central Bank of Ireland Consumer Protection Code; TFEU Art 101; Competition Act 2002

Arises at

all stages; role exposed: both; harmed: organisation; individuals

Sources

EIOPA Opinion (2025); EBA report (2025)

Two separate exposures: under the Product Liability Directive software and AI will count as products, with no-fault claims for defective ones; and what a chatbot tells a customer can bind the organisation under contract and consumer law.

Example scores: likelihood 2, impact 4 (financial), inherent Moderate 8; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

A claim succeeds over a defective AI product or a chatbot's promise

Key controls

A.8.2 honest user information; A.10.4 customer terms; A.10.2 responsibility allocation with vendors; insurance

Owner and indicator

Legal. Claims and complaints citing AI statements

Regulatory hooks

Directive (EU) 2024/2853 (transposition by 9 December 2026); contract and consumer law

Arises at

operation and monitoring; role exposed: both; harmed: organisation

Sources

Air Canada tribunal (2024)

AI supplier terms exclude liability, allow training on the customer's data, permit silent changes and give no audit rights.

Example scores: likelihood 3, impact 3 (financial), inherent Moderate 9; vulnerability 5, speed of onset 2, data quality High; target residual 2 × 2 Low 4.

Harm to score (per year)

A vendor failure leaves a loss the contract does not let the organisation recover

Key controls

AI clauses on notification, documentation, no-training, audit and exit (A.10.2, A.10.3); procurement checklist

Owner and indicator

Procurement and legal. Share of AI contracts meeting the clause standard

Regulatory hooks

AI Act Art 25(4) (high-risk components, by agreement); DORA Art 30 (financial entities); otherwise contractual

Arises at

inception; role exposed: deployer; harmed: organisation

Sources

ISO 42001 A.10

A person asks for access, correction or erasure of their data, or objects to its use, and the organisation cannot find or remove what the model has learned from it short of retraining, so the request cannot be met in time.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per person)

A rights request about a trained model cannot be met and the person complains to the Data Protection Commission

Key controls

A.7.5 record of personal data used in training; A.7.2 design for erasure (exclusion lists, output filters, retraining schedule); A.5.4 impact assessment covering rights handling

Owner and indicator

DPO. Rights requests involving AI systems, and the share met within one month

Regulatory hooks

GDPR Art 12(3), 15-17, 21; EDPB Opinion 28/2024

Arises at

operation and monitoring; role exposed: both; harmed: individuals; organisation

Sources

MIT 2.1 (V4 row 65.03.02); EDPB Opinion 28/2024; Garante decision on ChatGPT, December 2024

Many use cases run on one vendor's model and API; a price change, policy change, outage or exit hits all of them at once.

Example scores: likelihood 3, impact 4 (customer and operational), inherent High 12; vulnerability 5, speed of onset 3, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

An outage, price or policy change at the provider disrupts many use cases at once

Key controls

Concentration review in the portfolio view (Chapter 8); abstraction layer and a second source; exit plans (A.10.3)

Owner and indicator

CTO. Share of use cases on a single provider

Regulatory hooks

DORA Art 28-30

Arises at

inception; operation and monitoring; role exposed: deployer; harmed: organisation

Sources

FSB report on AI and financial stability (2024)

The supplier withdraws the product, is bought, changes terms or restricts use; the organisation must migrate under time pressure.

Example scores: likelihood 3, impact 3 (customer and operational), inherent Moderate 9; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

The supplier withdraws or changes the product and the organisation must migrate in a hurry

Key controls

Exit and migration plan; data portability; contractual notice periods

Owner and indicator

Procurement. Contracts with tested exit clauses

Regulatory hooks

DORA Art 28; ISO 42001 A.10.3

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation

Sources

ISO 42001 A.10

The service stops or slows; processes built around it stall; customer-facing commitments are missed.

Example scores: likelihood 4, impact 3 (customer and operational), inherent High 12; vulnerability 3, speed of onset 5, data quality High; target residual 3 × 2 Moderate 6.

Harm to score (per year)

An outage stalls processes and customer commitments are missed

Key controls

Fallback modes; service levels; queueing; A.6.2.6 continuity

Owner and indicator

Operations. Availability against the service level; fallback activations

Regulatory hooks

DORA Art 11

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation; individuals

Sources

ISO 42001 A.6.2.6

Token, compute or licence costs grow non-linearly with adoption or under an attack; budgets and margins break.

Example scores: likelihood 4, impact 3 (financial), inherent High 12; vulnerability 4, speed of onset 3, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

AI costs overrun the budget materially

Key controls

Quotas and rate limits; cost monitoring with thresholds; A.4.5 computing resources documented

Owner and indicator

Operations and finance. Cost per transaction against plan

Regulatory hooks

None specific

Arises at

operation and monitoring; role exposed: both; harmed: organisation

Sources

OWASP LLM10

A prompt, a threshold, a fine-tune, a new data feed or a new use is pushed without re-running the risk assessment, the impact assessment or validation.

Example scores: likelihood 3, impact 4 (customer and operational), inherent High 12; vulnerability 5, speed of onset 4, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per year)

An unassessed change causes a major operational failure

Key controls

6.3 planning of changes; 8.2 and 8.4 re-assessment at significant change; change board with AI criteria; version control

Owner and indicator

System owner. Changes without a linked re-assessment

Regulatory hooks

AI Act Art 9(2), 25(1)(b), 43(4), 72

Arises at

operation and monitoring; re-evaluation; role exposed: both; harmed: organisation; individuals

Sources

ISO 42001 6.3, 8.2, 8.4

A system that updates itself in production drifts from the validated state; nobody approved the version now running.

Example scores: likelihood 2, impact 4 (customer and operational), inherent Moderate 8; vulnerability 4, speed of onset 3, data quality Low; target residual 1 × 3 Low 3.

Harm to score (per year)

A self-updating system drifts into an unapproved state that causes a major failure

Key controls

Freeze or gate learning; validation of each update (A.6.2.4); A.6.2.8

Owner and indicator

Model owner. Versions in production without a validation record

Regulatory hooks

AI Act Art 9(2), 43(4)

Arises at

operation and monitoring; role exposed: provider; harmed: organisation; individuals

Sources

Microsoft Tay (2016)

Staff and teams use AI tools and build automations nobody has inventoried; the risks cannot be identified because the systems are not known.

Example scores: likelihood 3, impact 4 (regulatory and legal), inherent High 12; vulnerability 5, speed of onset 2, data quality High; target residual 2 × 3 Moderate 6.

Harm to score (per year)

An uninventoried AI use causes a data or regulatory incident

Key controls

Inventory process (A.4.2); discovery scans; acceptable-use policy (A.2.2); literacy (Art 4); an easy approved route

Owner and indicator

AI officer. Tools found against tools inventoried; approvals per month

Regulatory hooks

AI Act Art 4, 26; GDPR Art 30

Arises at

all stages; role exposed: deployer; harmed: organisation; individuals

Sources

ISO 42001 4.3, A.4.2

A proof of concept quietly becomes a live service with no assessment, owner, monitoring or documentation.

Example scores: likelihood 4, impact 3 (customer and operational), inherent High 12; vulnerability 5, speed of onset 2, data quality High; target residual 2 × 2 Low 4.

Harm to score (per year)

A pilot running live without an owner or monitoring fails in service

Key controls

A.6.2.5 deployment plan gate; pilot register with end dates; inventory

Owner and indicator

AI officer. Pilots still running past their end date

Regulatory hooks

AI Act Art 26

Arises at

deployment; role exposed: deployer; harmed: organisation; individuals

Sources

ISO 42001 A.6.2.5

Key engineers leave, the vendor's toolkit is deprecated, the model cannot be retrained because the data pipeline is gone; the system rots in place.

Example scores: likelihood 3, impact 3 (customer and operational), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

The system breaks and cannot be repaired or retrained in time

Key controls

A.4.2-A.4.6 resource documentation; maintainability requirements (A.6.2.2); succession

Owner and indicator

Operations. Systems with a one-person dependence

Regulatory hooks

None specific

Arises at

operation and monitoring; retirement; role exposed: both; harmed: organisation

Sources

ISO 42001 C.2.6

A retired system leaves data, access keys, dependent processes and customers' expectations behind; records needed for retention duties are lost.

Example scores: likelihood 2, impact 3 (regulatory and legal), inherent Moderate 6; vulnerability 4, speed of onset 2, data quality Low; target residual 1 × 2 Low 2.

Harm to score (per year)

A retired system leaves exposed data or keys, or records needed later are lost

Key controls

Retirement procedures in the lifecycle (A.6.2); retention of logs and assessments (A.5.3; Art 12, 18)

Owner and indicator

System owner. Decommissioned systems with a completed checklist

Regulatory hooks

AI Act Art 18

Arises at

retirement; role exposed: both; harmed: organisation; individuals

Sources

ISO 42001 C.3.6

Processor, edge-device or network failures corrupt or stop inference; safety functions in physical systems fail unsafe.

Example scores: likelihood 2, impact 4 (customer and operational), inherent Moderate 8; vulnerability 3, speed of onset 5, data quality Medium; target residual 1 × 3 Low 3.

Harm to score (per year)

An infrastructure failure stops inference or makes a physical system fail unsafe

Key controls

Redundancy; fail-safe design (A.6.2.2); monitoring

Owner and indicator

Operations. Infrastructure incidents affecting AI services

Regulatory hooks

Machinery Regulation (EU) 2023/1230

Arises at

operation and monitoring; role exposed: both; harmed: organisation; individuals

Sources

ISO 42001 C.3.5

Generated marketing, news, summaries or customer answers carry falsehoods at scale; the organisation becomes a source of misinformation.

Example scores: likelihood 3, impact 3 (reputational), inherent Moderate 9; vulnerability 4, speed of onset 4, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per output)

Published AI-generated content is shown to be false and the organisation is called out

Key controls

Review before publication; labelling of synthetic content (Art 50); A.8.2

Owner and indicator

Communications. Corrections issued; labelling compliance

Regulatory hooks

AI Act Art 50(2),(4)

Arises at

operation and monitoring; role exposed: both; harmed: society; organisation

Sources

CNET AI-written articles corrected (2023)

Synthetic voice or video of executives or staff used to defraud the organisation or its customers: an external threat whether or not the organisation uses AI itself (its own synthetic media is SOC-01). AI-written phishing and social engineering aimed at staff, and fabricated documents, claims or applications submitted to the organisation, belong here too.

Example scores: likelihood 3, impact 4 (financial), inherent High 12; vulnerability 4, speed of onset 5, data quality Medium; target residual 2 × 3 Moderate 6.

Harm to score (per year)

An AI-made deception (a deepfake, a phishing message or a fabricated submission) defrauds the organisation or its customers

Key controls

Payment verification independent of voice or video; staff training; detection; labelling of the organisation's own content

Owner and indicator

CISO and finance. Verification exceptions; attempted impersonations reported

Regulatory hooks

AI Act Art 50(4); TAKE IT DOWN Act (US, 2025)

Arises at

operation and monitoring; role exposed: none (external threat); harmed: organisation; individuals

Sources

Arup deepfake fraud, Hong Kong (2024)

Compute-heavy use breaks sustainability commitments and reporting and draws criticism.

Example scores: likelihood 3, impact 2 (reputational), inherent Moderate 6; vulnerability 3, speed of onset 1, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

AI use breaks a sustainability commitment and draws criticism

Key controls

Measure and report the compute footprint; model-size choices; A.4.5

Owner and indicator

Sustainability lead. Energy per thousand transactions

Regulatory hooks

Corporate Sustainability Reporting Directive; AI Act Art 95

Arises at

all stages; role exposed: both; harmed: society; organisation

Sources

ISO 42001 C.2.4

Roles change or disappear without consultation, retraining or the Act's worker-information duty; morale, litigation and skills loss follow.

Example scores: likelihood 3, impact 3 (reputational), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Medium; target residual 2 × 2 Low 4.

Harm to score (per year)

Litigation, a dispute or public criticism follows AI-driven role changes

Key controls

Art 26(7) worker information; consultation; retraining plans; 7.2 competence

Owner and indicator

HR. Consultations held before deployment; grievances

Regulatory hooks

AI Act Art 26(7); employment law

Arises at

deployment; role exposed: deployer; harmed: individuals; organisation

Sources

ISO 42001 7.2

Entry-level work that produced tomorrow's experts is automated; the organisation loses the pipeline that made its senior people.

Example scores: likelihood 2, impact 3 (customer and operational), inherent Moderate 6; vulnerability 4, speed of onset 1, data quality Very Low; target residual 1 × 3 Low 3.

Harm to score (per year)

The organisation cannot fill senior roles because entry-level paths were automated away

Key controls

Workforce planning; deliberate retention of developmental work; 7.2

Owner and indicator

HR and executive. Junior-role intake and progression

Regulatory hooks

None specific

Arises at

operation and monitoring; role exposed: deployer; harmed: organisation; society

Sources

ISO 42001 C.2.2; MIT 6.3

Large spend on AI programmes with no measured value, driven by fear of missing out; write-offs and distraction.

Example scores: likelihood 3, impact 3 (financial), inherent Moderate 9; vulnerability 4, speed of onset 2, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

An AI programme is written off without measured value

Key controls

Business case with measured objectives (6.2, A.6.1.2); stage gates; A.6.2.2

Owner and indicator

Executive. Realised against planned benefit per programme

Regulatory hooks

None specific

Arises at

inception; role exposed: both; harmed: organisation

Sources

Zillow Offers (2021)

Refusal to adopt leaves the organisation behind on cost, service and talent, while competitors' use raises customer expectations.

Example scores: likelihood 2, impact 3 (financial), inherent Moderate 6; vulnerability 3, speed of onset 1, data quality Low; target residual 2 × 3 Moderate 6.

Harm to score (per year)

Competitors' use of AI costs the organisation customers or staff

Key controls

AI strategy with objectives (5.2, 6.2); horizon scanning (Delphi, Chapter 6)

Owner and indicator

Executive. Competitor adoption benchmarks

Regulatory hooks

None specific

Arises at

inception; role exposed: both; harmed: organisation

Sources

MIT 6.4

Customers, staff or the public object to AI being used on them at all (customer service, hiring, benefits), regardless of how well it performs.

Example scores: likelihood 3, impact 3 (reputational), inherent Moderate 9; vulnerability 3, speed of onset 3, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

Customers, staff or the public object publicly to the organisation's use of AI

Key controls

A.5.5 societal impact; A.10.4 customer expectations; opt-outs and human routes; honest communication

Owner and indicator

Communications. Opt-out rates; sentiment; complaints about AI use

Regulatory hooks

None specific

Arises at

inception; operation and monitoring; role exposed: both; harmed: organisation

Sources

OECD AI Principles; NIST AI RMF Govern 5

The data labelling, content moderation or testing behind a model the organisation uses was done by workers on very low pay, or exposed to traumatic material without support; the abuse sits in the organisation's supply chain.

Example scores: likelihood 2, impact 3 (reputational), inherent Moderate 6; vulnerability 4, speed of onset 2, data quality Low; target residual 1 × 3 Low 3.

Harm to score (per year)

Labour abuse in an AI supplier's chain is reported and linked to the organisation

Key controls

A.10.3 supplier due diligence on labour conditions; A.10.2 contract clauses on labour standards and audit rights

Owner and indicator

Procurement lead. Share of AI suppliers with labour due-diligence evidence

Regulatory hooks

Directive (EU) 2024/1760 (CSDDD, as amended); Directive (EU) 2022/2464 (CSRD)

Arises at

inception; design and development; role exposed: deployer; harmed: supply-chain workers; organisation

Sources

MIT 6.2 (V4 rows 13.01.07, 18.06.05, 54.01.01, 58.07.09, 61.02.25, 65.23.07, 66.04.07); TIME investigation of data labellers in Kenya (January 2023)

AI cuts the income or business of the organisation's customers, borrowers, policyholders or suppliers, so exposures it already holds become riskier without any AI system of its own being involved. For a lender: borrowers in occupations exposed to AI see their incomes fall, debt service takes a larger share of what they earn, and where those occupations cluster, housing demand and collateral values fall as well.

Example scores: likelihood 2, impact 3 (financial), inherent Moderate 6; vulnerability 4, speed of onset 1, data quality Low; target residual 2 × 2 Low 4.

Harm to score (per year)

Losses rise on loans, policies or contracts because AI has cut the income or business of the customers behind them

Key controls

Review of external issues that includes AI's effect on customers and counterparties (4.1); exposure analysis by customer occupation and sector; AI scenarios in stress tests; concentration limits

Owner and indicator

Chief risk officer; for a lender, credit risk. Share of exposures to customers in occupations or sectors highly exposed to AI; arrears, claims or churn in those segments against the rest of the book

Regulatory hooks

None specific to AI; for banks, Directive 2013/36/EU (CRD) Art 81, concentration risk by economic sector and region

Arises at

inception; operation and monitoring; role exposed: both; harmed: organisation

Sources

MIT 6.2 (V4 rows 31.07.03, 41.01.01, 73.05.03), recorded there as risks to society; IMF Staff Discussion Note SDN/2024/001, Gen-AI: Artificial Intelligence and the Future of Work (January 2024)