At a glance
This competency is the conceptual foundation for everything else on the exam. You need to (1) state the accepted definition and types of AI, (2) identify the risks and harms AI can cause, (3) explain the characteristics of AI that make governance necessary, and (4) recognise and apply the principles of responsible AI. These ideas recur throughout Domains II–IV, so anchoring them now pays off everywhere.
A useful framing: AI needs its own governance because it is not ordinary software. It is data-dependent, probabilistic, opaque, autonomous and operates at scale — so traditional, deterministic controls are not enough.
I.A.1 — Definitions and types of AI
The accepted definition. The exam follows the OECD (2024) definition, which the EU AI Act (Art. 3(1)) adopted almost word-for-word:
An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.
The load-bearing words are infers, autonomy and adaptiveness — they are what distinguish AI from deterministic, hand-coded software.
Ways AI is categorised:
- By capability:
- Narrow AI (ANI) — task-specific. Everything deployed today is narrow AI.
- General AI (AGI) — hypothetical human-level breadth.
- Super AI (ASI) — hypothetical, beyond human capability.
- By technique:
- Symbolic / rule-based AI (“good old-fashioned AI”, expert systems) — explicit human-written rules; deterministic and transparent but brittle.
- Machine learning — learns from data. Sub-types: supervised (labelled data), unsupervised (find structure, e.g., clustering), reinforcement (learn by reward), and self-supervised (used to pretrain LLMs).
- Deep learning — multi-layer neural networks; powers most modern AI.
- By output / use:
- Discriminative / “classic” AI — predicts or classifies (e.g., fraud score, churn model).
- Generative AI — produces new content; built on foundation models / LLMs, increasingly multimodal and agentic.
The AI tech stack — governance questions attach to every layer, so know the rough anatomy: compute (specialised chips/GPUs, usually cloud — concentration and cost live here), data (storage, pipelines, labeling), models/frameworks (the ML libraries and the trained models themselves), and applications (the products users touch). A supply-chain question (“where does third-party risk enter?”) can point at any layer.
AI use-case types — the standard vocabulary for what an AI system does, which scenario questions expect you to label precisely:
- Recognition — identify what something is (match an image/face/voice to a known thing).
- Detection — find that something is present or anomalous (fraud flags, threat detection, event detection).
- Forecasting — predict a future value or trend from historical data.
- Personalization — tailor content or recommendations to a known individual’s profile.
- Optimization — find the best option under constraints (routing, pricing, scheduling).
- Interaction — converse with humans (chatbots, virtual assistants).
Adjacent term to keep separate: robotic process automation (RPA) — software “robots” that automate rule-based digital tasks by mimicking human actions. Classic RPA is deterministic automation, not learning AI — though it is increasingly augmented with AI components.
I.A.2 — Risks and harms posed by AI
Harms can land on individuals, groups, organizations and society — the BOK lists all four levels, and the exam often asks you to match a harm to the right level.
| Level | Example harms |
|---|---|
| Individuals | Privacy intrusion, discrimination, physical safety, economic loss, manipulation/loss of autonomy |
| Groups | Systematic bias against protected classes; representational harm; group surveillance |
| Organizations | Legal liability, regulatory penalties, reputational damage, financial loss, security compromise, IP exposure |
| Society | Disinformation at scale, erosion of public trust, labour displacement, environmental cost, concentration of power |
The BOK calls out three risk types explicitly:
- Misalignment with objectives — the system optimises for the wrong thing (a proxy goal), producing outputs that diverge from human intent (“reward hacking”, specification gaming).
- Ethics and bias risk — unfair or harmful outcomes, often from biased data or design. NIST (SP 1270) splits bias into systemic, statistical/computational and human-cognitive. Within those, know the named sub-types scenarios test: sampling bias (training data under-represents part of the population — e.g., training a hiring model only on your current, homogeneous top performers), temporal bias (data from one period no longer represents the present — e.g., a dataset skewed by a one-off event), and confirmation bias (humans favouring outputs that match what they already believe — a human-cognitive bias).
- Complexity and scalability — AI fails and propagates at speed and scale; a flaw can affect millions before anyone notices.
Other commonly tested AI-specific risks: hallucination (confident but false generative output), lack of robustness/brittleness, and security threats unique to ML — data poisoning, adversarial examples, model inversion/extraction, and prompt injection.
I.A.3 — Characteristics of AI that require governance
These are the properties that make AI different from ordinary software and therefore demand a dedicated governance approach. The BOK enumerates them — learn the list:
- Complexity — many interacting components; behaviour hard to fully specify or predict.
- Opacity — the “black box” problem; even builders may not be able to explain a specific output.
- Autonomy — acts and adapts with limited human intervention.
- Speed and scale — decisions made instantly across huge populations.
- Potential for harm or misuse — dual-use; can be repurposed maliciously.
- Data dependency — quality, bias and provenance of training data drive behaviour (“garbage in, garbage out”).
- Probabilistic vs. deterministic outputs — produces likelihoods, not guaranteed answers; the same input may yield different outputs.
I.A.4 — Principles of responsible AI
Across OECD, NIST, the EU and others the same cluster of principles recurs. The BOK’s canonical list:
- Fairness — avoid unjust bias and discriminatory outcomes; treat people equitably.
- Safety and reliability — perform as intended, robustly, without causing unreasonable harm.
- Privacy and security — protect personal data and defend the system against attack.
- Transparency and explainability — disclose that AI is in use and make its workings understandable.
- Accountability — assign clear human responsibility and provide redress.
- Human-centricity — keep humans in control; respect rights, autonomy and well-being (human oversight / “human in/over the loop”).
These map onto NIST’s characteristics of trustworthy AI: valid & reliable (the base), safe, secure & resilient, accountable & transparent, explainable & interpretable, privacy-enhanced, and fair – with harmful bias managed.
How this shows up later
- The responsible-AI principles reappear as legal requirements in Domain II (EU AI Act, GDPR) and as design controls in Domains III–IV.
- The characteristics of AI justify specific lifecycle controls — impact assessments, human oversight, monitoring for drift.
- The risk/harm taxonomy is the basis for risk classification (EU AI Act tiers) and impact assessments.