NIST AI RMF Training: Govern, Map, Measure and Manage AI Risk

Starts with a question: if one of your AI systems treated a customer unfairly tomorrow, who would notice and who would answer for it? After a self-rating, three recent cases follow: a lender whose untested models led to a two and a half million dollar settlement, an airline chatbot whose words bound its company, and a regulator asking firms to know their AI. You then learn what AI risk means, how likely a harm is and how large it would be, for individuals, organisations, society and the planet. You meet PQR Lending, a digital lending company in New York, plus a bank in London and an insurer in Toronto. The module outlines the thirteen modules, corrects myths such as the framework being a checklist, and ends with a recap.

What you will be able to do

• Explain what AI risk means in terms of likelihood and harm
• Recognise the governance gaps behind three recent AI cases
• Describe who the NIST AI RMF is for and how the course works

Why it matters

Seeing how unowned AI systems led to settlements and rulings shows why every model you build, buy or use needs an owner.

Practice: Practice: Why manage AI risk · 3 questions

Opens with a simple comparison: traditional software follows rules people wrote, while an AI system learns its behaviour from data that can change over time. You then learn how AI risks differ: data may not fit, models drift, they are harder to explain and test, and they act in a social setting, with a bank in London setting card limits and a manufacturer in Tokyo predicting machine wear. The module covers NIST's four challenges: measuring risk, setting tolerance, prioritising and integrating AI risk into enterprise risk. It explains the framework's two parts, the four functions Govern, Map, Measure and Manage, the numbers from NIST's four tables, the Playbook of suggested actions, and the revision of version 1.0. It ends with a common mistake and a recap.

What you will be able to do

• Explain why AI risks differ from traditional software risks
• Describe the four functions of the AI RMF Core
• Integrate AI risk into the enterprise risk register

Why it matters

Understanding how the framework is built lets you use it as a way of working rather than a form to fill in.

Practice: Practice: AI risk and the framework · 3 questions

Starts with a choice PQR Lending's data science team faced: Model A was most accurate but could not explain declines, while Model B was slightly less accurate but gave clear reasons and fair results across groups. You then learn NIST's seven characteristics of trustworthy AI, with valid and reliable as the base and accountability and transparency running through all of them. The module explains how transparency, explainability and interpretability answer different questions, and the three kinds of bias that can arise even without intent. A hospital network in Chicago with a sepsis alert and a fintech in Singapore scoring small businesses show trade-offs in context. It covers recording the choice and its reasons so the trade-off is transparent, a common mistake, and a recap.

What you will be able to do

• Explain the seven characteristics of trustworthy AI
• Tell transparency, explainability and interpretability apart
• Record an AI trade-off decision and its reasons

Why it matters

Balancing accuracy against fairness and explainability openly lets you justify the model you chose when someone asks.

Practice: Practice: Trustworthy AI and its trade-offs · 3 questions

Opens with PQR Lending's customer chatbot: another company developed the large language model behind it and sees general risks, while PQR deploys it and sees risks the developer never could, such as wrong answers on loan fees. You then learn who the AI actors are and their tasks across the life cycle, from design and development to deployment and operation, with testing and evaluation throughout, ideally by independent people. The module explains People and Planet at the centre of NIST's life cycle figure, and why risk management starts at planning. A retailer in Sydney buying a recommendation engine and a software firm in Mumbai selling hiring software show other roles. It covers defining human roles, records of override rates and independent validation, and a recap.

What you will be able to do

• Identify the AI actors involved across a system's life cycle
• Explain why developers and deployers see different risks
• Define human roles alongside AI and track override data

Why it matters

Knowing who sees which part of the risk helps you assign ownership so nothing falls between developer and deployer.

Practice: Practice: AI actors across the life cycle · 3 questions

Starts in PQR Lending's first month, when the chief risk officer found AI models in three teams, no complete list, no AI policy and nobody with authority to stop a risky system, and asked for a policy, an inventory, named owners and a board decision on risk tolerance. You then learn Govern 1: policies, processes and practices that cover law, trustworthy AI, risk tolerance, monitoring, an inventory and safe retirement, including Govern 1.4. The module explains what to keep for each inventoried system, and Govern 2: clear roles, training for staff and partners, and leaders who own AI risk decisions. A large insurer in Toronto with a board AI committee shows a heavier form. It covers risk tiers, a common mistake and a recap.

What you will be able to do

• Write an AI policy that covers risk tolerance and retirement
• Build an AI inventory with an owner for every system
• Assign roles and authority to stop an AI system

Why it matters

Governance gives every AI system an owner and a limit, so someone can say no before a risky model reaches customers.

Practice: Practice: Govern: policies and accountability · 3 questions

Opens during PQR Lending's first quarter, when the chatbot's model provider moved to a new model version without telling PQR, the chatbot began describing late fees differently, and a customer complaint was the first sign. You then learn Govern 3, diverse teams and defined human roles, and Govern 4 and 5, a culture that invites challenge, documents and shares risks, protects insiders who report problems and listens to people outside the team. The module covers Govern 6 on third-party software, data and supply chains, with the New York regulator's view on vendors and authentication that deepfakes cannot impersonate. A bank in Frankfurt with three lines of defence and a media studio in Los Angeles rewarding reports show the practice. It ends with a recap.

What you will be able to do

• Explain how diverse teams and a culture of challenge reduce AI risk
• Set up channels for affected people to report AI problems
• Govern third-party AI with tests, contract terms and contingency plans

Why it matters

Good culture and vendor contracts catch problems, such as a silent model update, before customers do.

Practice: Practice: Govern: culture, people and third parties · 3 questions

Returns to the Massachusetts case from Module 1, where a lender's models allegedly used inputs such as a school default rate and were not tested for disparate impact, and asks what Map would have asked before launch. You then learn the five Map categories: Map 1 on purposes, users, laws, settings, business value and requirements; Map 2 to 4 on categorising the system, weighing benefits and costs, defining scope and oversight, and mapping every component; and Map 5 on impacts by likelihood and size. You see part of PQR Lending's Map record for its underwriting model. A university hospital in Boston and a telecom company in Dubai show requests declined as out of scope. It covers records, the go or no-go decision, a common mistake and a recap.

What you will be able to do

• Document an AI system's context, users, laws and limits in a Map record
• Assess impacts by likelihood and magnitude
• Make and record a go or no-go decision with its reasons

Why it matters

Mapping the context first stops a model being reused for purposes it was never designed or tested for.

Practice: Practice: Map: context, categories and impacts · 3 questions

Starts with PQR Lending's underwriting model, ninety-two per cent accurate on one historic test set measured by its builders, until the model risk team found errors twice as high for applicants with thin credit files. You then learn Measure 1: choosing metrics for the most significant risks, documenting what cannot be measured and involving independent assessors. The module covers Measure 2: evaluating each trustworthy characteristic in conditions like real use, recording false positive and false negative rates, reporting variance, and red-team exercises under Measure 2.7. A hospital network in Chicago measuring missed sepsis cases and a bank in London measuring fairness and drift monthly show the practice. It ends with records, agreeing limits before results, a common mistake and a recap.

What you will be able to do

• Choose metrics that match an AI system's most significant risks
• Break headline results down by group and period
• Plan independent testing, including a red-team exercise

Why it matters

Measuring what matters, in real conditions and independently, shows whether a good headline number hides harm to some groups.

Practice: Practice: Measure: methods, metrics and TEVV · 3 questions

Opens with a credit model after interest rates rise: at one lender nobody compares live results with the original tests and defaults rise for months, while PQR's monthly check compares live inputs with the training data. You then learn to monitor systems in production with alerts and baselines, as a fintech in Singapore does with automated drift alerts. The module explains tracking known, unexpected and hard-to-measure risks with owners, and Measure 3.3 feedback processes that let users and affected communities report problems and appeal outcomes, with a retailer in Sydney giving staff a flag button. It covers analysing appeals, checking whether measurement works with domain experts and override data, sensitivity analysis after each update, records, a common mistake and a recap.

What you will be able to do

• Set up production monitoring that compares live and test results
• Use appeals and complaints as data for risk tracking
• Run a sensitivity analysis after each system update

Why it matters

Monitoring after launch tells you a model has stopped working before a customer, auditor or regulator does.

Practice: Practice: Measure: tracking and feedback · 3 questions

Starts on a Tuesday afternoon at PQR Lending, nine months into the programme, when the monitoring alert for the approval gap between groups fires and the company must know who can stop the model and what replaces it. You then learn Manage 1: deciding whether to proceed, prioritising risks, choosing a response for each and documenting residual risk. The module covers Manage 2 and 3: sustaining value, handling unknown risks, keeping backup systems and an off switch, and monitoring third-party and pre-trained models, with an insurer in Toronto triaging claims and a software firm in Mumbai using a code review assistant. It shows PQR's incident record, Manage 4 monitoring and improvement, the incident and near-miss log, bypass tests and a recap.

What you will be able to do

• Prioritise AI risks and choose a documented response for each
• Plan a tested way to switch off a critical AI system
• Log AI incidents and near misses with root causes and actions

Why it matters

A planned response and a tested off switch let you stop a harmful system within the hour and keep the business running.

Practice: Practice: Manage: respond, monitor, deactivate · 3 questions

Returns to the Canadian chatbot case from Module 1, where a tribunal held an airline responsible for its chatbot's wrong advice on a bereavement fare, and asks how often PQR's chatbot is confidently wrong. You then learn the Generative AI Profile's twelve risks, from confabulation to the value chain, including direct and indirect prompt injection, with actions tied to AI RMF subcategories. The module explains its four primary considerations: governance, pre-deployment testing, content provenance such as watermarking and metadata, and incident disclosure. An HR software firm in Paris and a bank in London with a coding assistant show the risks. It covers building a profile for one use case by comparing current and target states and closing gaps in priority order, records and a recap.

What you will be able to do

• Explain the main generative AI risks, such as confabulation
• Describe the four primary considerations of the profile
• Build a current and target profile for one generative AI use case

Why it matters

Profiling each generative AI use shows where chatbots and assistants could mislead customers or leak data, and what to fix first.

Practice: Practice: The Generative AI Profile · 3 questions

Opens with a partner bank in Frankfurt that wants to use PQR Lending's underwriting model and asks whether it is ready for the EU AI Act and whether PQR is certified to ISO/IEC 42001. PQR has neither, but has a year of AI RMF records. You then learn that the AI RMF is a voluntary framework, ISO/IEC 42001 a certifiable management system standard and the EU AI Act a law. The module walks NIST's crosswalk, such as Govern 2.3 mapping to clause 5.1, and checks PQR's records against each area. A manufacturer in Tokyo seeking certification and a fintech in Singapore serving the USA and the EU from one programme show the approach. It ends with credit scoring as high-risk, the need for a gap analysis, and a recap.

What you will be able to do

• Explain how the AI RMF, ISO/IEC 42001 and the EU AI Act differ
• Map AI RMF outcomes to ISO/IEC 42001 clauses with the crosswalk
• Identify the gaps to close for EU AI Act readiness

Why it matters

Organising AI RMF work well lets one programme answer clients asking about ISO/IEC 42001 and the EU AI Act.

Practice: Practice: ISO/IEC 42001 and the EU AI Act · 3 questions

Starts with a question: if your board asked tomorrow how well your organisation manages AI risk, what would you show them? You then follow PQR Lending's year, from models in three teams with no inventory, no owner and an unreviewed chatbot, to a policy, an inventory, Map records, independent tests, monitoring and a tested way to switch each critical system off. You see Map records for high-tier systems, planned responses and fallbacks for every high-priority risk, and the incident log with the vendor data feed. The module gives a road map, govern, list, map, measure, manage and profile, with a plan for your first ninety days. It ends with the key takeaways and the scenario-based final exam with its 60% mark.

What you will be able to do

• Trace how one company applied all four AI RMF functions in a year
• Plan the first ninety days of an AI risk programme
• Explain the evidence a board needs on AI risk management

Why it matters

Seeing a full programme built in one year gives you a practical plan to start with the system that touches people most.

Practice: Practice: Putting it together at PQR Lending · 3 questions