This skill enables AI agents to perform a comprehensive
AI risk assessment
using the
NIST AI Risk Management Framework (AI RMF 1.0)
, published January 2023 by the National Institute of Standards and Technology.
The AI RMF is a voluntary, technology- and sector-agnostic framework designed to help organizations manage risks associated with AI systems throughout their lifecycle. It promotes trustworthy AI development by addressing risks that affect individuals, organizations, and society.
Use this skill to identify, assess, and manage AI risks; establish governance structures; ensure trustworthy AI characteristics; and align with international AI risk management best practices.
Combine with "ISO 42001 AI Governance" for comprehensive compliance coverage or "OWASP LLM Top 10" for security-focused assessment.
When to Use This Skill
Invoke this skill when:
Assessing risks of AI systems before deployment
Establishing AI governance and accountability structures
Evaluating trustworthiness of AI products and services
Preparing for regulatory compliance (EU AI Act, state AI laws)
Conducting periodic AI risk reviews
Evaluating third-party AI tools and vendors
Building organizational AI risk management programs
Documenting AI system risks for stakeholders
Inputs Required
When executing this assessment, gather:
ai_system_description
Description of the AI system (purpose, capabilities, deployment context, users, data sources) [REQUIRED]
system_lifecycle_stage
Current stage (design, development, deployment, monitoring, decommissioning) [OPTIONAL, defaults to deployment]
Current risk management processes or controls in place [OPTIONAL]
specific_concerns
Known risks, incidents, or areas of focus [OPTIONAL]
stakeholders
Key stakeholders and affected communities [OPTIONAL]
Trustworthy AI Characteristics
The AI RMF identifies seven characteristics of trustworthy AI that serve as evaluation criteria across all functions:
Valid and Reliable
System performs as intended with consistent results
Safe
System does not endanger human life, health, property, or the environment
Secure and Resilient
System withstands adverse events and recovers gracefully
Accountable and Transparent
Information about the system is available to stakeholders
Explainable and Interpretable
Mechanisms and outputs can be understood
Privacy-Enhanced
Human autonomy and data rights are protected
Fair with Harmful Bias Managed
System does not produce discriminatory outcomes
The 4 Core Functions
The AI RMF Core is composed of four functions, each broken into categories and subcategories:
GOVERN Function
Establishes organizational policies, processes, and accountability for AI risk management. GOVERN is cross-cutting and applies across all other functions.
GOVERN 1: Policies and Processes
Policies, processes, procedures, and practices across the organization related to the mapping, measuring, and managing of AI risks are in place, transparent, and implemented effectively.
GOVERN 1.1
Legal and regulatory requirements involving AI are understood, managed, and documented
GOVERN 1.2
Trustworthy AI characteristics integrated into organizational policies, processes, and practices
GOVERN 1.3
Processes to determine needed risk management activity levels based on organizational risk tolerance
GOVERN 1.4
Risk management process and outcomes established through transparent policies, procedures, and controls
GOVERN 1.5
Ongoing monitoring and periodic review of risk management process with clear roles and responsibilities
GOVERN 1.6
Mechanisms to inventory AI systems resourced by organizational risk priorities
GOVERN 1.7
Processes for decommissioning and phasing out AI systems safely
GOVERN 2: Accountability Structures
Accountability structures ensure appropriate teams and individuals are empowered, responsible, and trained for AI risk management.
GOVERN 2.1
Roles, responsibilities, and communication lines documented and clear
GOVERN 2.2
Personnel receive AI risk management training
GOVERN 2.3
Executive leadership takes responsibility for AI decisions
GOVERN 3: Workforce Diversity and Inclusion
Workforce diversity, equity, inclusion, and accessibility processes are prioritized in AI risk management.
GOVERN 3.1
Decision-making informed by diverse team (demographics, disciplines, expertise)
GOVERN 3.2
Policies define roles for human-AI configurations and oversight
GOVERN 4: Risk Culture
Organizational teams are committed to a culture that considers and communicates AI risk.
GOVERN 4.1
Policies foster critical thinking and safety-first mindset
GOVERN 4.2
Teams document and communicate risks and impacts broadly
GOVERN 4.3
Practices enable AI testing, incident identification, and information sharing
GOVERN 5: Stakeholder Engagement
Processes are in place for robust engagement with relevant AI actors.
GOVERN 5.1
Policies collect, consider, and integrate external feedback on impacts
GOVERN 5.2
Mechanisms regularly incorporate adjudicated feedback into system design
GOVERN 6: Third-Party Risk
Policies and procedures address AI risks from third-party software, data, and supply chain.
GOVERN 6.1
Policies address risks from third-party entities including IP infringement
GOVERN 6.2
Contingency processes handle failures in high-risk third-party systems
MAP Function
Identifies and contextualizes AI system risks within the operational environment.
MAP 1: Context Established
Context is established and understood.
MAP 1.1
Intended purposes, beneficial uses, laws, norms, and deployment settings documented
MAP 1.2
Interdisciplinary AI actors with demographic diversity participate and documented
MAP 1.3
Organization's mission and goals for AI technology understood and documented
MAP 1.4
Business value or context clearly defined or re-evaluated
MAP 1.5
Organizational risk tolerances determined and documented
MAP 1.6
System requirements elicited with socio-technical considerations
MAP 2: System Categorization
Categorization of the AI system is performed.
MAP 2.1
Specific tasks and methods defined (classifiers, generative models, recommenders)
MAP 2.2
System knowledge limits and human oversight documented
MAP 2.3
Scientific integrity and TEVV considerations identified
MAP 3: Capabilities and Costs
AI capabilities, targeted usage, goals, expected benefits, and costs are understood.
MAP 3.1
Potential benefits of intended functionality examined and documented
MAP 3.2
Potential costs (monetary and non-monetary) from AI errors documented
MAP 3.3
Targeted application scope specified based on capability
MAP 3.4
Operator and practitioner proficiency assessed
MAP 3.5
Human oversight processes defined and documented
MAP 4: Component Risks
Risks and benefits are mapped for all components including third-party.
MAP 4.1
Approaches for mapping technology and legal risks documented
MAP 4.2
Internal risk controls for components identified and documented
MAP 5: Impact Characterization
Impacts to individuals, groups, communities, organizations, and society are characterized.
MAP 5.1
Likelihood and magnitude of impacts (beneficial and harmful) documented
MAP 5.2
Practices for regular engagement with relevant AI actors documented
MEASURE Function
Employs tools, techniques, and methodologies to assess, benchmark, and monitor AI risk.
MEASURE 1: Methods and Metrics
Appropriate methods and metrics are identified and applied.
MEASURE 1.1
Approaches and metrics selected starting with most significant risks
MEASURE 1.2
Appropriateness of metrics regularly assessed and updated
MEASURE 1.3
Internal experts or independent assessors involved in assessments
MEASURE 2: Trustworthiness Evaluation
AI systems are evaluated for trustworthy characteristics.
MEASURE 2.1
Test sets, metrics, and tool details documented during TEVV
MEASURE 2.2
Evaluations with human subjects meet requirements and represent relevant populations
MEASURE 2.3
Performance or assurance criteria measured and demonstrated
MEASURE 2.4
Functionality and behavior monitored in production
MEASURE 2.5
System demonstrated valid and reliable with generalizability limitations documented
MEASURE 2.6
System regularly evaluated for safety risks with residual risk within tolerance
MEASURE 2.7
Security and resilience evaluated and documented
MEASURE 2.8
Transparency and accountability risks examined
MEASURE 2.9
AI model explained, validated, and output interpreted within context
MEASURE 2.10
Privacy risk examined and documented
MEASURE 2.11
Fairness and bias evaluated with results documented
MEASURE 2.12
Environmental impact and sustainability assessed
MEASURE 2.13
Effectiveness of TEVV metrics evaluated
MEASURE 3: Risk Tracking
Mechanisms for tracking identified AI risks over time are in place.
MEASURE 3.1
Approaches track existing, unanticipated, and emergent risks
MEASURE 3.2
Risk tracking considered for settings where assessment is difficult
MEASURE 3.3
Feedback processes for end users to report problems and appeal outcomes
MEASURE 4: Measurement Efficacy
Feedback about efficacy of measurement is gathered and assessed.
MEASURE 4.1
Measurement approaches informed by domain experts and end users
MEASURE 4.2
Results validated for consistency with intended performance
MEASURE 4.3
Measurable performance improvements or declines identified
MANAGE Function
Allocates resources to mapped and measured risks on a regular basis.
MANAGE 1: Risk Prioritization
AI risks based on assessments are prioritized, responded to, and managed.
MANAGE 1.1
Determination made whether AI system achieves intended purposes
MANAGE 1.2
Treatment of risks prioritized based on impact, likelihood, and resources
MANAGE 1.3
Responses to high-priority risks developed (mitigate, transfer, avoid, accept)
MANAGE 1.4
Negative residual risks documented for downstream users
MANAGE 2: Benefit Maximization
Strategies to maximize AI benefits and minimize negative impacts are planned and documented.
MANAGE 2.1
Resources to manage risks considered alongside non-AI alternatives
MANAGE 2.2
Mechanisms to sustain value of deployed systems
MANAGE 2.3
Procedures to respond to and recover from unknown risks
MANAGE 2.4
Mechanisms to supersede, disengage, or deactivate inconsistent systems
MANAGE 3: Third-Party Risk Management
AI risks and benefits from third-party entities are managed.
MANAGE 3.1
Third-party risks regularly monitored with controls applied
MANAGE 3.2
Pre-trained models monitored as part of regular maintenance
MANAGE 4: Communication and Monitoring
Risk treatments and communication plans are documented and monitored.
MANAGE 4.1
Post-deployment monitoring plans with user input, appeal, and decommissioning mechanisms
MANAGE 4.2
Continual improvement activities integrated into updates
MANAGE 4.3
Incidents communicated to relevant actors; tracking and recovery documented
Audit Procedure
Follow these steps systematically:
Step 1: System Understanding (15 minutes)
Review AI system:
Analyze
ai_system_description
and
system_lifecycle_stage
Identify system type (classifier, generative, recommender, autonomous, etc.)
Document data sources, models, and deployment environment
Note stakeholders and affected communities
Understand context:
Review
organization_context
and regulatory environment
Identify applicable laws and standards
Note risk tolerance and existing controls
Define scope:
Determine which functions and categories to assess
Prioritize based on lifecycle stage and concerns
Step 2: GOVERN Assessment (20 minutes)
Evaluate organizational governance:
G1
Are AI risk policies in place and transparent?
G1.1
Legal/regulatory requirements understood and documented?
G1.2
Trustworthy AI characteristics in organizational policies?
G1.5
Monitoring and review processes planned?
G1.6
AI system inventory maintained?
G2
Accountability structures defined?
G2.1
Roles and responsibilities clear?
G2.3
Executive leadership accountable?
G3.1
Diverse team informing decisions?
G4
Risk culture fostered?
G5
Stakeholder engagement processes in place?
G6
Third-party risks addressed?
Step 3: MAP Assessment (20 minutes)
Evaluate risk identification and context:
M1.1
Intended purposes and deployment context documented?