The Significance of AI Risk Assessments

One of the most important contributions of Google DeepMind's new AGI Safety and Security paper is a clean, actionable framing of risk types. Instead of lumping all AI risks into one “doomer” narrative, they break it down into 4 clear categories- with very different implications for mitigation: 1. Misuse → The user is the adversary This isn’t the model behaving badly on its own. It’s humans intentionally instructing it to cause harm- think jailbreak prompts, bioengineering recipes, or social engineering scripts. If we don’t build strong guardrails around access, it doesn’t matter how aligned your model is. Safety = security + control 2. Misalignment → The AI is the adversary The model understands the developer’s intent- but still chooses a path that’s misaligned. It optimizes the reward signal, not the goal behind it. This is the classic “paperclip maximizer” problem, but much more subtle in practice. Alignment isn’t a static checkbox. We need continuous oversight, better interpretability, and ways to build confidence that a system is truly doing what we intend- even as it grows more capable. 3. Mistakes → The world is the adversary Sometimes the AI just… gets it wrong. Not because it’s malicious, but because it lacks the context, or generalizes poorly. This is where brittleness shows up- especially in real-world domains like healthcare, education, or policy. Don’t just test your model- stress test it. Mistakes come from gaps in our data, assumptions, and feedback loops. It's important to build with humility and audit aggressively. 4. Structural Risks → The system is the adversary These are emergent harms- misinformation ecosystems, feedback loops, market failures- that don’t come from one bad actor or one bad model, but from the way everything interacts. These are the hardest problems- and the most underfunded. We need researchers, policymakers, and industry working together to design incentive-aligned ecosystems for AI. The brilliance of this framework: It gives us language to ask better questions. Not just “is this AI safe?” But: - Safe from whom? - In what context? - Over what time horizon? We don’t need to agree on timelines for AGI to agree that risk literacy like this is step one. I’ll be sharing more breakdowns from the paper soon- this is one of the most pragmatic blueprints I’ve seen so far. 🔗Link to the paper in comments. -------- If you found this insightful, do share it with your network ♻️ Follow me (Aishwarya Srinivasan) for more AI news, insights, and educational content to keep you informed in this hyperfast AI landscape 💙

This new white paper "Introduction to AI assurance" by the UK Department for Science, Innovation, and Technology from Feb 12, 2024, provides an EXCELLENT overview of assurance methods and international technical standards that can be utilized to create and implement ethical AI systems. The new guidance is based on the UK AI governance framework, laid out in the 2023 white paper "A pro-innovation approach to AI regulation". This white paper defined 5 universal principles applicable across various sectors to guide and shape the responsible development and utilization of AI technologies throughout the economy: - Safety, Security, and Robustness - Appropriate Transparency and Explainability - Fairness - Accountability and Governance - Contestability and Redress The 2023 white paper also introduced a suite of tools designed to aid organizations in understanding "how" these outcomes can be achieved in practice, emphasizing tools for trustworthy AI, including assurance mechanisms and global technical standards. See: https://lnkd.in/gydvi9Tt The new publication, "Introduction to AI assurance," is a deep dive into these assurance mechanisms and standards. AI assurance encompasses a spectrum of techniques for evaluating AI systems throughout their lifecycle. These range from qualitative assessments for evaluating potential risks and societal impacts to quantitative assessments for measuring performance and legal compliance. Key techniques include: - Risk Assessment: Identifies potential risks like bias, privacy, misuse of technology, and reputational damage. - Impact Assessment: Anticipates broader effects on the environment, human rights, and data protection. - Bias Audit: Examines data and outcomes for unfair biases. - Compliance Audit: Reviews adherence to policies, regulations, and legal requirements. - Conformity Assessment: Verifies if a system meets required standards, often through performance testing. - Formal Verification: Uses mathematical methods to confirm if a system satisfies specific criteria. The white paper also explains how organizations in the UK can ensure their AI systems are responsibly governed, risk-assessed, and compliant with regulations: 1.) For demonstrating good internal governance processes around AI, a conformity assessment against standards like ISO/IEC 42001 (AI Management System) is recommended. 2.) To understand the potential risks of AI systems being acquired, an algorithmic impact assessment by a accredited conformity assessment body is advised. This involves (self) assessment against a proprietary framework or responsible AI toolkit. 3.) Ensuring AI systems adhere to existing data protection regulations involves a compliance audit by a third-party assurance provider. This white paper also has exceptional infographics! Pls, check it out, and TY Victoria Beckman for posting and providing us with great updates as always!

AI use in 𝗔𝗡𝗬 government is 𝗡𝗢𝗧 a partisan issue - it affects 💥everyone.💥 I am just as excited about the opportunities that AI can bring as those that are leading the way. However, prioritizing AI without strong risk management opens the door WIDE to unintended consequences. There are AI Risk Management Frameworks developed (take your pick of one) that lay out clear guidelines to prevent those unintended consequences Here are a few concerns that stand out: ⚫ Speed Over Scrutiny Rushing AI into deployment can mean skipping critical evaluations. For example, NIST emphasizes iterative testing and thorough risk assessments throughout an AI system’s lifecycle. Without these, we risk rolling out systems that aren't fully understood. ⚫ Reduced Human Oversight When AI takes center stage, human judgment can get pushed to the sidelines. Most frameworks stress the importance of oversight and accountability, ensuring that AI-driven decisions remain ethical and transparent. Without clear human responsibility, who do we hold accountable when things go wrong? ⚫ Amplified Bias and Injustice AI is only as fair as the data and design behind it. We’ve already seen hiring algorithms and law enforcement tools reinforce discrimination. If bias isn’t identified and mitigated, AI could worsen existing inequities. It's not a technical issue—it’s a societal risk. ⚫ Security and Privacy Trade-offs A hasty AI rollout without strong security measures could expose critical systems to cyber threats and privacy breaches. An AI-first approach promises efficiency and innovation, but without caution, it is overflowing with risk. Yes...our government should be innovative and leverage technological breakthroughs 𝗕𝗨𝗧...and this is a 𝗕𝗜𝗚 one...it 𝗛𝗔𝗦 𝗧𝗢 𝗕𝗘 secure, transparent, and accountable. Are we prioritizing speed over safety? -------------------------------------------------------------- Opinions are my own and not the views of my employer. -------------------------------------------------------------- 👋 Chris Hockey | Manager at Alvarez & Marsal 📌 Expert in Information and AI Governance, Risk, and Compliance 🔍 Reducing compliance and data breach risks by managing data volume and relevance 🔍 Aligning AI initiatives with the evolving AI regulatory landscape ✨ Insights on: • AI Governance • Information Governance • Data Risk • Information Management • Privacy Regulations & Compliance 🔔 Follow for strategic insights on advancing information and AI governance 🤝 Connect to explore tailored solutions that drive resilience and impact

⚠️Privacy Risks in AI Management: Lessons from Italy’s DeepSeek Ban⚠️ Italy’s recent ban on #DeepSeek over privacy concerns underscores the need for organizations to integrate stronger data protection measures into their AI Management System (#AIMS), AI Impact Assessment (#AIIA), and AI Risk Assessment (#AIRA). Ensuring compliance with #ISO42001, #ISO42005 (DIS), #ISO23894, and #ISO27701 (DIS) guidelines is now more material than ever. 1. Strengthening AI Management Systems (AIMS) with Privacy Controls 🔑Key Considerations: 🔸ISO 42001 Clause 6.1.2 (AI Risk Assessment): Organizations must integrate privacy risk evaluations into their AI management framework. 🔸ISO 42001 Clause 6.1.4 (AI System Impact Assessment): Requires assessing AI system risks, including personal data exposure and third-party data handling. 🔸ISO 27701 Clause 5.2 (Privacy Policy): Calls for explicit privacy commitments in AI policies to ensure alignment with global data protection laws. 🪛Implementation Example: Establish an AI Data Protection Policy that incorporates ISO27701 guidelines and explicitly defines how AI models handle user data. 2. Enhancing AI Impact Assessments (AIIA) to Address Privacy Risks 🔑Key Considerations: 🔸ISO 42005 Clause 4.7 (Sensitive Use & Impact Thresholds): Mandates defining thresholds for AI systems handling personal data. 🔸ISO 42005 Clause 5.8 (Potential AI System Harms & Benefits): Identifies risks of data misuse, profiling, and unauthorized access. 🔸ISO 27701 Clause A.1.2.6 (Privacy Impact Assessment): Requires documenting how AI systems process personally identifiable information (#PII). 🪛 Implementation Example: Conduct a Privacy Impact Assessment (#PIA) during AI system design to evaluate data collection, retention policies, and user consent mechanisms. 3. Integrating AI Risk Assessments (AIRA) to Mitigate Regulatory Exposure 🔑Key Considerations: 🔸ISO 23894 Clause 6.4.2 (Risk Identification): Calls for AI models to identify and mitigate privacy risks tied to automated decision-making. 🔸ISO 23894 Clause 6.4.4 (Risk Evaluation): Evaluates the consequences of noncompliance with regulations like #GDPR. 🔸ISO 27701 Clause A.1.3.7 (Access, Correction, & Erasure): Ensures AI systems respect user rights to modify or delete their data. 🪛 Implementation Example: Establish compliance audits that review AI data handling practices against evolving regulatory standards. ➡️ Final Thoughts: Governance Can’t Wait The DeepSeek ban is a clear warning that privacy safeguards in AIMS, AIIA, and AIRA aren’t optional. They’re essential for regulatory compliance, stakeholder trust, and business resilience. 🔑 Key actions: ◻️Adopt AI privacy and governance frameworks (ISO42001 & 27701). ◻️Conduct AI impact assessments to preempt regulatory concerns (ISO 42005). ◻️Align risk assessments with global privacy laws (ISO23894 & 27701).   Privacy-first AI shouldn't be seen just as a cost of doing business, it’s actually your new competitive advantage.

I was at Hugging Face during the critical year before and after ChatGPT's release. One thing became painfully clear: the ways AI systems can fail are exponentially more numerous than traditional software. Enterprise leaders today are under-estimating AI risks. Data privacy and hallucinations are just the tip of the iceberg. What enterprises aren't seeing: The gap between perceived and actual AI failure modes is staggering. - Enterprises think they're facing 10 potential failure scenarios…  - when the reality is closer to 100. AI risks fall into two distinct categories that require completely different approaches: Internal risks: When employees use AI tools like ChatGPT, they often inadvertently upload proprietary information. Your company's competitive edge is now potentially training competitor's models. Despite disclaimer pop-ups, this happens constantly. External risks: These are far more dangerous. When your customers interact with your AI-powered experiences, a single harmful response can destroy brand trust built over decades. Remember when Gemini's image generation missteps wiped billions off Google's market cap? Shout out to Dr. Ratinder, CTO Security and Gen AI, Pure Storage. When I got on a call with Ratinder, he very enthusiastically explained to me their super comprehensive approach: ✅ Full DevSecOps program with threat modeling, code scanning, and pen testing, secure deployment and operations ✅ Security policy generation system that enforces rules on all inputs/outputs ✅ Structured prompt engineering with 20+ techniques ✅ Formal prompt and model evaluation framework ✅ Complete logging via Splunk for traceability ✅ Third-party pen testing certification for customer trust center ✅ OWASP Top 10 framework compliance ✅ Tests for jailbreaking attempts during the development phase Their rigor is top-class… a requirement for enterprise-grade AI. For most companies, external-facing AI requires 2-3x the guardrails of internal systems. Your brand reputation simply can't afford the alternative. Ask yourself: What AI risk factors is your organization overlooking? The most dangerous ones are likely those you haven't even considered.

ISO 5338 has key AI risk management considerations useful to security and compliance leaders. It's a non-certifiable standard laying out best practices for the AI system lifecycle. And it’s related to ISO 42001 because control A6 from Annex A specifically mentions ISO 5338. Here are some key things to think about at every stage: INCEPTION -> Why do I need a non-deterministic system? -> What types of data will the system ingest? -> What types of outputs will it create? -> What is the sensitivity of this info? -> Any regulatory requirements? -> Any contractual ones? -> Is this cost-effective? DESIGN AND DEVELOPMENT -> What type of model? Linear regressor? Neural net? -> Does it need to talk to other systems (an agent)? -> What are the consequences of bad outputs? -> What is the source of the training data? -> How / where will data be retained? -> Will there be continuous training? -> Do we need to moderate outputs? -> Is system browsing the internet? VERIFICATION AND VALIDATION -> Confirm system meets business requirements. -> Consider external review (per NIST AI RMF). -> Do red-teaming and penetration testing. -> Do unit, integration, and UA testing DEPLOYMENT -> Would deploying  system be within our risk appetite? -> If not, who is signing off? What is the justification? -> Train users and impacted parties. -> Update shared security model. -> Publish documentation. -> Add to asset inventory. OPERATION AND MONITORING -> Do we have a vulnerability disclosure program? -> Do we have a whistleblower portal? -> How are we tracking performance? -> Model drift? CONTINUOUS VALIDATION -> Is the system still meeting our business requirements? -> If there is an incident or vulnerability, what do we do? -> What are our legal disclosure requirements? -> Should we disclose even more? -> Do regular audits. RE-EVALUATION -> Has the system exceeded our risk appetite? -> If an incident, do a root cause analysis. -> Do we need to change policies? -> Revamp procedures? RETIREMENT -> Is there business need to retain model or data? Legal? -> Delete everything we don’t need, including backups. -> Audit the deletion. Are you using ISO 5338 for AI risk management?

To all Executives looking to build AI systems responsibly, Yoshua Bengio and a team of 100+ of AI Advisory Experts from more than 30 countries recently published the International AI Safety Report 2025, consisting of ~300 pages of insights. Below is a TLDR (with the help of AI) of the content of the document that you should pay attention to, including risks and mitigation strategies, as you continuously deploy new AI-powered experiences for your customers. 🔸AI Capabilities Are Advancing Rapidly: • AI is improving at an unprecedented pace, especially in programming, scientific reasoning, and automation • AI agents that can act autonomously with little human oversight are in development • Expect continuous breakthroughs, but also new risks as AI becomes more powerful 🔸Key Risks for Businesses and Society: • Malicious Use: AI is being used for deepfake scams, cybersecurity attacks, and disinformation campaigns • Bias & Unreliability: AI models still hallucinate, reinforce biases, and make incorrect recommendations, which could damage trust and credibility • Systemic Risks: AI will most likely impact labor markets while creating new job categories, but will increase privacy violations, and escalate environmental concerns • Loss of Control: Some experts worry that AI systems may become difficult to control, though opinions differ on how soon this could happen 🔸Risk Management & Mitigation Strategies: • Regulatory Uncertainty: AI laws and policies are not yet standardized, making compliance challenging • Transparency Issues: Many companies keep AI details secret, making it hard to assess risks • Defensive AI Measures: Companies must implement robust monitoring, safety protocols, and legal safeguards • AI Literacy Matters: Executives should ensure that teams understand AI risks and governance best practices 🔸Business Implications: • AI Deployment Requires Caution. Companies must weigh efficiency gains against potential legal, ethical, and reputational risks • AI Policy is Evolving. Companies must stay ahead of regulatory changes to avoid compliance headaches • Invest in AI Safety. Companies leading in ethical AI use will have a competitive advantage • AI Can Enhance Security. AI can also help detect fraud, prevent cyber threats, and improve decision-making when used responsibly 🔸The Bottom Line • AI’s potential is massive, but poor implementation can lead to serious risks • Companies must proactively manage AI risks, monitor developments, and engage in AI governance discussions • AI will not “just happen.” Human decisions will shape its impact. Download the report below, and share your thoughts on the future of AI safety! Thanks to all the researchers around the world who took created this report and took the time to not only surface the risks, but provided actionable recommendations on how to address them. #genai #technology #artificialintelligence

Everyone is rushing to adopt #AI as quickly as possible. Few are doing much more than nodding to the potential risks, but addressing these risks will become increasingly important as AI becomes more ubiquitous, interconnected, and powerful. Researchers have created a database of 777 AI risks. You may find this excessive, but the effort is designed to provide a framework for organizations to consider and simplify their risks.  The database breaks these risks into different causal and domain categories. The causal factors include (1) Entity: Human, AI; (2) Intentionality: Intentional, Unintentional; and (3) Timing: Pre-deployment; Post-deployment. And the Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental, and (7) AI system safety, failures, & limitations. The researchers' interesting observation is that contrary to popular opinion, the risks of AI are NOT well understood or being universally addressed. One of the researchers noted, “We found that the average frameworks mentioned just 34% of the 23 risk subdomains we identified, and nearly a quarter covered less than 20%." If you'd like to learn more, the TechCrunch article does a nice job of summarizing the research: https://lnkd.in/ghpmZ4TU You can read the research report here: https://lnkd.in/gjeEwtYa And the database of AI risks is available to you here: https://airisk.mit.edu/

The UK Department for Science, Innovation and Technology published the guide "Introduction to AI assurance," to provide an overview of assurance mechanisms and global technical standards for industry and #regulators to build and deploy responsible #AISystems. #Artificialintelligence assurance processes can help to build confidence  in #AI systems by measuring and evaluating reliable, standardized, and accessible evidence about their capabilities. It measures whether such systems will work as intended, hold limitations, or pose potential risks; as well as how those #risks are being mitigated to ensure that ethical considerations are built-in throughout the AI development #lifecycle. The guide outlines different AI assurance mechanisms, including: - Risk assessments - Algorithmic impact assessment - Bias and compliance audits - Conformity assessment - Formal verification It also provides some recommendations for organizations interested in developing their understanding of AI assurance: 1. Consider existing regulations relevant for AI systems (#privacylaws, employment laws, etc) 2. Develop necessary internal skills to understand AI assurance and anticipate future requirements. 3. Review internal governance and #riskmanagement practices and ensure effective decision-making at appropriate levels.  4. Keep abreast of sector-specific guidance on how to operationalize and implement proposed principles in each regulatory domain.  5. Consider engaging with global standards development organizations to ensure the development of robust and universally accepted standard protocols. https://lnkd.in/eiwRZRXz

AI poses serious risks when used the wrong way. Our present situation with the emergence of AI reminds me of the early years of my engineering career. Graphing calculators and engineering software were introduced and some thought it was the beginning of the end of quality engineering. In reality, these tools have been a net positive, but only once we put them in capable hands and in a proper workflow. Fast forward 20 years and AI is here in safety, and its here to stay. But, how do we use it well and avoid the traps? I see four potential scenarios: - Effective and Efficient: A knowledgeable person who knows how to use AI to accelerate, enhance, and review their work. - Effective but Inefficient: A knowledgeable and skilled person who does not use AI. - Ineffective and Inefficient: An ignorant or unskilled person who doesn’t use AI. - Dangerous: An ignorant or unskilled person using AI to rapidly produce bad output The risk of the “dangerous” category is very real. That’s why our team is equally focused on two things: (1) enhancing the fidelity of the AI and (2) ensuring the AI is used effectively. --- Here is an example of a good and bad use of ChatSafetyAI: ✅ DO: Use ChatSafetyAI to check your high-energy control assessments (HECA) to see if you missed anything. ❌ DONT: Use ChatSafetyAI to do your HECA for you. Proper workflow: Integrate the ChatSafetyAI API after an initial assessment to provide feedback and recommendations. This additive function helps the assessors to “fill in the gaps” with more intelligence. This workflow leverages both human and artificial intelligence, assuming effort is placed in the initial assessment. Our council, comprised of the licensees of ChatSafetyAI, is working on this. Consider joining us. I would love to hear your ideas on the effective use of AI for safety.