Risks of AI in Financial Applications

☢️Manage Third-Party AI Risks Before They Become Your Problem☢️ AI systems are rarely built in isolation as they rely on pre-trained models, third-party datasets, APIs, and open-source libraries. Each of these dependencies introduces risks: security vulnerabilities, regulatory liabilities, and bias issues that can cascade into business and compliance failures. You must move beyond blind trust in AI vendors and implement practical, enforceable supply chain security controls based on #ISO42001 (#AIMS). ➡️Key Risks in the AI Supply Chain AI supply chains introduce hidden vulnerabilities: 🔸Pre-trained models – Were they trained on biased, copyrighted, or harmful data? 🔸Third-party datasets – Are they legally obtained and free from bias? 🔸API-based AI services – Are they secure, explainable, and auditable? 🔸Open-source dependencies – Are there backdoors or adversarial risks? 💡A flawed vendor AI system could expose organizations to GDPR fines, AI Act nonconformity, security exploits, or biased decision-making lawsuits. ➡️How to Secure Your AI Supply Chain 1. Vendor Due Diligence – Set Clear Requirements 🔹Require a model card – Vendors must document data sources, known biases, and model limitations. 🔹Use an AI risk assessment questionnaire – Evaluate vendors against ISO42001 & #ISO23894 risk criteria. 🔹Ensure regulatory compliance clauses in contracts – Include legal indemnities for compliance failures. 💡Why This Works: Many vendors haven’t certified against ISO42001 yet, but structured risk assessments provide visibility into potential AI liabilities. 2️. Continuous AI Supply Chain Monitoring – Track & Audit 🔹Use version-controlled model registries – Track model updates, dataset changes, and version history. 🔹Conduct quarterly vendor model audits – Monitor for bias drift, adversarial vulnerabilities, and performance degradation. 🔹Partner with AI security firms for adversarial testing – Identify risks before attackers do. (Gemma Galdon Clavell, PhD , Eticas.ai) 💡Why This Works: AI models evolve over time, meaning risks must be continuously reassessed, not just evaluated at procurement. 3️. Contractual Safeguards – Define Accountability 🔹Set AI performance SLAs – Establish measurable benchmarks for accuracy, fairness, and uptime. 🔹Mandate vendor incident response obligations – Ensure vendors are responsible for failures affecting your business. 🔹Require pre-deployment model risk assessments – Vendors must document model risks before integration. 💡Why This Works: AI failures are inevitable. Clear contracts prevent blame-shifting and liability confusion. ➡️ Move from Idealism to Realism AI supply chain risks won’t disappear, but they can be managed. The best approach? 🔸Risk awareness over blind trust 🔸Ongoing monitoring, not just one-time assessments 🔸Strong contracts to distribute liability, not absorb it If you don’t control your AI supply chain risks, you’re inheriting someone else’s. Please don’t forget that.

The Cybersecurity and Infrastructure Security Agency together with the National Security Agency, the Federal Bureau of Investigation (FBI), the National Cyber Security Centre, and other international organizations, published this advisory providing recommendations for organizations in how to protect the integrity, confidentiality, and availability of the data used to train and operate #artificialintelligence. The advisory focuses on three main risk areas: 1. Data #supplychain threats: Including compromised third-party data, poisoning of datasets, and lack of provenance verification. 2. Maliciously modified data: Covering adversarial #machinelearning, statistical bias, metadata manipulation, and unauthorized duplication. 3. Data drift: The gradual degradation of model performance due to changes in real-world data inputs over time. The best practices recommended include: - Tracking data provenance and applying cryptographic controls such as digital signatures and secure hashes. - Encrypting data at rest, in transit, and during processing—especially sensitive or mission-critical information. - Implementing strict access controls and classification protocols based on data sensitivity. - Applying privacy-preserving techniques such as data masking, differential #privacy, and federated learning. - Regularly auditing datasets and metadata, conducting anomaly detection, and mitigating statistical bias. - Securely deleting obsolete data and continuously assessing #datasecurity risks. This is a helpful roadmap for any organization deploying #AI, especially those working with limited internal resources or relying on third-party data.

Agentic AI is about to shake up finance — and most banks aren’t even close to ready. At IBM, we just dropped a new report that dives deep into what happens when autonomous AI agents collide with highly regulated industries. Spoiler: it’s not just a finance issue. Anyone building or scaling AI should be paying attention. Agentic AI isn’t a futuristic concept. It’s here — making real-time decisions in onboarding, fraud detection, compliance, loan approvals, and more. Here are 6 hard-hitting takeaways from the report: ⬇️ 1. Legacy controls are toast. → When agents are making real decisions, static controls won’t cut it. You’ll need 30+ dynamic guardrails before going live. 2. Multi-agent = multi-risk. → Agents coordinating with other agents sounds great — until one misfires. Cue bias, drift, or even deception. 3. Memory is both a weapon and a liability. → Agents remember. That’s powerful — but dangerous without reset, expiry, and audit policies aligned to financial data regulations. 4. The top risks? Deception, bias, and misuse. → The report shows real-world examples of agents going rogue. Monitoring must be real-time. Patching after the fact isn’t enough. 5. Forget dashboards — think registries. → You need to track every agent like a microservice: metadata, permissions, logs, and all. This is DevOps meets AI Governance. 6. Compliance isn’t paperwork anymore. It’s architecture. → If it’s not “compliance by design,” you’re already behind. Regulators won’t wait for you to catch up. This isn’t theory. Agentic systems are already in production. The big question: Will you shape their future — or get blindsided by it?

New #Fintech Snark Tank post: 𝗪𝗵𝗲𝗻 𝗔𝗜 𝗚𝗼𝗲𝘀 𝗢𝗳𝗳 𝗧𝗵𝗲 𝗥𝗮𝗶𝗹𝘀: 𝗟𝗲𝘀𝘀𝗼𝗻𝘀 𝗙𝗿𝗼𝗺 𝗧𝗵𝗲 𝗚𝗿𝗼𝗸 𝗗𝗲𝗯𝗮𝗰𝗹𝗲 I'm guessing that, by now, most of you have heard that Elon Musk’s AI chatbot, Grok, went disturbingly off the rails. What began as a mission to create an alternative to “woke” AI assistants turned into a case study in how LLMs can spiral into hateful, violent, and unlawful behavior. 𝙈𝙮 𝙩𝙖𝙠𝙚: The Grok debacle is more than just a PR blunder. It's a wake-up call to nearly every industry, in particular banking and financial services. Here’s what banks and credit unions should do now: ▶️ 𝗘𝘀𝘁𝗮𝗯𝗹𝗶𝘀𝗵 𝗮𝗻 𝗔𝗜 𝗿𝗶𝘀𝗸 𝗺𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁 𝘁𝗲𝗮𝗺. Nearly every bank and credit union I’ve spoken to in the past 18 months has developed an “AI policy” and has--or is looking to--establish an “AI governance board.” Not good enough. The issue is much more operational. Financial institutions need feet on the ground to: 1) review model behaviors and outputs; 2) coordinate compliance, technology, risk, and legal departments; and 3) manage ethical, legal, and reputational risks. ▶️ 𝗔𝘂𝗱𝗶𝘁 𝗔𝗜 𝘃𝗲𝗻𝗱𝗼𝗿𝘀. Ask AI providers: 1) What data was the model trained on? 2) What are its safeguards for bias, toxicity, hallucination? 3) How are model outputs tested and monitored in real-time? Refuse “black box” answers. Require documentation of evaluation metrics and alignment strategies. ▶️ 𝗧𝗿𝗲𝗮𝘁 𝗽𝗿𝗼𝗺𝗽𝘁𝘀 𝗹𝗶𝗸𝗲 𝗽𝗼𝗹𝗶𝗰𝗶𝗲𝘀. Every system prompt should be reviewed like a policy manual. Instruct models not just on how to behave—but also on what to avoid. Prompts should include escalation rules, prohibited responses, and fallback protocols for risky queries. A lot more analysis and recommendations in the article. Please give it a read. The link is in the comments. #ElonMusk #Grok #xAI #GenAI #GenerativeAI

Live from a long flight home: I did some heavy reading so you don’t have to 😏 → A Spring 2025 overview of top AI Security Risks for Enterprise. 1. Prompt injection & jailbreaking - Bypass the model’s guardrails - Indirect injection is on the rise using PDFs, emails, etc. - Manipulate it, leak training data: customer info, IP, … 2. Model/supply chain compromise - Devs often use pre-trained AI models from 3rd parties - Hidden backdoor in a model = you’re compromised! - Ex: Sleepy Pickle, with malicious code hidden in the model, and triggered once deployed 3. Poisoned datasets - A poisoned dataset can make a model misbehave - Ex: fail to detect fraud, or misclassify malware - Cheap! As little as $60 to poison a dataset like LAION 4. Extremely convincing deepfakes - Think perfect (fake) videos of your CTO asking for a network policy change - Crafted with public samples of the CTO’s voice/video - Leads to a security breach 5. Agentic AI threats - AI agents can have vast powers on a system - But they can be compromised by new kinds of malware - That malware can write its own code and “learn” to break a system over time ---- It doesn’t mean we need to slow down on AI. It’s important however to: - Educate teams - Put the right guardrails in place - Manage risk at every point of the AI lifecycle - Leverage frameworks such as OWASP/MITRE Annnnddd.... Leveraging a solution such as Cisco AI Defense can really help manage AI risk: - Get full visibility across AI apps, models, etc. - Define & enforce granular policy around the use of AI - Validate models before they go in prod (including through algorithmic jailbreaking) - Protect AI apps during runtime Anand, Manu, DJ and all other AI security gurus here: what did I forget?

3 AI Governance Risks — Confidentiality, Reputation, Litigation On my podcast, Security & GRC Decoded, I had the chance to sit down with Walter Haydock CEO of StackAware. Walter didn’t mince words: "There are three broad buckets of risk that businesses are focused on... specific to AI." With his security & governance experience at the Office of the Director of National Intelligence, the U.S. House of Representatives, PTC, and the Cloud Security Alliance, Walter has seen risks from every angle. In this clip, he outlined 3 risks business leaders and security teams needs to understand: 1️⃣ Confidentiality risk — are you leaking sensitive data? AI systems often rely on external tools and data flows. If sensitive information slips into third-party systems, your governance program has a serious gap. 2️⃣ Reputation risk — when AI makes promises you have to keep. Walter shared a real-world example: "There's an incident with Air Canada where someone essentially got that chat bot to tell him something about the refund policy, which didn't end up to be true. And then that person sued Air Canada…to perform what its chat bot said it would do." 3️⃣ Litigation risk — are you training or using AI in ways that could get you sued? From training data to how the AI product itself works, companies need to think through legal exposure from every angle. Walter’s advice for governance teams was clear: "I would pick whatever bucket is most salient to the organization, and then do a deep dive and present that to your leadership and say, ‘Here are the risks, here's what's happened, here's what could happen. Here's my assessment on the probability and impact and, as a business leader, what do you think we should do?’" 📌 You can find the full conversation linked in the first comment. If you’re interested in the voices shaping security and governance today, check out recent episodes of Security & GRC Decoded, featuring: 🗣️ Mosi Platt - Senior Security Compliance Engineer at Netflix 🗣️ Carlos Batista - who’s held critical IT security roles at AWS, Bakkt & SunTrust 🗣️ Abhay Kshirsagar - Director of Security Services & Tools at Salesforce, previously with Cisco and BPM LLP And, is there someone with thoughts and ideas on how security GRC is developing that you follow? Let me know! Perhaps we can invite them to the show. #AIgovernance #AIGRC #CyberRisk #SecurityGRC #RiskManagement

𝐀𝐈 𝐫𝐢𝐬𝐤 𝐢𝐬𝐧’𝐭 𝐨𝐧𝐞 𝐭𝐡𝐢𝐧𝐠. 𝐈𝐭’𝐬 𝟏,𝟔𝟎𝟎 𝐭𝐡𝐢𝐧𝐠𝐬. That’s not hyperbole. A new meta-review compiled over 1,600 distinct AI risks from 65 frameworks and surfaced a tough truth: most organizations are underestimating both the scope and structure of AI risk. It’s not just about bias, fairness, or hallucination. Risks emerge at different stages, from different actors, with different incentives: • Pre-deployment design decisions • Post-deployment human misuse • Model failure, misalignment, drift • Unclear accountability across teams The taxonomy distinguishes between human and AI causes, intentional and unintentional behaviors, and domain-specific vs. systemic risks. But here’s the real insight: Most AI risks don’t stem from malicious design. They emerge from fragmented ownership and unmanaged complexity. No single team sees the whole picture. Governance lives in compliance. Development lives in product. Monitoring lives in infra. And no one owns the handoffs. → Strategic takeaway: You don’t need another checklist. You need a cross-functional risk architecture. One that maps responsibility, observability, and escalation paths, before the headlines do it for you. AI systems won’t fail in one place. They’ll fail at the intersections. 𝐓𝐫𝐞𝐚𝐭 𝐀𝐈 𝐫𝐢𝐬𝐤 𝐚𝐬 𝐚 𝐜𝐡𝐞𝐜𝐤𝐛𝐨𝐱, 𝐚𝐧𝐝 𝐢𝐭 𝐰𝐢𝐥𝐥 𝐬𝐡𝐨𝐰 𝐮𝐩 𝐥𝐚𝐭𝐞𝐫 𝐚𝐬 𝐚 𝐡𝐞𝐚𝐝𝐥𝐢𝐧𝐞.

JPMorgan’s CISO warns: AI is creating security risks Speed without safety is a system-wide risk. We're deploying AI faster than we’re securing it. That gap is growing - and it’s exploitable. Pat Opet just sounded the alarm. In his open letter, he points to: → Rushed SaaS deployments in finance → Weak or missing security protocols → Interconnected systems at risk of collapse His warning? “Maybe they can’t breach your castle. But who’s guarding the road to it?” This isn’t just a banking problem. It’s a digital economy problem. - Slow down. - Test before you scale. - Validate before you deploy. Innovation is nothing without integrity. And AI is useless if it’s built on sand. Found this helpful? Follow Arturo and repost.

Sam Altman Warns: AI Fraud Crisis Looms Over Financial Industry ⸻ Introduction: Altman Urges Banking Sector to Prepare for AI-Driven Threats Speaking at a Federal Reserve conference in Washington, D.C., OpenAI CEO Sam Altman issued a stark warning to financial executives and regulators: artificial intelligence is enabling a coming wave of sophisticated fraud, and many banks remain dangerously unprepared. His remarks underscore the urgency of rethinking authentication and cybersecurity protocols in an age when AI can convincingly mimic human behavior — even voices. ⸻ Key Highlights from Altman’s Remarks • Voice Authentication No Longer Secure • Altman expressed concern that some banks still rely on voice prints to authorize major transactions. • “That is a crazy thing to still be doing,” he said, emphasizing that AI can now easily replicate voices, rendering such security methods obsolete. • AI has “fully defeated” most forms of biometric or behavioral authentication — except strong passwords, he noted. • Rise in AI-Enabled Scams • Financial institutions are increasingly targeted by deepfake and impersonation-based fraud, made possible by publicly accessible AI tools. • The sophistication of these attacks is growing faster than many firms’ ability to defend against them, Altman warned. • Urgency for Regulatory Response • The comments were made in an onstage interview with Michelle Bowman, the Fed’s new vice chair for supervision. • Altman’s presence at the Fed’s event highlights how AI security is becoming a top-tier concern for financial oversight bodies. • Broader Implications for the Industry • The conversation sparked concern among attendees about the need for: • Stronger multi-factor authentication • Better fraud detection systems • Industry-wide cooperation to stay ahead of AI threats ⸻ Why It Matters: Financial Systems Face a Tipping Point Altman’s warning comes at a pivotal moment, as AI capabilities rapidly evolve while outdated financial protocols remain in place. The growing risk of synthetic identity fraud, voice spoofing, and real-time impersonation could cost banks billions — and erode customer trust. As banks digitize services, the balance between convenience and security is more fragile than ever. Altman’s call to action is clear: the financial sector must abandon obsolete verification methods and invest in advanced, AI-resilient systems — before fraudsters exploit the gap. ⸻ https://lnkd.in/gEmHdXZy