How to Balance Trust and Skepticism in AI

The AI gave a clear diagnosis. The doctor trusted it. The only problem? The AI was wrong. A year ago, I was called in to consult for a global healthcare company. They had implemented an AI diagnostic system to help doctors analyze thousands of patient records rapidly. The promise? Faster disease detection, better healthcare. Then came the wake-up call. The AI flagged a case with a high probability of a rare autoimmune disorder. The doctor, trusting the system, recommended an aggressive treatment plan. But something felt off. When I was brought in to review, we discovered the AI had misinterpreted an MRI anomaly. The patient had an entirely different condition—one that didn’t require aggressive treatment. A near-miss that could have had serious consequences. As AI becomes more integrated into decision-making, here are three critical principles for responsible implementation: - Set Clear Boundaries Define where AI assistance ends and human decision-making begins. Establish accountability protocols to avoid blind trust. - Build Trust Gradually Start with low-risk implementations. Validate critical AI outputs with human intervention. Track and learn from every near-miss. - Keep Human Oversight AI should support experts, not replace them. Regular audits and feedback loops strengthen both efficiency and safety. At the end of the day, it’s not about choosing AI 𝘰𝘳 human expertise. It’s about building systems where both work together—responsibly. 💬 What’s your take on AI accountability? How are you building trust in it?

We have to internalize the probabilistic nature of AI. There’s always a confidence threshold somewhere under the hood for every generated answer and it's important to know that AI doesn’t always have reasonable answers. In fact, occasional "off-the-rails" moments are part of the process. If you're an AI PM Builder (as per my 3 AI PM types framework from last week) - my advice: 1. Design for Uncertainty: ✨Human-in-the-loop systems: Incorporate human oversight and intervention where necessary, especially for critical decisions or sensitive tasks. ✨Error handling: Implement robust error handling mechanisms and fallback strategies to gracefully manage AI failures (and keep users happy). ✨User feedback: Provide users with clear feedback on the confidence level of AI outputs and allow them to provide feedback on errors or unexpected results. 2. Embrace an experimental culture & Iteration / Learning: ✨Continuous monitoring: Track the AI system's performance over time, identify areas for improvement, and retrain models as needed. ✨A/B testing: Experiment with different AI models and approaches to optimize accuracy and reliability. ✨Feedback loops: Encourage feedback from users and stakeholders to continuously refine the AI product and address its limitations. 3. Set Realistic Expectations: ✨Educate users: Clearly communicate the potential for AI errors and the inherent uncertainty involved about accuracy and reliability i.e. you may experience hallucinations.. ✨Transparency: Be upfront about the limitations of the system and even better, the confidence levels associated with its outputs.

How do you know what you know? Now, ask the same question about AI. We assume AI "knows" things because it generates convincing responses. But what if the real issue isn’t just what AI knows, but what we think it knows? A recent study on Large Language Models (LLMs) exposes two major gaps in human-AI interaction: 1. The Calibration Gap – Humans often overestimate how accurate AI is, especially when responses are well-written or detailed. Even when AI is uncertain, people misread fluency as correctness. 2. The Discrimination Gap – AI is surprisingly good at distinguishing between correct and incorrect answers—better than humans in many cases. But here’s the problem: we don’t recognize when AI is unsure, and AI doesn’t always tell us. One of the most fascinating findings? More detailed AI explanations make people more confident in its answers, even when those answers are wrong. The illusion of knowledge is just as dangerous as actual misinformation. So what does this mean for AI adoption in business, research, and decision-making? ➡️ LLMs don’t just need to be accurate—they need to communicate uncertainty effectively. ➡️Users, even experts, need better mental models for AI’s capabilities and limitations. ➡️More isn’t always better—longer explanations can mislead users into a false sense of confidence. ➡️We need to build trust calibration mechanisms so AI isn't just convincing, but transparently reliable. 𝐓𝐡𝐢𝐬 𝐢𝐬 𝐚 𝐡𝐮𝐦𝐚𝐧 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐚𝐬 𝐦𝐮𝐜𝐡 𝐚𝐬 𝐚𝐧 𝐀𝐈 𝐩𝐫𝐨𝐛𝐥𝐞𝐦. We need to design AI systems that don't just provide answers, but also show their level of confidence -- whether that’s through probabilities, disclaimers, or uncertainty indicators. Imagine an AI-powered assistant in finance, law, or medicine. Would you trust its output blindly? Or should AI flag when and why it might be wrong? 𝐓𝐡𝐞 𝐟𝐮𝐭𝐮𝐫𝐞 𝐨𝐟 𝐀𝐈 𝐢𝐬𝐧’𝐭 𝐣𝐮𝐬𝐭 𝐚𝐛𝐨𝐮𝐭 𝐠𝐞𝐭𝐭𝐢𝐧𝐠 𝐭𝐡𝐞 𝐫𝐢𝐠𝐡𝐭 𝐚𝐧𝐬𝐰𝐞𝐫𝐬—𝐢𝐭’𝐬 𝐚𝐛𝐨𝐮𝐭 𝐡𝐞𝐥𝐩𝐢𝐧𝐠 𝐮𝐬 𝐚𝐬𝐤 𝐛𝐞𝐭𝐭𝐞𝐫 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬. What do you think: should AI always communicate uncertainty? And how do we train users to recognize when AI might be confidently wrong? #AI #LLM #ArtificialIntelligence

There’s been a lot of schadenfreude lately - over Klarna backtracking on its AI rollout, over Duolingo's “AI-first” messaging misfire, over companies that ran headfirst into the future... and stumbled. It’s easy to point and laugh. It’s harder, and more useful, to ask: what can we learn? Klarna and Duolingo didn’t have the wrong idea. The problem wasn’t the tech - It was the tone. They’ve given the entire ecosystem a live demo of what works, what breaks, and where the line between ambition and alienation really is. So instead of dunking on the companies bold enough to go first, let’s ask: what does good AI leadership look like? ⚙️ AI ≠ Headcount Reduction. AI = Leverage. If the headline is “X jobs eliminated,” you’ve already lost the room. Lead with how it empowers your team and improves customer experience - not how many roles it eliminates. 🧠 People + Process + Product > Just Product. AI adoption isn’t a software install. It’s an organizational rewire. Start by layering it into real workflows. Watch. Learn. Then redesign roles based on what actually changes. 🤝 Build Trust Before You Break Things. Trust is your AI transformation rate-limiter. Communicate like crazy. What you’re doing, why, how it’ll impact people. Make room for opt-in experiments. Let people become part of the story - not casualties of it. And never automate the human out of high-emotion moments. 🧭 Run Two Operating Systems in Parallel. Your business needs a stable core and an experimental edge. Let your “AI beta org” operate in parallel where risk is isolated, iteration is fast, and learnings feed back into the whole. Protect the main business. 🧍If It Touches People, Lead With Empathy. AI decisions are cultural decisions. Design change with dignity, especially when roles are evolving. Preserve the human layer in functions where nuance matters: support, legal, HR, anything reputation-facing. Narrate the transition like a human, not like a shareholder update. In the end, AI is just leverage. It multiplies what’s already there - trust or fear, clarity or chaos, empathy or ego. Klarna and Duolingo took the first swing. Not perfect, but brave. The rest of us now get to move smarter. More human. Still bold.

Prompting isn’t the hard part anymore. Trusting the output is. You finally get a model to reason step-by-step… And then? You're staring at a polished paragraph, wondering:    > “Is this actually right?”    > “Could this go to leadership?”    > “Can I trust this across markets or functions?” It looks confident. It sounds strategic. But you know better than to mistake that for true intelligence. 𝗛𝗲𝗿𝗲’𝘀 𝘁𝗵𝗲 𝗿𝗶𝘀𝗸: Most teams are experimenting with AI. But few are auditing it. They’re pushing outputs into decks, workflows, and decisions— With zero QA and no accountability layer 𝗛𝗲𝗿𝗲’𝘀 𝘄𝗵𝗮𝘁 𝗜 𝘁𝗲𝗹𝗹 𝗽𝗲𝗼𝗽𝗹𝗲: Don’t just validate the answers. Validate the reasoning. And that means building a lightweight, repeatable system that fits real-world workflows. 𝗨𝘀𝗲 𝘁𝗵𝗲 𝗥.𝗜.𝗩. 𝗟𝗼𝗼𝗽: 𝗥𝗲𝘃𝗶𝗲𝘄 – What’s missing, vague, or risky? 𝗜𝘁𝗲𝗿𝗮𝘁𝗲 – Adjust one thing (tone, data, structure). 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗲 – Rerun and compare — does this version hit the mark? Run it 2–3 times. The best version usually shows up in round two or three, not round one.  𝗥𝘂𝗻 𝗮 60-𝗦𝗲𝗰𝗼𝗻𝗱 𝗢𝘂𝘁𝗽𝘂𝘁 𝗤𝗔 𝗕𝗲𝗳𝗼𝗿𝗲 𝗬𝗼𝘂 𝗛𝗶𝘁 𝗦𝗲𝗻𝗱: • Is the logic sound? • Are key facts verifiable? • Is the tone aligned with the audience and region? • Could this go public without risk? 𝗜𝗳 𝘆𝗼𝘂 𝗰𝗮𝗻’𝘁 𝘀𝗮𝘆 𝘆𝗲𝘀 𝘁𝗼 𝗮𝗹𝗹 𝗳𝗼𝘂𝗿, 𝗶𝘁’𝘀 𝗻𝗼𝘁 𝗿𝗲𝗮𝗱𝘆. 𝗟𝗲𝗮𝗱𝗲𝗿𝘀𝗵𝗶𝗽 𝗜𝗻𝘀𝗶𝗴𝗵𝘁: Prompts are just the beginning. But 𝗽𝗿𝗼𝗺𝗽𝘁 𝗮𝘂𝗱𝗶𝘁𝗶𝗻𝗴 is what separates smart teams from strategic ones. You don’t need AI that moves fast. You need AI that moves smart. 𝗛𝗼𝘄 𝗮𝗿𝗲 𝘆𝗼𝘂 𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝘁𝗿𝘂𝘀𝘁 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗔𝗜 𝗼𝘂𝘁𝗽𝘂𝘁𝘀? 𝗙𝗼𝗹𝗹𝗼𝘄 𝗺𝗲 for weekly playbooks on leading AI-powered teams. 𝗦𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲 to my newsletter for systems you can apply Monday morning, not someday.