How to Validate AI Responses

Prompting isn’t the hard part anymore. Trusting the output is. You finally get a model to reason step-by-step… And then? You're staring at a polished paragraph, wondering:    > “Is this actually right?”    > “Could this go to leadership?”    > “Can I trust this across markets or functions?” It looks confident. It sounds strategic. But you know better than to mistake that for true intelligence. 𝗛𝗲𝗿𝗲’𝘀 𝘁𝗵𝗲 𝗿𝗶𝘀𝗸: Most teams are experimenting with AI. But few are auditing it. They’re pushing outputs into decks, workflows, and decisions— With zero QA and no accountability layer 𝗛𝗲𝗿𝗲’𝘀 𝘄𝗵𝗮𝘁 𝗜 𝘁𝗲𝗹𝗹 𝗽𝗲𝗼𝗽𝗹𝗲: Don’t just validate the answers. Validate the reasoning. And that means building a lightweight, repeatable system that fits real-world workflows. 𝗨𝘀𝗲 𝘁𝗵𝗲 𝗥.𝗜.𝗩. 𝗟𝗼𝗼𝗽: 𝗥𝗲𝘃𝗶𝗲𝘄 – What’s missing, vague, or risky? 𝗜𝘁𝗲𝗿𝗮𝘁𝗲 – Adjust one thing (tone, data, structure). 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗲 – Rerun and compare — does this version hit the mark? Run it 2–3 times. The best version usually shows up in round two or three, not round one.  𝗥𝘂𝗻 𝗮 60-𝗦𝗲𝗰𝗼𝗻𝗱 𝗢𝘂𝘁𝗽𝘂𝘁 𝗤𝗔 𝗕𝗲𝗳𝗼𝗿𝗲 𝗬𝗼𝘂 𝗛𝗶𝘁 𝗦𝗲𝗻𝗱: • Is the logic sound? • Are key facts verifiable? • Is the tone aligned with the audience and region? • Could this go public without risk? 𝗜𝗳 𝘆𝗼𝘂 𝗰𝗮𝗻’𝘁 𝘀𝗮𝘆 𝘆𝗲𝘀 𝘁𝗼 𝗮𝗹𝗹 𝗳𝗼𝘂𝗿, 𝗶𝘁’𝘀 𝗻𝗼𝘁 𝗿𝗲𝗮𝗱𝘆. 𝗟𝗲𝗮𝗱𝗲𝗿𝘀𝗵𝗶𝗽 𝗜𝗻𝘀𝗶𝗴𝗵𝘁: Prompts are just the beginning. But 𝗽𝗿𝗼𝗺𝗽𝘁 𝗮𝘂𝗱𝗶𝘁𝗶𝗻𝗴 is what separates smart teams from strategic ones. You don’t need AI that moves fast. You need AI that moves smart. 𝗛𝗼𝘄 𝗮𝗿𝗲 𝘆𝗼𝘂 𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝘁𝗿𝘂𝘀𝘁 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗔𝗜 𝗼𝘂𝘁𝗽𝘂𝘁𝘀? 𝗙𝗼𝗹𝗹𝗼𝘄 𝗺𝗲 for weekly playbooks on leading AI-powered teams. 𝗦𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲 to my newsletter for systems you can apply Monday morning, not someday.

Honestly, most AI developers are still stuck in the last century. It blows my mind how few people are aware of Error Analysis. This is *literally* the fastest and most effective way to evaluate AI applications, and most teams are still stuck chasing ghosts. Please, stop tracking generic metrics and follow these steps: 1. Collect failure samples Start reviewing the responses generated by your application. Write notes about each response, especially those that were mistakes. You don't need to format your notes in any specific way. Focus on describing what went wrong with the response. 2. Categorize your notes After you have reviewed a good set of responses, take an LLM and ask it to find common patterns in your notes. Ask it to classify each note based on these patterns. You'll end up with categories covering every type of mistake your application made. 3. Diagnose the most frequent mistakes Begin by focusing on the most common type of mistake. You don't want to waste time working with rare mistakes. Drill into the conversations, inputs, and logs leading to those incorrect samples. Try to understand what might be causing the problems. 4. Design targeted fixes At this point, you want to determine how to eliminate the mistakes you diagnosed in the previous step as quickly and cheaply as possible. For example, you could tweak your prompts, add extra validation rules, find more training data, or modify the model. 5. Automate the evaluation process You need to implement a simple process to rerun an evaluation set through your application and evaluate whether your fixes were effective. My recommendation is to use an LLM-as-a-Judge to run samples through the application, score them with a PASS/FAIL tag, and compute the results. 6. Keep an eye on your metrics Each category you identified during error analysis is a metric you want to track over time. You will get nowhere by obsessing over "relevance", "correctness", "completeness", "coherence", and any other out-of-the-box metrics. Forget about these and focus on the real issues you found.

Tackling Hallucination in LLMs: Mitigation & Evaluation Strategies As Large Language Models (LLMs) redefine how we interact with AI, one critical challenge is hallucination—when models generate false or misleading responses. This issue affects the reliability of LLMs, particularly in high-stakes applications like healthcare, legal, and education. To ensure trustworthiness, it’s essential to adopt robust strategies for mitigating and evaluating hallucination. The workflow outlined above presents a structured approach to addressing this challenge: 1️⃣ Hallucination QA Set Generation Starting with a raw corpus, we process knowledge bases and apply weighted sampling to create diverse, high-quality datasets. This includes generating baseline questions, multi-context queries, and complex reasoning tasks, ensuring a comprehensive evaluation framework. Rigorous filtering and quality checks ensure datasets are robust and aligned with real-world complexities. 2️⃣ Hallucination Benchmarking By pre-processing datasets, answers are categorized as correct or hallucinated, providing a benchmark for model performance. This phase involves tools like classification models and text generation to assess reliability under various conditions. 3️⃣ Hallucination Mitigation Strategies In-Context Learning: Enhancing output reliability by incorporating examples directly in the prompt. Retrieval-Augmented Generation: Supplementing model responses with real-time data retrieval. Parameter-Efficient Fine-Tuning: Fine-tuning targeted parts of the model for specific tasks. By implementing these strategies, we can significantly reduce hallucination risks, ensuring LLMs deliver accurate and context-aware responses across diverse applications. 💡 What strategies do you employ to minimize hallucination in AI systems? Let’s discuss and learn together in the comments!

Happy Friday! This week in #learnwithmz, let’s talk about 𝐀𝐈 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧𝐬 and why PMs need to lean in. As AI features become core to product roadmaps, evaluating AI systems is no longer just a research problem. It's a product responsibility. Whether you're building copilots, agents, search, or agentic systems, you need to know how to measure what “good” looks like. 𝐓𝐨𝐨𝐥𝐬 & 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬 𝐟𝐨𝐫 𝐀𝐈 𝐄𝐯𝐚𝐥𝐬 Ragas: End-to-end evals for RAG pipelines 🔗 https://lnkd.in/g-upbP3p Gaia Eval Harness (Anthropic): Tests groundedness and reasoning in Claude-like models 🔗 https://lnkd.in/ggcasAdQ OpenAI Evals: Structured prompt test harness for model behaviors 🔗 https://lnkd.in/gXNcwvSU Arize AI Phoenix: Evaluation + observability for LLMs in production 🔗 https://lnkd.in/gAb9aguA Giskard: Automated testing for ML model quality and ethics 🔗 https://lnkd.in/gzQ_heQW Bonus read: Aakash Gupta’s breakdown on AI evals is an excellent read https://lnkd.in/gJkCDxFT I have posted before on key evaluation metrics: https://lnkd.in/gx5CBNsG 𝐊𝐞𝐲 𝐀𝐫𝐞𝐚𝐬 𝐭𝐨 𝐖𝐚𝐭𝐜𝐡 (𝐚𝐬 𝐚 𝐏𝐌) Guardrails aren’t optional, they’re product requirements - Groundedness: Is the model hallucinating or based in fact? - Helpfulness: Does it solve the actual user need? - Bias & Harm: How inclusive, fair, and safe are the outputs? - Consistency: Is the model deterministic where it needs to be? - Evaluation Triggers: Can we detect failure modes early? 𝐄𝐱𝐚𝐦𝐩𝐥𝐞 Evaluating an NL2SQL Copilot Goal: User types a question like “Show me the top 5 customers by revenue last quarter” The system should generate correct, optimized SQL against a given schema. 𝐊𝐞𝐲 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐃𝐢𝐦𝐞𝐧𝐬𝐢𝐨𝐧𝐬 - Correctness (Semantic Accuracy) Does the SQL produce the expected result? Is it aligned with schema constraints (e.g., table and column names)? Automate this with unit tests or snapshot comparisons - Executability Does the generated SQL run without error? You can use test DBs or mock query runners - Faithfulness (Groundedness) Does the SQL only use tables and columns present in the schema? Hallucinated column/table = major fail - Performance/Affordability Is the SQL optimized for cost and latency (no SELECT *)? Use static query analysis or query plan inspection - Helpfulness (UX/Intent Match) Does the SQL actually answer the user's intent? This can require human-in-the-loop eval 𝐖𝐡𝐲 𝐭𝐡𝐢𝐬 𝐦𝐚𝐭𝐭𝐞𝐫𝐬 You can’t ship AI responsibly without evals and you can’t evaluate well without cross-functional design. PMs, DS, and Eng need shared language, goals, and metrics. Which eval tools are in your stack or on your radar? Let’s crowdsource some best practices #AI #ProductManagement #LLM #AIEvals #ResponsibleAI #RAG #AIObservability #LearnWithMZ