How to Evaluate Model Performance

I’m jealous of AI Because with a model you can measure confidence Imagine you could do that as a human? Measure how close or far off you are? here's how to measure for technical and non-technical teams For business teams: Run a ‘known answers’ test. Give the model questions or tasks where you already know the answer. Think of it like a QA test for logic. If it can't pass here, it's not ready to run wild in your stack. Ask for confidence directly. Prompt it: “How sure are you about that answer on a scale of 1-10?” Then: “Why might this be wrong?” You'll surface uncertainty the model won't reveal unless asked. Check consistency. Phrase the same request five different ways. Is it giving stable answers? If not, revisit the product strategy for the llm Force reasoning. Use prompts like “Show step-by-step how you got this result.” This lets you audit the logic, not just the output. Great for strategy, legal, and product decisions. For technical teams: Use the softmax output to get predicted probabilities. Example: Model says “fraud” with 92% probability. Use entropy to spot uncertainty. High entropy = low confidence. (Shannon entropy: −∑p log p) Language models Extract token-level log-likelihoods from the model if you have API or model access. These give you the probability of each word generated. Use sequence likelihood to rank alternate responses. Common in RAG and search-ranking setups. For uncertainty estimates, try: Monte Carlo Dropout: Run the same input multiple times with dropout on. Compare outputs. High variance = low confidence. Ensemble models: Aggregate predictions from several models to smooth confidence. Calibration testing: Use a reliability diagram to check if predicted probabilities match actual outcomes. Use Expected Calibration Error (ECE) as a metric. Good models should show that 80% confident = ~80% correct. How to improve confidence (and make it trustworthy) Label smoothing during training Prevents overconfident predictions and improves generalization. Temperature tuning (post-hoc) Adjusts the softmax sharpness to better align confidence and accuracy. Temperature < 1 → sharper, more confident Temperature > 1 → more cautious, less spiky predictions Fine-tuning on domain-specific data Shrinks uncertainty and reduces hedging in model output. Especially effective for LLMs that need to be assertive in narrow domains (legal, medicine, strategy). Use focal loss for noisy or imbalanced datasets. It down-weights easy examples and forces the model to pay attention to harder cases, which tightens confidence on the edge cases. Reinforcement learning from human feedback (RLHF) Aligns the model's reward with correct and confident reasoning. Bottom line: A confident model isn't just better - it's safer, cheaper, and easier to debug. If you’re building workflows or products that rely on AI, but you’re not measuring model confidence, you’re guessing. #AI #ML #LLM #MachineLearning #AIConfidence #RLHF #ModelCalibration

You've built your AI agent... but how do you know it's not failing silently in production? Building AI agents is only the beginning. If you’re thinking of shipping agents into production without a solid evaluation loop, you’re setting yourself up for silent failures, wasted compute, and eventully broken trust. Here’s how to make your AI agents production-ready with a clear, actionable evaluation framework: 𝟭. 𝗜𝗻𝘀𝘁𝗿𝘂𝗺𝗲𝗻𝘁 𝘁𝗵𝗲 𝗥𝗼𝘂𝘁𝗲𝗿 The router is your agent’s control center. Make sure you’re logging: - Function Selection: Which skill or tool did it choose? Was it the right one for the input? - Parameter Extraction: Did it extract the correct arguments? Were they formatted and passed correctly? ✅ Action: Add logs and traces to every routing decision. Measure correctness on real queries, not just happy paths. 𝟮. 𝗠𝗼𝗻𝗶𝘁𝗼𝗿 𝘁𝗵𝗲 𝗦𝗸𝗶𝗹𝗹𝘀 These are your execution blocks; API calls, RAG pipelines, code snippets, etc. You need to track: - Task Execution: Did the function run successfully? - Output Validity: Was the result accurate, complete, and usable? ✅ Action: Wrap skills with validation checks. Add fallback logic if a skill returns an invalid or incomplete response. 𝟯. 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗲 𝘁𝗵𝗲 𝗣𝗮𝘁𝗵 This is where most agents break down in production: taking too many steps or producing inconsistent outcomes. Track: - Step Count: How many hops did it take to get to a result? - Behavior Consistency: Does the agent respond the same way to similar inputs? ✅ Action: Set thresholds for max steps per query. Create dashboards to visualize behavior drift over time. 𝟰. 𝗗𝗲𝗳𝗶𝗻𝗲 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 𝗠𝗲𝘁𝗿𝗶𝗰𝘀 𝗧𝗵𝗮𝘁 𝗠𝗮𝘁𝘁𝗲𝗿 Don’t just measure token count or latency. Tie success to outcomes. Examples: - Was the support ticket resolved? - Did the agent generate correct code? - Was the user satisfied? ✅ Action: Align evaluation metrics with real business KPIs. Share them with product and ops teams. Make it measurable. Make it observable. Make it reliable. That’s how enterprises scale AI agents. Easier said than done.

Honestly, most AI developers are still stuck in the last century. It blows my mind how few people are aware of Error Analysis. This is *literally* the fastest and most effective way to evaluate AI applications, and most teams are still stuck chasing ghosts. Please, stop tracking generic metrics and follow these steps: 1. Collect failure samples Start reviewing the responses generated by your application. Write notes about each response, especially those that were mistakes. You don't need to format your notes in any specific way. Focus on describing what went wrong with the response. 2. Categorize your notes After you have reviewed a good set of responses, take an LLM and ask it to find common patterns in your notes. Ask it to classify each note based on these patterns. You'll end up with categories covering every type of mistake your application made. 3. Diagnose the most frequent mistakes Begin by focusing on the most common type of mistake. You don't want to waste time working with rare mistakes. Drill into the conversations, inputs, and logs leading to those incorrect samples. Try to understand what might be causing the problems. 4. Design targeted fixes At this point, you want to determine how to eliminate the mistakes you diagnosed in the previous step as quickly and cheaply as possible. For example, you could tweak your prompts, add extra validation rules, find more training data, or modify the model. 5. Automate the evaluation process You need to implement a simple process to rerun an evaluation set through your application and evaluate whether your fixes were effective. My recommendation is to use an LLM-as-a-Judge to run samples through the application, score them with a PASS/FAIL tag, and compute the results. 6. Keep an eye on your metrics Each category you identified during error analysis is a metric you want to track over time. You will get nowhere by obsessing over "relevance", "correctness", "completeness", "coherence", and any other out-of-the-box metrics. Forget about these and focus on the real issues you found.

Happy Friday! This week in #learnwithmz, let’s talk about 𝐀𝐈 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧𝐬 and why PMs need to lean in. As AI features become core to product roadmaps, evaluating AI systems is no longer just a research problem. It's a product responsibility. Whether you're building copilots, agents, search, or agentic systems, you need to know how to measure what “good” looks like. 𝐓𝐨𝐨𝐥𝐬 & 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬 𝐟𝐨𝐫 𝐀𝐈 𝐄𝐯𝐚𝐥𝐬 Ragas: End-to-end evals for RAG pipelines 🔗 https://lnkd.in/g-upbP3p Gaia Eval Harness (Anthropic): Tests groundedness and reasoning in Claude-like models 🔗 https://lnkd.in/ggcasAdQ OpenAI Evals: Structured prompt test harness for model behaviors 🔗 https://lnkd.in/gXNcwvSU Arize AI Phoenix: Evaluation + observability for LLMs in production 🔗 https://lnkd.in/gAb9aguA Giskard: Automated testing for ML model quality and ethics 🔗 https://lnkd.in/gzQ_heQW Bonus read: Aakash Gupta’s breakdown on AI evals is an excellent read https://lnkd.in/gJkCDxFT I have posted before on key evaluation metrics: https://lnkd.in/gx5CBNsG 𝐊𝐞𝐲 𝐀𝐫𝐞𝐚𝐬 𝐭𝐨 𝐖𝐚𝐭𝐜𝐡 (𝐚𝐬 𝐚 𝐏𝐌) Guardrails aren’t optional, they’re product requirements - Groundedness: Is the model hallucinating or based in fact? - Helpfulness: Does it solve the actual user need? - Bias & Harm: How inclusive, fair, and safe are the outputs? - Consistency: Is the model deterministic where it needs to be? - Evaluation Triggers: Can we detect failure modes early? 𝐄𝐱𝐚𝐦𝐩𝐥𝐞 Evaluating an NL2SQL Copilot Goal: User types a question like “Show me the top 5 customers by revenue last quarter” The system should generate correct, optimized SQL against a given schema. 𝐊𝐞𝐲 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐃𝐢𝐦𝐞𝐧𝐬𝐢𝐨𝐧𝐬 - Correctness (Semantic Accuracy) Does the SQL produce the expected result? Is it aligned with schema constraints (e.g., table and column names)? Automate this with unit tests or snapshot comparisons - Executability Does the generated SQL run without error? You can use test DBs or mock query runners - Faithfulness (Groundedness) Does the SQL only use tables and columns present in the schema? Hallucinated column/table = major fail - Performance/Affordability Is the SQL optimized for cost and latency (no SELECT *)? Use static query analysis or query plan inspection - Helpfulness (UX/Intent Match) Does the SQL actually answer the user's intent? This can require human-in-the-loop eval 𝐖𝐡𝐲 𝐭𝐡𝐢𝐬 𝐦𝐚𝐭𝐭𝐞𝐫𝐬 You can’t ship AI responsibly without evals and you can’t evaluate well without cross-functional design. PMs, DS, and Eng need shared language, goals, and metrics. Which eval tools are in your stack or on your radar? Let’s crowdsource some best practices #AI #ProductManagement #LLM #AIEvals #ResponsibleAI #RAG #AIObservability #LearnWithMZ

As we scale GenAI from demos to real-world deployment, one thing becomes clear: 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗱𝗮𝘁𝗮𝘀𝗲𝘁𝘀 𝗰𝗮𝗻 𝗺𝗮𝗸𝗲 𝗼𝗿 𝗯𝗿𝗲𝗮𝗸 𝗮 𝗚𝗲𝗻𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺. A model can be trained on massive amounts of data, but that doesn’t guarantee it understands context, nuance, or intent at inference time. You can teach a student all the textbook theory in the world. But unless you ask the right questions, in the right setting, under realistic pressure, you’ll never know what they truly grasp. This snapshot outlines the 6 dataset types that AI teams use to rigorously evaluate systems at every stage of maturity: The Evaluation Spectrum 1. 𝐐𝐮𝐚𝐥𝐢𝐟𝐢𝐞𝐝 𝐚𝐧𝐬𝐰𝐞𝐫𝐬 Meaning: Expert-reviewed responses Use: Measure answer quality (groundedness, coherence, etc.) Goal: High-quality, human-like responses 2. 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 Meaning: AI-generated questions and answers Use: Test scale and performance Goal: Maximize response accuracy, retrieval quality, and tool use precision 3. 𝐀𝐝𝐯𝐞𝐫𝐬𝐚𝐫𝐢𝐚𝐥 Meaning: Malicious or risky prompts (e.g., jailbreaks) Use: Ensure safety and resilience Goal: Avoid unsafe outputs 4. 𝐎𝐎𝐃 (𝐎𝐮𝐭 𝐨𝐟 𝐃𝐨𝐦𝐚𝐢𝐧) Meaning: Unusual or irrelevant topics Use: See how well the model handles unfamiliar territory Goal: Avoid giving irrelevant or misleading answers 5. 𝐓𝐡𝐮𝐦𝐛𝐬 𝐝𝐨𝐰𝐧 Meaning: Real examples where users rated answers poorly Use: Identify failure modes Goal: Internal review, error analysis 6. 𝐏𝐑𝐎𝐃 Meaning: Cleaned, real user queries from deployed systems Use: Evaluate live performance Goal: Ensure production response quality This layered approach is essential for building: • Trustworthy AI • Measurable safety • Meaningful user experience Most organizations still rely on "accuracy-only" testing. But GenAI in production demands multi-dimensional evaluation — spanning risk, relevance, and realism. If you’re deploying GenAI at scale, ask: Are you testing the right things with the right datasets? Let’s sharpen the tools we use to measure intelligence. Because better testing = better AI. 👇 Would love to hear how you’re designing your eval pipelines. #genai #evaluation #llmops #promptengineering #aiarchitecture #openai