Measuring Response Quality in Language Models

Evaluating LLMs accurately/reliably is difficult, but we can usually automate the evaluation process with another (more powerful) LLM... Automatic metrics: Previously, generative text models were most commonly evaluated using automatic metrics like ROUGE and BLEU, which simply compare how well a model’s output matches a human-written target resopnse. In particular, BLEU score was commonly used to evaluatate machine translation models, while ROUGE was most often used for evaluating summarization models. Serious limitations: With modern LLMs, researchers began to notice that automatic metrics did a poor job of comprehensively capturing the quality of an LLM’s generations. Oftentimes, ROUGE scores were poorly correlated with human preferences—higher scores don’t seem to indicate a better generation/summary [1]. This problem is largely due to the open-ended nature of most tasks solved with LLMs. There can be many good responses to a prompt. LLM-as-a-judge [2] leverages a powerful LLM (e.g., GPT-4) to evaluate the quality of an LLM’s output. To evaluate an LLM with another LLM, there are three basic structures or strategies that we can employ: (1) Pairwise comparison: The LLM is shown a question with two responses and asked to choose the better response (or declare a tie). This approach was heavily utilized by models like Alpaca/Vicuna to evaluate model performance relative to proprietary LLMS like ChatGPT. (2) Single-answer grading: The LLM is shown a response with a single answer and asked to provide a score for the answer. This strategy is less reliable than pairwise comparison due to the need to assign an absolute score to the response. However, authors in [2] observe that GPT-4 can nonetheless assign relatively reliable/meaningful scores to responses. (3) Reference-guided grading: The LLM is provided a reference answer to the problem when being asked to grade a response. This strategy is useful for complex problems (e.g., reasoning or math) in which even GPT-4 may struggle with generating a correct answer. In these cases, having direct access to a correct response may aid the grading process. “LLM-as-a-judge offers two key benefits: scalability and explainability. It reduces the need for human involvement, enabling scalable benchmarks and fast iterations.” - from [2] Using MT-bench, authors in [2] evaluate the level of agreement between LLM-as-a-judge and humans (58 expert human annotators), where we see that there is a high level of agreement between these strategies. Such a finding caused this evaluation strategy to become incredibly popular for LLMs—it is currently the most widely-used and effective alternative to human evaluation. However, LLM-as-a-judge does suffer from notable limititations (e.g., position bias, verbosity bias, self-enhancement bias, etc.) that should be considered when interpretting data.

What’s the best way to benchmark the performance of AI agents? Traditional metrics like ROUGE are becoming increasingly difficult to rely on due to the growing complexity of conversational agents. Challenges such as evaluating semantic correctness, detecting hallucinations, and assessing conversational flow require more nuanced approaches. One method I’ve found helpful is the 𝐄2𝐄 (𝐄𝐧𝐝-𝐭𝐨-𝐄𝐧𝐝) 𝐁𝐞𝐧𝐜𝐡𝐦𝐚𝐫𝐤 (see here: https://lnkd.in/e7epy2tg) , which uses semantic similarity metrics to compare chatbot responses to human-provided “golden answers.” This approach, also discussed in a 2023 study, focuses on cosine similarity between embeddings generated by models like the Universal Sentence Encoder (USE) and Sentence Transformer (ST). The paper does a good job of testing this method in real-world scenarios, using outputs from a product support chatbot and analyzing its effectiveness compared to traditional metrics like ROUGE. Key insights from their analysis include: • 𝐈𝐦𝐩𝐫𝐨𝐯𝐞𝐝 𝐬𝐞𝐧𝐬𝐢𝐭𝐢𝐯𝐢𝐭𝐲: The E2E Benchmark, particularly with ST embeddings, was better at capturing subtle improvements in response quality than ROUGE. For example, enhanced prompts improved the chatbot’s outputs, and the E2E scores reflected this improvement, while ROUGE showed inconsistent results. • 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐭𝐨 𝐟𝐚𝐜𝐭𝐮𝐚𝐥 𝐚𝐜𝐜𝐮𝐫𝐚𝐜𝐲: By emphasizing semantic similarity, the benchmark effectively identified responses misaligned with golden answers, offering a more reliable measure of factual accuracy in knowledge-intensive tasks like product support. • 𝐑𝐨𝐛𝐮𝐬𝐭𝐧𝐞𝐬𝐬 𝐚𝐠𝐚𝐢𝐧𝐬𝐭 𝐧𝐨𝐢𝐬𝐞: When tested against random outputs, the E2E Benchmark reliably scored meaningless responses close to zero, reinforcing its robustness compared to word-based metrics. While no single benchmark is perfect, this method is particularly useful for tracking long-term performance, spotting issues in conversational flows, and dynamically updating prompts to make AI agents more adaptive. The ability to align scores with human preferences and detect subtle errors makes it a strong candidate for evaluating conversational agents. What do you think? Have you used this or other methods to evaluate conversational AI agents? I’d love to hear what’s worked for you. Photo source: https://lnkd.in/eT_Cgyyd

Unlocking the Next Era of RAG System Evaluation: Insights from the Latest Comprehensive Survey Retrieval-Augmented Generation (RAG) has become a cornerstone for enhancing large language models (LLMs), especially when accuracy, timeliness, and factual grounding are critical. However, as RAG systems grow in complexity-integrating dense retrieval, multi-source knowledge, and advanced reasoning-the challenge of evaluating their true effectiveness has intensified. A recent survey from leading academic and industrial research organizations delivers the most exhaustive analysis yet of RAG evaluation in the LLM era. Here are the key technical takeaways: 1. Multi-Scale Evaluation Frameworks The survey dissects RAG evaluation into internal and external dimensions. Internal evaluation targets the core components-retrieval and generation-assessing not just their standalone performance but also their interactions. External evaluation addresses system-wide factors like safety, robustness, and efficiency, which are increasingly vital as RAG systems are deployed in real-world, high-stakes environments. 2. Technical Anatomy of RAG Systems Under the hood, a typical RAG pipeline is split into two main sections: - Retrieval: Involves document chunking, embedding generation, and sophisticated retrieval strategies (sparse, dense, hybrid, or graph-based). Preprocessing such as corpus construction and intent recognition is essential for optimizing retrieval relevance and comprehensiveness. - Generation: The LLM synthesizes retrieved knowledge, leveraging advanced prompt engineering and reasoning techniques to produce contextually faithful responses. Post-processing may include entity recognition or translation, depending on the use case. 3. Diverse and Evolving Evaluation Metrics The survey catalogues a wide array of metrics: - Traditional IR Metrics: Precision@K, Recall@K, F1, MRR, NDCG, MAP for retrieval quality. - NLG Metrics: Exact Match, ROUGE, BLEU, METEOR, BertScore, and Coverage for generation accuracy and semantic fidelity. - LLM-Based Metrics: Recent trends show a rise in LLM-as-judge approaches (e.g., RAGAS, Databricks Eval), semantic perplexity, key point recall, FactScore, and representation-based methods like GPTScore and ARES. These enable nuanced, context-aware evaluation that better aligns with real-world user expectations. 4. Safety, Robustness, and Efficiency The survey highlights specialized benchmarks and metrics for: - Safety: Evaluating robustness to adversarial attacks (e.g., knowledge poisoning, retrieval hijacking), factual consistency, privacy leakage, and fairness. - Efficiency: Measuring latency (time to first token, total response time), resource utilization, and cost-effectiveness-crucial for scalable deployment.

As we scale GenAI from demos to real-world deployment, one thing becomes clear: 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗱𝗮𝘁𝗮𝘀𝗲𝘁𝘀 𝗰𝗮𝗻 𝗺𝗮𝗸𝗲 𝗼𝗿 𝗯𝗿𝗲𝗮𝗸 𝗮 𝗚𝗲𝗻𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺. A model can be trained on massive amounts of data, but that doesn’t guarantee it understands context, nuance, or intent at inference time. You can teach a student all the textbook theory in the world. But unless you ask the right questions, in the right setting, under realistic pressure, you’ll never know what they truly grasp. This snapshot outlines the 6 dataset types that AI teams use to rigorously evaluate systems at every stage of maturity: The Evaluation Spectrum 1. 𝐐𝐮𝐚𝐥𝐢𝐟𝐢𝐞𝐝 𝐚𝐧𝐬𝐰𝐞𝐫𝐬 Meaning: Expert-reviewed responses Use: Measure answer quality (groundedness, coherence, etc.) Goal: High-quality, human-like responses 2. 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 Meaning: AI-generated questions and answers Use: Test scale and performance Goal: Maximize response accuracy, retrieval quality, and tool use precision 3. 𝐀𝐝𝐯𝐞𝐫𝐬𝐚𝐫𝐢𝐚𝐥 Meaning: Malicious or risky prompts (e.g., jailbreaks) Use: Ensure safety and resilience Goal: Avoid unsafe outputs 4. 𝐎𝐎𝐃 (𝐎𝐮𝐭 𝐨𝐟 𝐃𝐨𝐦𝐚𝐢𝐧) Meaning: Unusual or irrelevant topics Use: See how well the model handles unfamiliar territory Goal: Avoid giving irrelevant or misleading answers 5. 𝐓𝐡𝐮𝐦𝐛𝐬 𝐝𝐨𝐰𝐧 Meaning: Real examples where users rated answers poorly Use: Identify failure modes Goal: Internal review, error analysis 6. 𝐏𝐑𝐎𝐃 Meaning: Cleaned, real user queries from deployed systems Use: Evaluate live performance Goal: Ensure production response quality This layered approach is essential for building: • Trustworthy AI • Measurable safety • Meaningful user experience Most organizations still rely on "accuracy-only" testing. But GenAI in production demands multi-dimensional evaluation — spanning risk, relevance, and realism. If you’re deploying GenAI at scale, ask: Are you testing the right things with the right datasets? Let’s sharpen the tools we use to measure intelligence. Because better testing = better AI. 👇 Would love to hear how you’re designing your eval pipelines. #genai #evaluation #llmops #promptengineering #aiarchitecture #openai

Evaluating ML is easy. Use metrics like AUC or MSE. But what about LLMs? ↓ LLM evaluation is not easy. Unless the task is a simple classification like flagging an email as ham or spam, it's difficult since... ☒ Manual review is costly ☒ Task input/output is open-ended ☒ Benchmarks like MMLU generic for custom cases So, how do you evaluate on a scale? Here are 3 strategies to employ ↓ 𝟭. 𝗦𝗲𝗺𝗮𝗻𝘁𝗶𝗰 𝗦𝗶𝗺𝗶𝗹𝗮𝗿𝗶𝘁𝘆 Two texts with similar meanings will have embedding vectors that are close together. Use cosine similarity to compare ideal output samples with LLM-generated responses. A higher score indicates a better response. 𝟮. 𝗟𝗟𝗠-𝗮𝘀-𝗮-𝗝𝘂𝗱𝗴𝗲 Getting a human to evaluate LLM output is costly. So, create an LLM agent that mimics a human reviewer. Create a prompt with a grading rubric with examples. Get the reviewer agent to evaluate the main agent on a scale. 𝟯. 𝗘𝘅𝗽𝗹𝗶𝗰𝗶𝘁 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 Add a UI to the chat interface to gather thumbs up/down and re-generation feedback. This helps measure the quality of the output from the users themselves. With this feedback loop in place, optimize your LLM system with prompt engineering, fine-tuning, RAG, and other techniques. Let's bounce ideas around. How do you evaluate LLM? Drop a comment ↓

If you’ve ever tried to measure the quality of a large language model’s (LLM) responses, you know it’s no small feat. Traditional metrics - like BLEU scores for translation - don’t always capture the nuances of complex, human-like responses. That’s where the concept of using LLMs themselves as judges comes in. A recent paper I looked into provides a really thoughtful structure for thinking about LLM-as-a-judge methods. It breaks the problem down into three clear angles: - What to Judge: Which elements of a response are you evaluating? Accuracy? Creativity? Coherence? - How to Judge: Are you scoring answers directly, comparing them to a reference, or having the model rank multiple responses? - Where to Judge: Is the evaluation happening inline as the response is generated, or is it done after the fact on a separate platform or dataset? They also present several benchmarks that illustrate various methods of having LLMs assess responses. Even if these benchmarks aren’t directly plug-and-play for your specific domain (especially in enterprise scenarios with highly specialized data), they can help you understand patterns and best practices. What I found particularly useful was seeing how LLM-as-a-judge can sometimes outperform standard metrics - if properly calibrated. Calibration is key, because these models can still be biased or drift in their judging standards over time. If you’re just starting out with the idea of LLMs evaluating their own outputs, this taxonomy provides a great roadmap. It can help you figure out what’s feasible, where to start, and which pitfalls to watch out for. Have any of you experimented with letting an LLM serve as its own judge? What approaches or challenges have you encountered? #innovation #technology #future #management #startups

How does the GitLab team evaluate LLMs? GitLab's evaluation of large language models (LLMs) involves a structured, multi-step process to ensure quality, performance, and reliability. They start by creating a prompt library with ideal answers (ground truth), avoiding customer data, to simulate real user interactions. - Prompt Library Creation: GitLab designs prompts specific to their features to accurately test model responses. - Baseline Testing: Models are evaluated against ground truth using metrics like Cosine Similarity, setting a baseline performance level. - Feature Development: Validated models are integrated into GitLab's features, with daily tests to ensure consistent improvements. - Iterative Refinement: Smaller, targeted tests refine model behaviors without affecting other functionalities. - Full Validation: The Centralized Evaluation Framework (CEF) assesses each model iteration, ensuring alignment with performance benchmarks across scenarios. As eval driven development becomes more mainstream, we'd see more such usecases around evals come up. Link to the article in comments.

OpenAI Releases SimpleQA: A New AI Benchmark that Measures the Factuality of Language Models OpenAI recently open-sourced SimpleQA: a new benchmark that measures the factuality of responses generated by language models. SimpleQA is unique in its focus on short, fact-seeking questions with a single, indisputable answer, making it easier to evaluate the factual correctness of model responses. Unlike other benchmarks that often become outdated or saturated over time, SimpleQA was designed to remain challenging for the latest AI models. The questions in SimpleQA were created in an adversarial manner against responses from GPT-4, ensuring that even the most advanced language models struggle to answer them correctly. The benchmark contains 4,326 questions spanning various domains, including history, science, technology, art, and entertainment, and is built to be highly evaluative of both model precision and calibration. The importance of SimpleQA lies in its targeted evaluation of language models’ factual abilities. In a landscape where many benchmarks have been “solved” by recent models, SimpleQA is designed to remain challenging even for frontier models like GPT-4 and Claude. For instance, models such as GPT-4o scored only about 38.4% in terms of correct answers, highlighting the benchmark’s ability to probe areas where even advanced models face difficulties. Other models, including Claude-3.5, performed similarly or worse, indicating that SimpleQA poses a consistent challenge across model types. This benchmark, therefore, provides valuable insights into the calibration and reliability of language models—particularly their ability to discern when they have enough information to answer confidently and correctly... Read the full article here: https://lnkd.in/gT-ME4Ui Paper: https://lnkd.in/g7u5Zf_C GitHub Page: https://lnkd.in/gDtCiHf7 Details: https://lnkd.in/gAXjsjf3 OpenAI Jason Wei Karina Nguyen Hyung Won Chung Spencer Papay Amelia (Mia) Glaese William (Liam) Fedus

I recently came across an intriguing paper titled A Framework for Robust Cognitive Evaluation of LLMs that offers a fresh perspective on how we can assess the “cognitive” abilities of large language models (LLMs). This research, conducted by a multidisciplinary team from the University of Minnesota, Hamline University, and the University of Wisconsin-Stout, introduces a new experimental pipeline called COGNITIVEVAL. Key Insights from the Paper: Bridging Cognitive Science and AI: The study tackles the challenge of understanding LLMs beyond mere language generation. It leverages classic cognitive science experiments such as the Wisconsin Card Sorting Task, Flanker Task, Digit Span Tasks, and the DRM task to explore how these models process information, make decisions, and handle memory. Innovative Methodology: COGNITIVEVAL introduces two major innovations: 1. Automatic Prompt Permutations: By generating diverse prompt variations, the framework minimizes biases associated with specific prompt formulations. 2. Dual Metric Collection: The approach captures both the LLMs’ direct responses and their internal probability estimates, offering a more nuanced evaluation of model confidence and performance. Addressing the Evaluation Gap: Traditional methods for evaluating LLMs often overlook the intricacies of cognitive processes. This framework aims to provide a standardized way to measure aspects like short-term memory, working memory, and executive function—areas where LLMs have shown surprising strengths and notable weaknesses. Findings and Implications: The experiments reveal that while LLMs demonstrate robust short-term memory, they tend to struggle with tasks that require working memory and cognitive flexibility. These insights not only deepen our understanding of LLM behavior but also pave the way for further interdisciplinary research between AI and cognitive science. This paper is a significant step toward developing a comprehensive evaluation framework that can help researchers better interpret the internal “thought” processes of LLMs. It’s exciting to see such innovative work that could reshape how we benchmark and understand AI models. #AI #CognitiveScience #LLMs #ResearchInnovation #InterdisciplinaryResearch

Large Language Models (#LLMs) are rapidly transforming various tasks, from code generation to question answering. Their ease of use, leveraging pre-trained models and contextual data, makes them a powerful tool. But a key question remains: How do we ensure the accuracy and quality of their outputs? Google’s AutoSxS is one example, which performs LLM responses evaluation using an LLM. It compares the output of two models and identifies which model provides better response. It is a simple use case. Complex scenarios that involves a chain of inputs and responses demand a more granular approach. Evaluating each step (span) fosters transparency and trust in LLM outputs. It also aids troubleshooting, prompt/data improvement, and model selection. Arize AI provides an LLM #observability and #evaluation platform to support LLM applications in production. #Arize offers an open source package named #Phoenix, which provides a suite of evaluators covering areas like hallucination, retrieval, code generation, and more. These evaluators are tested against benchmark datasets such as WikiQA, WikiToxic, MS Marco, Hallucination QA Dataset, Hallucination #RAG Dataset, WikiSQL, and others. I find tools like this very promising and ease my heart about adopting LLM apps with more confidence. But here's the catch: As Chip Huyen aptly points out in her recent post, even #AI evaluators need evaluation! We need to ensure these "judges" are themselves benchmarked and continuously vetted. ◆ How are you evaluating your LLM applications? ◆ What are your biggest challenges in ensuring LLM output quality?