Evaluating AI-Generated Content With LLMs

Using powerful LLMs (GPT-4) as an evaluator for smaller models is becoming the de facto standard. However, relying on closed-source models is suboptimal due to missing control, transparency, and versioning. 🤔 The recent paper "Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models" shows that open LLMs can match GPT-4 evaluation skills. 🚀 🔥𝗣𝗿𝗼𝗺𝗲𝘁𝗵𝗲𝘂𝘀 𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 1️⃣ Created a new dataset with 1000 scoring rubrics, 20K instructions (20 each), and 100K responses with feedback scores (1-5) generated by GPT-4 (5 each). → 100k training samples 2️⃣ Fine-tuned Llama-2-Chat-13B on this dataset (1️⃣) to generate the feedback (Prometheus 🔥) 3️⃣ Evaluated Prometheus on seen and unseen rubrics (including MT Bench), comparing correlation with human scores and GPT-4 scores ✨𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀 🥇 Scores a Pearson correlation of 0.897 with human evaluators, on par with GPT-4 (0.882), and outperforms GPT-3.5. (0.392) 🧑⚖️ Can be used a replacement of GPT-4 for LLM-as-a-Judge 🧬 High correlation with GPT-4 → due to imitation learning? 🔢 Requires 4 components in the input: prompt, generation to evaluate, a score rubric, and a reference generation. 😍 Prometheus can be further improved on training on customized rubrics and feedback, e.g. company specific domains 🧠 Can be used as a Reward Model for RLHF or for DPO to create preference pairs. 🤗 Dataset and Model available on Hugging Face Paper: https://lnkd.in/eXx-n_tx Dataset: https://lnkd.in/e8gVRGm4 Model: https://lnkd.in/eF9tKiTc Kudos to the researchers for this contribution to make AI more explainable, reproducible, and open! 🤗

A proposed qualitative evaluation framework for Generative AI writing tools: This post is my first draft of an evaluation framework for assessing generative AI tools (e.g. Claude, ChatGPT, Gemini). It's something I’ve been working on with Ryan Low — originally in the interest of selecting the best option for Rotational. At some point we realized sharing these ideas might help us and others out there trying to pick the best AI solution for your company's writing needs. We want to be clear that this is not another LLM benchmarking tool. It's not about picking the solution that can count the r's in strawberry or repeatably do long division. This is more about the everyday human experience of using AI tools for our jobs, doing the kinds of things we do all day solving our customers' problems 🙂. We're trying to zoom in on things that directly impact our productivity, efficiency, and creativity. Do these resonate with anyone else out there? Has anyone else tried to do something like this? What other things would you add? Proposed Qualitative Evaluation Criteria 1 - Trust and Accuracy Do I trust it? How often does it say things that I know to be incorrect?  Do I feel safe? Do I understand how my data is being used when I interact with it? 2 - Autonomous Capabilities How much work will it do on my behalf? What kinds of research and summarization tasks will it do for me? Will it research candidates for me and draft targeted emails? Will it read documents from our corporate document drive and use the content to help us develop proposals? Will it review a technical paper, provided a URL? 3 - Context Management and Continuity How well does the tool maintain our conversation context? Not to sound silly, but does the tool remember me? Is it caching stuff? Is there a way for me to upload information about myself into the user interface so that I don’t have to continually reintroduce myself? Does it offer a way to group our conversations by project or my train of thought? Does it remember our past conversations? How far back? Can I get it to understand time from my perspective? 4 - User Experience Does the user interface feel intuitive? 5 - Images How does it do with images? Is it good at creating the kind of images that I need? Can the images it generates be used as-is or do they require modification? 6 - Integrations Does it integrate with our other tools (e.g. for project management, for video conferences, for storing documents, for sales, etc)? 7 - Trajectory Is it getting better? Does the tool seem to be improving based on community feedback? Am I getting better at using it?

Using #LLM Outputs Across Different #AI Platforms: Terms of Service Analysis As AI tools become increasingly integrated into workflows, a crucial question arises: Does using one LLM’s output and pasting it into another violate terms of service? This report examines the legal and policy implications of transferring AI-generated content across platforms like #Grok, #Perplexity, #ChatGPT, and #Google #Gemini. 🚨 Key Findings 🔹 Perplexity AI – Among the most restrictive, claiming ownership of API outputs and prohibiting copying, caching, or creating derivative works. Their restrictive policies align with their “answer engine” business model and ongoing copyright lawsuits from publishers like Dow Jones. 🔹 Google Gemini – Similar restrictions on redistribution, but more transparency with citation metadata. Google differentiates between free and paid API tiers, impacting how user data is used. 🔹 Grok (xAI) – More permissive, allowing broader use of outputs, provided users attribute Grok as the source. This aligns with Elon Musk’s stance on AI openness. 🔹 ChatGPT (OpenAI) – Unclear stance on output ownership. However, legal precedents suggest OpenAI does not have strong intellectual property claims on ChatGPT’s outputs, though terms of service may still restrict certain uses. ⚠️ Potential Consequences Violating an LLM’s terms could lead to: ❌ Account suspension or bans ⚖️ Legal action in extreme cases 🚀 Risks of “jailbreaking” if it circumvents intended platform controls Conclusion: Copy-pasting outputs across LLMs may violate terms on some platforms (especially Perplexity and Gemini), while others (like Grok) are more lenient. To ensure compliance, always review the latest TOS before using AI-generated content across multiple platforms. 📌 What are your thoughts on AI-generated content ownership? Should LLM outputs be freely transferable? Drop your insights below! 👇 #AI #LLM #ArtificialIntelligence #LegalTech #MachineLearning #AICompliance #PerplexityAI #ChatGPT #GoogleGemini #GrokAI #AIRegulations

LLM-as-a-Judge is a dangerous way to do evaluation. In most cases, it just doesn't work. While it gives a false impression of having a grasp on your system's performance, it lures you with general metrics such as correctness, faithfulness, or completeness. They hide several complexities: - What does "completeness" mean for your application? In the case of a marketing AI assistant, what characterizes a complete post from an incomplete one? If the score goes higher, does it mean that the post is better? - Often, these metrics are scores between 1-5. But what is the difference between a 3 and a 4? Does a 5 mean the output is perfect? What is perfect, then? - If you "calibrate" the LLM-as-a-judge with scores given by users during a test session, how do you ensure the LLM scoring matches user expectations? If I arbitrarily set all scores to 4, will I perform better than your model? However, if LLM-as-a-judge is limited, it doesn't mean it's impossible to evaluate your AI system. Here are some strategies you can adopt: - Online evaluation is the new king in the GenAI era Log and trace LLM outputs, retrieved chunks, routing… Each step of the process. Link it to user feedback as binary classification: was the final output good or bad? Then take a look at the data. Yourself, no LLM. Take notes and try to find patterns among good and bad traces. At this stage, you can use an LLM to help you find clusters within your notes, not the data. After taking this time, you'll already have some clues about how to improve the system. - Evaluate deterministic steps that come before the final output Was the retrieval system performant? Did the router correctly categorize the initial query? Those steps in the agentic system are deterministic, meaning you can evaluate them precisely: Retriever: Hit Rate@k, Mean Reciprocal Rank, Precision, Recall Router: Precision, Recall, F1-Score Create a small benchmark, synthetically or not, to evaluate offline those steps. It enables you to improve them later on individually (hybrid search instead of vector search, fine-tune a small classifier instead of relying on LLMs…) - Don't use tools that promise to externalize evaluation Your evaluation system is your moat. If you externalize it, not only will you have no control over your AI application, but you also won't be able to improve it. This is your problem, your users, your revenue. Evaluate your system. Not a generic one. All problems are different. Yours is unique as well. ... Those are some unequivocal ideas proposed by the AI community. Yet, I still see AI projects relying on LLM-as-a-judge and generic metrics among companies. Being able to evaluate your system gives you the power to improve it. So take the time to create the perfect evaluation for your use case.

AI Evaluation Frameworks As AI systems evolve, one major challenge remains: how do we measure their performance accurately? This is where the concept of “AI Judges” comes in, from LLMs to autonomous agents and even humans. Here is how each type of judge works - 1. LLM-as-a-Judge - An LLM acts as an evaluator, comparing answers or outputs from different models and deciding which one is better. - It focuses on text-based reasoning and correctness - great for language tasks, but limited in scope. -Key Insight : LLMs can not run code or verify real-world outcomes. They are best suited for conversational or reasoning-based evaluations. 2. Agent-as-a-Judge - An autonomous agent takes evaluation to the next level. - It can execute code, perform tasks, measure accuracy, and assess efficiency, just like a real user or system would. -Key Insight : This allows for scalable, automated, and realistic testing, making it ideal for evaluating AI agents and workflows in action. 3. Human-as-a-Judge - Humans manually test and observe agents to determine which performs better. - They offer detailed and accurate assessments, but the process is slow and hard to scale. - Key Insight : While humans remain the gold standard for nuanced judgment, agent-based evaluation is emerging as the scalable replacement for repetitive testing. The future of AI evaluation is shifting - from static text comparisons (LLM) to dynamic, real-world testing (Agent). Humans will still guide the process, but AI agents will soon take over most of the judging work. If you are building or testing AI systems, start adopting Agent-as-a-Judge methods. They will help you evaluate performance faster, more accurately, and at scale.

Interview Question: How do you evaluate LLMs for hallucination? First, let define Hallucination. Its when the model confidently generates factually incorrect or unverifiable content. Typically quantifying this behavior requires combining automatic metrics, human judgment, and task-specific grounding. The most straightforward way to detect hallucinations is by comparing model output to a ground truth reference or trusted corpus. Use metrics like BLEU, ROUGE, or BERTScore to measure similarity between generated text and ground-truth reference. However, these are often insufficient, since hallucinated answers can look similar lexically. Another approach around Fact Verification is using NER and dependecny parsing to extract subject–predicate–object triple that form the basis of claims. E.g. “Einstein was born in Ulm” → (Einstein, born in, Ulm). Then using NLI (natural-lauguage inference) models to check if the claim is supported, contradicted, or not mentioned in the text the LLM the is grounded on. Another is to use LLMs as judges, where we use an LLM (or a group of them) and prompt it to - “Compare the answer with the source text and identify unsupported or incorrect claims.” This technique works surprisingly well, but it must be regularly validated with human annotators. When evaluating LLMs in retrieval-augmented generation (RAG) systems (like question-answering over documents), hallucination means generating facts not supported by retrieved passages. Here we use faithfulness score i.e. the fraction of generated claims that can be directly traced to retrieved evidence. You can automatically check this with sentence-level entailment models i.e. if a generated statement is not entailed by the retrieved text, it’s considered a hallucination. Takeaway - You need to combine 👉 Grounded automatic checks (retrieval, entailment, fact verification), 👉 LLM-based judgments (self-critique or other LLMs as "judges") 👉 Human annotation (to establish truth baselines). Comment down the methods you have used before! -- 🚶➡️ To land your next Data Science role, follow me - Karun! ♻️ Share so others can learn, and you can build your LinkedIn presence! 📕 For more questions, grab a copy of Decoding ML Interviews - https://lnkd.in/gc76-4eP #datascience #interviewquestion

How effective is Retrieval-Augmented Generation (RAG) in making AI more reliable for specialized, high-stakes data?  The BCG X team, led by Chris Meier and Nigel Markey, recently investigated the quality of AI-generated first drafts of documents required for clinical trials.   At first glance, off-the-shelf LLMs produced well-written content, scoring highly in relevance and medical terminology. However, a deeper look revealed inconsistencies and deviations from regulatory guidelines. The challenge: LLMs can not always use relevant, real-world data.   The solution: RAG systems can improve LLM accuracy, logical reasoning, and compliance. Team's assessment showed that RAG-enhanced LLMs significantly outperformed standard models in clinical trial documentation, particularly in ensuring regulatory alignment. Now, imagine applying this across industries 1️⃣ Finance: Market insights based on the latest data, not outdated summaries. 2️⃣ E-commerce: Personalised recommendations that reflect live inventories 3️⃣ Healthcare: Clinical trial documentation aligned with evolving regulations. As LLMs move beyond just content generation, their ability to reason, synthesize, and verify real-world data will define their value. Ilyass El Mansouri Gaëtan Rensonnet Casper van Langen Read the full report here:  https://lnkd.in/gTcSjGAE #BCGX #AI #LLMs #RAG #MachineLearning

Exciting new research alert! Just read a fascinating paper on "LLM-as-a-Judge" - a groundbreaking approach to automated evaluation using Large Language Models. The concept is revolutionizing how we evaluate complex tasks across various domains. Here's what makes it incredible: Key Innovations: - Replaces traditional human evaluators with LLMs for scalable, cost-effective assessments - Processes diverse input types including text, semi-structured data, and multi-modal content - Offers superior consistency compared to human evaluations while maintaining alignment with human judgment Under the Hood: The system works through a sophisticated pipeline: 1. In-Context Learning Methods: - Score generation (1-10 or 0-100 scales) - Yes/No evaluations - Pairwise comparisons - Multiple-choice selections 2. Model Implementation: - Uses advanced LLMs like GPT-4 or fine-tuned specialized models - Employs custom prompt templates - Implements post-processing methods for reliable output extraction 3. Evaluation Process: - Combines input data with context using specialized operators - Generates auto-regressive evaluations - Applies sophisticated post-processing for accuracy Real-world Applications: - Academic paper reviews - Competition judging - Software engineering evaluations - Educational assessments This technology is transforming how we handle evaluations at scale while maintaining human-like judgment quality. Truly exciting times for AI! 

I like using this completion probabilities visualization tool with my team to help them understand how LLMs work in practice. It’s a bit technical, but it does a great job of visually breaking down the whole LLM stack and showing how LLMs process and generate responses. The tool lets you visualize the probability distribution of the completions (~words). In the video, I walked through a few examples to show how the probabilities change with different contexts, here are some insights: 1/ Models don’t generate words randomly. They calculate likelihoods based on training data and context. For example, if you prompt with "What is the best project management tool?", the model predicts possible completions based on probability. The highest-ranked options might include "Trello", "Asana", or "Jira", with each word’s likelihood depending on past training data. Once the model commits to the first letter, the probabilities narrow dramatically. If it starts with "T", it’s likely completing with "Trello". If it starts with "A", it’s probably "Asana". The initial probability distribution shifts based on the wording of the prompt and any additional context, like previous user or system instructions. 2/ Context changes probabilities. The model continuously updates probabilities based on the preceding text. If specific words or phrases appear earlier in the prompt, they influence which words are more likely to be selected next. Even minor changes in wording or structure can shift the probability distribution. 3/ This applies to search, RAG, and prompt engineering. RAG modifies token probabilities by injecting external information before the model generates a response. Retrieved snippets affect which words are predicted by reinforcing certain completions over others. When no external data is used, the model relies solely on its training data distribution. This highlights how small tweaks in wording, context, or retrieved content can significantly influence AI-generated responses. If you're optimizing for AI search, you should consider these factors in shaping what gets surfaced. I’ll dive deeper into how to optimize for them in upcoming posts. This is part of my AI Optimization Series, where I break down how LLMs process information and how to adapt content for AI search. You can check my two previous posts in this series here. How big is AI search: [https://lnkd.in/eNUidXtg] How AI is transforming how we get information [https://lnkd.in/e7WPd_2t]