LLM Assessment Methods for Knowledge Extraction Research

Explaining the Evaluation method LLM-as-a-Judge (LLMaaJ). Token-based metrics like BLEU or ROUGE are still useful for structured tasks like translation or summarization. But for open-ended answers, RAG copilots, or complex enterprise prompts, they often miss the bigger picture. That’s where LLMaaJ changes the game. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁? You use a powerful LLM as an evaluator, not a generator. It’s given: - The original question - The generated answer - And the retrieved context or gold answer 𝗧𝗵𝗲𝗻 𝗶𝘁 𝗮𝘀𝘀𝗲𝘀𝘀𝗲𝘀: ✅ Faithfulness to the source ✅ Factual accuracy ✅ Semantic alignment—even if phrased differently 𝗪𝗵𝘆 𝘁𝗵𝗶𝘀 𝗺𝗮𝘁𝘁𝗲𝗿𝘀: LLMaaJ captures what traditional metrics can’t. It understands paraphrasing. It flags hallucinations. It mirrors human judgment, which is critical when deploying GenAI systems in the enterprise. 𝗖𝗼𝗺𝗺𝗼𝗻 𝗟𝗟𝗠𝗮𝗮𝗝-𝗯𝗮𝘀𝗲𝗱 𝗺𝗲𝘁𝗿𝗶𝗰𝘀: - Answer correctness - Answer faithfulness - Coherence, tone, and even reasoning quality 📌 If you’re building enterprise-grade copilots or RAG workflows, LLMaaJ is how you scale QA beyond manual reviews. To put LLMaaJ into practice, check out EvalAssist; a new tool from IBM Research. It offers a web-based UI to streamline LLM evaluations: - Refine your criteria iteratively using Unitxt - Generate structured evaluations - Export as Jupyter notebooks to scale effortlessly A powerful way to bring LLM-as-a-Judge into your QA stack. - Get Started guide: https://lnkd.in/g4QP3-Ue - Demo Site: https://lnkd.in/gUSrV65s - Github Repo: https://lnkd.in/gPVEQRtv - Whitepapers: https://lnkd.in/gnHi6SeW

Unlocking the Next Era of RAG System Evaluation: Insights from the Latest Comprehensive Survey Retrieval-Augmented Generation (RAG) has become a cornerstone for enhancing large language models (LLMs), especially when accuracy, timeliness, and factual grounding are critical. However, as RAG systems grow in complexity-integrating dense retrieval, multi-source knowledge, and advanced reasoning-the challenge of evaluating their true effectiveness has intensified. A recent survey from leading academic and industrial research organizations delivers the most exhaustive analysis yet of RAG evaluation in the LLM era. Here are the key technical takeaways: 1. Multi-Scale Evaluation Frameworks The survey dissects RAG evaluation into internal and external dimensions. Internal evaluation targets the core components-retrieval and generation-assessing not just their standalone performance but also their interactions. External evaluation addresses system-wide factors like safety, robustness, and efficiency, which are increasingly vital as RAG systems are deployed in real-world, high-stakes environments. 2. Technical Anatomy of RAG Systems Under the hood, a typical RAG pipeline is split into two main sections: - Retrieval: Involves document chunking, embedding generation, and sophisticated retrieval strategies (sparse, dense, hybrid, or graph-based). Preprocessing such as corpus construction and intent recognition is essential for optimizing retrieval relevance and comprehensiveness. - Generation: The LLM synthesizes retrieved knowledge, leveraging advanced prompt engineering and reasoning techniques to produce contextually faithful responses. Post-processing may include entity recognition or translation, depending on the use case. 3. Diverse and Evolving Evaluation Metrics The survey catalogues a wide array of metrics: - Traditional IR Metrics: Precision@K, Recall@K, F1, MRR, NDCG, MAP for retrieval quality. - NLG Metrics: Exact Match, ROUGE, BLEU, METEOR, BertScore, and Coverage for generation accuracy and semantic fidelity. - LLM-Based Metrics: Recent trends show a rise in LLM-as-judge approaches (e.g., RAGAS, Databricks Eval), semantic perplexity, key point recall, FactScore, and representation-based methods like GPTScore and ARES. These enable nuanced, context-aware evaluation that better aligns with real-world user expectations. 4. Safety, Robustness, and Efficiency The survey highlights specialized benchmarks and metrics for: - Safety: Evaluating robustness to adversarial attacks (e.g., knowledge poisoning, retrieval hijacking), factual consistency, privacy leakage, and fairness. - Efficiency: Measuring latency (time to first token, total response time), resource utilization, and cost-effectiveness-crucial for scalable deployment.

The "black box" nature of LLMs poses significant challenges for regulation and ensuring safety. Due to their opaque and complex internal workings, it is often not clear how these models arrive at specific answers or why they generate certain outputs. This lack of transparency complicates efforts to establish robust regulatory frameworks, as regulators find it difficult to assess compliance with ethical and legal standards, including privacy and fairness. Furthermore, without a clear understanding of how answers are generated, users may question the reliability and trustworthiness of the responses they receive. This uncertainty can deter wider adoption and reliance on LLMs. This study (https://lnkd.in/efjmvwiw) aims to address some of these issues by introducing CausalBench which is designed to address the limitations of existing causal evaluation methods by enhancing the complexity and diversity of the data, tasks, and prompt formats used in the assessments. The purpose of CausalBench is to test and understand the limits of LLMs in identifying and reasoning about causality particularly how well they can perform under conditions that mimic real-world examples. Using CausalBench, the authors then evaluated 19 leading LLMs on their capability to discern direct and indirect correlations, construct causal skeletons, and identify explicit causality from structured and unstructured data. Here are the key takeaways: • 𝗦𝗲𝗻𝘀𝗶𝘁𝗶𝘃𝗶𝘁𝘆 𝘁𝗼 𝗗𝗮𝘁𝗮𝘀𝗲𝘁 𝗦𝗰𝗮𝗹𝗲: LLMs are capable of recognizing direct correlations in smaller datasets, but their performance declines with larger, more complex datasets, particularly in detecting indirect correlations. This indicates a need for models trained on larger and more complex network structures. • 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗘𝗱𝗴𝗲 𝗼𝗳 𝗖𝗹𝗼𝘀𝗲𝗱-𝘀𝗼𝘂𝗿𝗰𝗲 𝗟𝗟𝗠𝘀: Closed-source LLMs like GPT3.5-Turbo and GPT4 outperform open-source models in causality-related tasks, suggesting that the extensive training data and diverse datasets used for these models enhance their ability to handle complex causal queries. • 𝗜𝗺𝗽𝗮𝗰𝘁 𝗼𝗳 𝗣𝗿𝗼𝗺𝗽𝘁 𝗗𝗲𝘀𝗶𝗴𝗻: The effectiveness of LLMs varies with different prompt formats, with combinations of variable names with structured data or background knowledge proving particularly beneficial. The development of comprehensive benchmarks like CausalBench is pivotal in demystifying the "black box" nature of LLMs. This enhanced transparency aids in complex reasoning tasks, guiding the selection of appropriate models for specific applications based on empirical performance data. Additionally, a more granular understanding of LLM capabilities and behaviors facilitates more effective regulation and risk management, addressing both ethical and practical concerns in deploying these models in sensitive or high-stakes environments.

⛳ MMLU is a hot topic in LLM research as it's the go-to benchmark, yet it's not the most reliable. This paper digs into the problems and revamps it for better evaluation. 💡 MMLU, or Massive Multitask Language Understanding, serves as a benchmark for assessing the language comprehension abilities of LLMs across a range of subjects like mathematics, history, computer science, logic, and law. It is a standard metric used to evaluate SoTA foundational LLMs. 👉 However, concerns have been raised by AI researchers regarding parsing mistakes, missing context, and incorrect annotations in MMLU. These errors can lead to misleading evaluations and hinder progress in NLP research. 📖 In this work, the authors introduce a comprehensive framework for identifying and categorizing dataset errors, leading to the creation of MMLU-Redux, a subset of MMLU with manually re-annotated questions to correct errors. ⛳ They manually analyze the MMLU dataset and created MMLU-Redux by re-annotating 3,000 questions across 30 subsets. ⛳ They find that errors in MMLU significantly impact the evaluation of LLMs, leading to changes in performance metrics and rankings of leading models. ⛳ MMLU-Redux serves as a stepping stone to correcting MMLU and can be used as a benchmark for automatic error detection in NLP datasets. Check out the paper here: https://lnkd.in/eb29ua4i

In a recent study conducted by one of my PhD students, Sonish Sivarajkumar, we delved into An Empirical Evaluation of Prompting Strategies for Large Language Models in Clinical Natural Language Processing. Our research presents a comprehensive and methodical experimental examination of prompt engineering across five clinical NLP tasks: Clinical Sense Disambiguation, Biomedical Evidence Extraction, Coreference Resolution, Medication Status Extraction, and Medication Attribute Extraction. To carry out this investigation, we thoroughly assessed various prompt types that have been proposed in recent literature, including the simple prefix, simple cloze, chain of thought, and anticipatory prompts. Additionally, we introduced two novel prompt categories, namely heuristic prompting and ensemble prompting. Our performance evaluations were conducted using three state-of-the-art LLMs: GPT-3.5, BARD, and LLAMA2. We also conducted a comparative analysis between zero-shot prompting and few-shot prompting. In doing so, we were able to offer fresh insights and practical guidelines for prompt engineering when working with LLMs for clinical NLP. https://lnkd.in/g9nbQzs8

Recently was studying knowledge graphs, I came across this excellent paper showing how the LinkedIn ML team implemented KGs for customer service. Customer service is extremely important, as TAT matters, and is one of the most common LLM usecases. The Approach: Merging RAG with KGs The essence of LinkedIn's method lies in constructing a Knowledge Graph from historical issue tickets, preserving both the intra-issue structure and the inter-issue relations. This graph-based structure allows for a more nuanced understanding and retrieval of past issues, ensuring that the system can accurately pinpoint the most relevant past instances when responding to new customer queries. Key Components of the Method: Knowledge Graph Construction - Intra-Issue Structure: Each customer service ticket is parsed into a tree-like structure, where sections of the ticket (such as the issue summary, description, and solution) are treated as nodes connected by hierarchical relationships. This retains the logical flow and relationships within a single ticket. - Inter-Issue Connections: The method then connects these trees into a comprehensive graph, linking tickets based on explicit references (e.g., one ticket being related to another) and implicit connections derived from semantic similarities. This interconnected graph ensures that related issues are easily accessible. Query Parsing and Retrieval: - Entity Recognition and Intent Detection: When a customer submits a query, the system identifies the key entities (e.g., "login issue") and the intent (e.g., "how to resolve"). This allows the system to navigate the Knowledge Graph effectively. - Sub-Graph Retrieval: The system then uses embedding-based retrieval to locate the most relevant sub-graphs within the Knowledge Graph. By preserving the context and relationships within these sub-graphs, the system can generate more accurate and comprehensive answers. Answer Generation: Leveraging the retrieved sub-graphs, the LLM generates a response that is both accurate and contextually relevant, drawing on the full breadth of related past issues rather than just isolated text segments. Empirical Results and Impact The effectiveness of this approach was empirically validated through rigorous testing against traditional text-based retrieval methods. The results were impressive, with a 77.6% improvement in Mean Reciprocal Rank (MRR) and a 0.32 increase in BLEU scores, underscoring the enhanced accuracy and quality of answers generated by the system. Perhaps more compelling is the real-world impact observed after deploying this system within LinkedIn's customer service team. The integration of RAG with KGs led to a 28.6% reduction in the median time required to resolve customer issues.

Apple Researchers Present KGLens: A Novel AI Method Tailored for Visualizing and Evaluating the Factual Knowledge Embedded in LLMs Researchers from Apple introduced KGLENS, an innovative knowledge probing framework that has been developed to measure knowledge alignment between KGs and LLMs and identify LLMs’ knowledge blind spots. The framework employs a Thompson sampling-inspired method with a parameterized knowledge graph (PKG) to probe LLMs efficiently. KGLENS features a graph-guided question generator that converts KGs into natural language using GPT-4, designing two types of questions (fact-checking and fact-QA) to reduce answer ambiguity. Human evaluation shows that 97.7% of generated questions are sensible to annotators. KGLENS employs a unique approach to efficiently probe LLMs’ knowledge using a PKG and Thompson sampling-inspired method. The framework initializes a PKG where each edge is augmented with a beta distribution, indicating the LLM’s potential deficiency on that edge. It then samples edges based on their probability, generates questions from these edges, and examines the LLM through a question-answering task. The PKG is updated based on the results, and this process iterates until convergence. Also, This framework features a graph-guided question generator that converts KG edges into natural language questions using GPT-4. It creates two types of questions: Yes/No questions for judgment and Wh-questions for generation, with the question type controlled by the graph structure. Entity aliases are included to reduce ambiguity. Read our full take on KGLens: https://lnkd.in/gtJTifF6 Paper: https://lnkd.in/gxHShs7i Apple He Bai Yizhe Zhang Yi Su Xiaochuan Niu Navdeep Jaitly

Have you wondered about LLM as a judge? Here is the paper for you. ✅ Can Large Language Models Capture Human Annotator Disagreements? Takeaways - ➡️ Large Language Models (LLMs) are increasingly used for automatic annotation, but evaluation often focuses solely on predicting majority labels, overlooking valuable human disagreement. ➡️ Verbalized distribution expression (asking for probabilities) is found to be significantly more effective for disagreement prediction compared to sampling-based methods. ➡️ RLVR-style reasoning degrades LLM performance in predicting annotation disagreements, particularly when human variance is high, while Chain-of-Thought reasoning with RLHF LLMs shows improvement. ➡️ Fine-tuning smaller encoder-only models on thousands of real human annotations often outperforms larger LLMs in disagreement modeling, emphasizing the importance of human data. ➡️ This work evaluates LLMs' ability to predict the distribution of human annotation disagreements using metrics like Variance Correlation and Distributional Alignment across various tasks and LLM settings. Both VarCorr and DistAlign are important dimensions to evaluate the prediction of disagreement. Using LLMs as annotators, especially for subjective tasks where diverse opinions are common, presents challenges in capturing the full spectrum of human judgment. Evaluating LLM annotators solely on majority label accuracy can mask their limitations in modeling disagreement. This research highlights the need for evaluation metrics that assess an LLM's ability to predict the distribution of human labels. While LLM advancements and techniques like CoT reasoning are promising, incorporating real human annotation data remains a powerful approach for accurate disagreement prediction. #ALERT #NaturalLanguageProcessing #LLMEvaluation #DataAnnotation #ComputationalLinguistics

One of our core areas of work is curating datasets, and this also represents our use of LLMs for label extraction. When we begun labeling our Emory CXR dataset, we extracted both CheXpert labels and LLM labels. At that point we needed to make a decision - will we keep chasing every new LLM update (while remaining cost conscious) to update our labels or what was going to be our guide? We share our experience in this paper - https://rdcu.be/ekNCl led by Bardia Khosravi. We developed #Radprompter to help with tracking and versioning our prompts with new updates. We found - 1. LLM-based labeling outperformed the CheXpert labeler, with the best LLM achieving 95% sensitivity for fracture detection versus CheXpert’s 51%. 2. Larger models showed better sensitivity, while chain-of-thought (COT) prompting had variable effects. 3. Image classifiers showed resilience to labeling noise when tested externally. Simply stated - We can rely on using the smallest models with good performance. Be careful with COT as the models tend to "overthink" resulting in inaccuracies. For downstream task model training, LLM extracted labels are almost as good as human extracted labels - which means you can direct your resources to the test dataset. We found that the models trained on these labels were very robust to label noise.

Researchers at UC San Diego and Tsinghua just solved a major challenge in making LLMs reliable for scientific tasks: knowing when to use tools versus solving problems directly. Their method, called Adapting While Learning (AWL), achieves this through a novel two-component training approach: (1) World knowledge distillation - the model learns to solve problems directly by studying tool-generated solutions (2) Tool usage adaptation - the model learns to intelligently switch to tools only for complex problems it can't solve reliably The results are impressive: * 28% improvement in answer accuracy across scientific domains * 14% increase in tool usage precision * Strong performance even with 80% noisy training data * Outperforms GPT-4 and Claude on custom scientific datasets Current approaches either make LLMs over-reliant on tools or prone to hallucinations when solving complex problems. This method mimics how human experts work - first assessing if they can solve a problem directly before deciding to use specialized tools. Paper https://lnkd.in/g37EK3-m — Join thousands of world-class researchers and engineers from Google, Stanford, OpenAI, and Meta staying ahead on AI http://aitidbits.ai