Explainable AI Tools

Breakthrough in AI: Microsoft Researchers Revolutionize Recommendation Systems with RSLLM A groundbreaking paper introduces RSLLM (Recommendation Systems as Language in Large Models), a novel framework that seamlessly integrates traditional recommendation systems with Large Language Models (LLMs). Key Technical Innovations: - Unified Prompting Method combining ID-based item embeddings with textual features - Two-stage fine-tuning framework utilizing contrastive learning - Multi-granular alignment at ID, token, and user/item levels - Hybrid encoding system that merges behavioral tokens with textual representations Under the hood, RSLLM employs a sophisticated architecture that: 1. Projects item embeddings into LLM input space using a two-layer perceptron 2. Combines text tokens with behavioral tokens through a specialized concatenation process 3. Implements two contrastive losses for user-item and item-item alignments The results are impressive - RSLLM outperforms existing methods across multiple benchmarks including MovieLens, Steam, and LastFM datasets. The framework shows significant improvements in both prediction accuracy (HitRatio@1) and instruction following (ValidRatio). This research represents a significant step toward unifying recommendation systems with large language models, potentially transforming how we approach personalized recommendations in e-commerce, streaming services, and social media.

🚨 LLMs Could Describe Complex Internal Processes that Drive Their Decisions. Determinism plus interpretability: that is the real foundation of trustworthy AI. This new paper shows something remarkable: with the right fine-tuning, LLMs can accurately describe the internal weights and processes they use when making complex decisions. Not just outputs, but the actual quantitative preferences driving those outputs. Even more, this “self-interpretability” improves with training and generalizes beyond the tasks it was trained on. Why it matters: - It moves beyond black-box probing or neuron-level reverse engineering. - It suggests that models have privileged access to their own internal processes, and can be trained to report them. - It could open a new path for interpretability, control, and safety—complementing the determinism breakthroughs we saw with Thinking Machines. Caveats: - Explanations may still drift toward plausible narratives rather than ground truth. - The cost of fine-tuning and generalization limits need more evidence. - Self-reports remain a proxy, not direct transparency. Still, this is a step forward. Deterministic outputs are essential—but equally essential is knowing why a model chose what it did. Self-interpretability could be the missing bridge. You can read the full paper here: https://lnkd.in/dY94qq4H #AI #ArtificialIntelligence #GenerativeAI #LLM #LargeLanguageModels #MachineLearning #DeepLearning #AIinBanking #AIinFinance #FinTech #BankingInnovation

🔬 Excited to introduce OptiLLMBench - a new benchmark for evaluating test-time optimization techniques in Large Language Models! We've designed this benchmark to help researchers and practitioners understand how different optimization approaches can enhance LLM capabilities across diverse tasks: • Mathematical reasoning (GSM8K) • Formal mathematics (MMLU Math) • Logical reasoning (AQUA-RAT) • Yes/No comprehension (BoolQ) First results with Google's Gemini 2.0 Flash model reveal interesting insights: ✨ Key Findings: • Base performance: 51% accuracy • ReRead (RE2): Achieved 56% accuracy while being 2x faster • Chain-of-Thought Reflection: Boosted accuracy to 56% • Executecode approach: Best performer at 57% 🔍 Category-wise highlights: • Perfect score (100%) on GSM8K math word problems with base inference • Significant improvements in logical reasoning with RE2 • CoT Reflection consistently enhanced performance across categories This benchmark helps answer a crucial question: Can we make LLMs perform better without fine-tuning or increasing model size? Our initial results suggest yes - through clever inference optimization techniques! Try it yourself: 📊 Dataset: https://lnkd.in/gsSriPJH 🛠️ Code: https://lnkd.in/gN6_kNky Looking forward to seeing how different models and optimization approaches perform on this benchmark. Let's push the boundaries of what's possible with existing models! #AI #MachineLearning #LLM #Benchmark #OptiLLM #Research #DataScience

📚 Recommender Systems + Gen AI 🔹 A recent paper by Fabian Paischer, Liu Yang, Linfeng Liu, Shuai S., Kaveh Hassani, Jiacheng Li, Ricky Chen, Gabriel (Zhang) LI, Xialo Gao, Wei Shao, Xue Feng, Nima Noorshams, Sem Park, Bo Long, Hamid Eghbalzadeh from Meta , "Preference Discerning with LLM-Enhanced Generative Retrieval", introduces "Preference Discerning," using Gen AI (LLMs) to extract & condition recommendations on user preferences in text. 🔍 How it works: Preference Approximation: Extracts user preferences from reviews and interaction history via LLMs. Preference Conditioning: Dynamically integrates preferences into a generative retrieval framework. 🎯 The Mender model achieves state-of-the-art results across benchmarks, excelling in fine-grained personalization and preference steering by leveraging Gen AI's contextual understanding. Key takeaway: Combining LLMs' expressiveness with recsys unlocks next-gen personalization and user-centric recommendations. 🔗 paper: https://lnkd.in/g4kAiagj 🔗 blog post on vinija.ai with a detailed review https://lnkd.in/gQbrNtjt This is written in collaboration with Aman Chadha, let us know what you'd like us to review next.

Process Reward Models (PRM) can provide feedback on each step of LLM reasoning but normally require a huge amount of human label data. Google DeepMind is trying to solve this by using progress (likelihood) improvements after each reasoning step and a “prover” LLM to correctly predict the answer, leading to 8% higher accuracy and up to 6x better data efficiency compared to standard outcome-based Reward Models. Implementation 1️⃣ Select a base LLM and a distinct prover LLM (can be weaker). 2️⃣ Generate reasoning traces using the base LLM on a reasoning dataset with correct answers. 3️⃣ Use the prover to solve the problem multiple times before and after the step. 4️⃣ Calculate Advantage/progress for each step by subtracting the "before" success rate from the "after" success rate. 5️⃣ Create Training Data for each step with the problem, steps taken so far (the prefix), the current step, and calculated advantage (target label for RM). 6️⃣ Train a Reward Model to predict these advantage values and use it during RLHF Insights 🎯 Achieve >8% higher accuracy and 1.5-5x better compute efficiency than traditional outcome reward models 🔄 Weaker prover can help improve stronger base models through better exploration ⚡ More efficient exploration by rewarding intermediate progress steps Paper: https://lnkd.in/eK2JQrib

Google DeepMind just open-sourced TxGemma: the first efficient, generalist LLM suite for therapeutic development! Drug discovery has long been hindered by high failure rates, expensive experiments + the need for specialized AI models for each step of the pipeline. 𝗧𝘅𝗚𝗲𝗺𝗺𝗮 𝗶𝘀 𝗮 𝗳𝗮𝗺𝗶𝗹𝘆 𝗼𝗳 𝗹𝗮𝗿𝗴𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗺𝗼𝗱𝗲𝗹𝘀 𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝘁𝗼 𝗽𝗿𝗲𝗱𝗶𝗰𝘁 𝘁𝗵𝗲𝗿𝗮𝗽𝗲𝘂𝘁𝗶𝗰 𝗽𝗿𝗼𝗽𝗲𝗿𝘁𝗶𝗲𝘀, 𝗲𝗻𝗮𝗯𝗹𝗲 𝗶𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝘃𝗲 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗼𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗲 𝗰𝗼𝗺𝗽𝗹𝗲𝘅 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝗳𝗶𝗰 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀. 1. Achieved superior or comparable results to state-of-the-art models on 64 out of 66 therapeutic tasks, surpassing specialist models on 26. 2. Reduced the need for large training datasets in fine-tuning, making it suitable for data-limited applications like clinical trial outcome prediction.   𝟯. 𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗲𝗱 𝗮𝗻 𝗲𝘅𝗽𝗹𝗮𝗶𝗻𝗮𝗯𝗹𝗲 𝗔𝗜 𝘁𝗵𝗮𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁𝘀 𝘁𝗼 𝗶𝗻𝘁𝗲𝗿𝗮𝗰𝘁 𝗶𝗻 𝗻𝗮𝘁𝘂𝗿𝗮𝗹 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲, 𝗿𝗲𝗰𝗲𝗶𝘃𝗲 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗽𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻𝘀, 𝗮𝗻𝗱 𝗲𝗻𝗴𝗮𝗴𝗲 𝗶𝗻 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝗳𝗶𝗰 𝗱𝗶𝘀𝗰𝘂𝘀𝘀𝗶𝗼𝗻𝘀.   4. Developed a therapeutic AI agent powered by Gemini 2.0, which surpassed leading models in complex chemistry and biology reasoning benchmarks (+9.8% on Humanity’s Last Exam, +5.6% on ChemBench-Preference) Since Evo2 by NVIDIA, I've been on the lookout for papers using mechanistic interpretability for explainability. It has obvious benefits for medicine. The use of a conversational variant that explains its reasoning here is a great for informing the user both the strengths and limitations of the model. I'd recommend looking at the example of this when the model is given a molecule’s SMILES string and asked if it can cross the blood‑brain barrier. I'm a firm believer that more researchers in the field should be incorporating explainability into their models. Will be highlighting research more that does so here. It is essential for our ability to iterate on the right things faster to improve the model and actually trust the models. Here's the awesome work: https://lnkd.in/gP--FXVU Congrats to Eric Wang, Samuel Schmidgall, Fan Zhang, Paul F. Jaeger, Rory P. and Tiffany Chen! I post my takes on the latest developments in health AI – 𝗰𝗼𝗻𝗻𝗲𝗰𝘁 𝘄𝗶𝘁𝗵 𝗺𝗲 𝘁𝗼 𝘀𝘁𝗮𝘆 𝘂𝗽𝗱𝗮𝘁𝗲𝗱! Also, check out my health AI blog here: https://lnkd.in/g3nrQFxW

One of my students, Ninaad Shenoy who is an outgoing senior at Ramaiah Institute Of Technology dove into the use of LLMs for recommendation engines. But the approach taken isn't to simply plug an LLM into the workflow. An LLM integration shines in #recsys as a reasoning and explanation engine. Once you've developed the core recommender system using existing approaches (e.g., Collaborative Filtering), you can use LLM to reason about the user's preferences and build a detailed and rich interest profile. This interest profile can also be used as an input to embedding models to find other similar users and their liked products. Additionally, the candidate recommended items can be ranked by the LLM and choices explained. These explanations can be very useful in adding context to the recommendations made to the users. At Koo, we developed very detailed justification texts for our personalized creator recommendations (e.g., because you follow Virat Kohli, similar to Ronaldinho, popularly followed with Anupam Kher). Check out the blog by Ninaad. https://lnkd.in/eMjw7yBQ

😅 If not properly configured, LLM judges can cause more trouble than they solve. LLM judges are quickly becoming a go-to for evaluating LLM results, cutting down on human effort. However, they must be carefully configured, either through training, proper cues, or human annotations. Here’s a nice paper from Meta that shows how to achieve this using only synthetic training data, without relying on human annotations. Some Insights: ⛳ The paper uses unlabeled instructions and prompting to generate synthetic preference pairs, where one response is intentionally made inferior to the other. ⛳ An LLM is then used to generate reasoning traces and judgments for these pairs, creating labeled data from the synthetic examples. ⛳ This labeled data is used to retrain the LLM-as-a-Judge, with the process repeated in cycles to progressively improve the model’s evaluation capabilities. ⛳ On the Llama-3-70B-Instruct model, the approach obtains accuracy on RewardBench from 75.4 to 88.7 (with majority vote) or 88.3 (without majority vote). The method matches or even outperforms traditional reward models trained on human-annotated data, demonstrating the potential of using synthetic data for model evaluation without relying on human input. Link: https://lnkd.in/eRhF4ykx

Evaluating LLMs accurately/reliably is difficult, but we can usually automate the evaluation process with another (more powerful) LLM... Automatic metrics: Previously, generative text models were most commonly evaluated using automatic metrics like ROUGE and BLEU, which simply compare how well a model’s output matches a human-written target resopnse. In particular, BLEU score was commonly used to evaluatate machine translation models, while ROUGE was most often used for evaluating summarization models. Serious limitations: With modern LLMs, researchers began to notice that automatic metrics did a poor job of comprehensively capturing the quality of an LLM’s generations. Oftentimes, ROUGE scores were poorly correlated with human preferences—higher scores don’t seem to indicate a better generation/summary [1]. This problem is largely due to the open-ended nature of most tasks solved with LLMs. There can be many good responses to a prompt. LLM-as-a-judge [2] leverages a powerful LLM (e.g., GPT-4) to evaluate the quality of an LLM’s output. To evaluate an LLM with another LLM, there are three basic structures or strategies that we can employ: (1) Pairwise comparison: The LLM is shown a question with two responses and asked to choose the better response (or declare a tie). This approach was heavily utilized by models like Alpaca/Vicuna to evaluate model performance relative to proprietary LLMS like ChatGPT. (2) Single-answer grading: The LLM is shown a response with a single answer and asked to provide a score for the answer. This strategy is less reliable than pairwise comparison due to the need to assign an absolute score to the response. However, authors in [2] observe that GPT-4 can nonetheless assign relatively reliable/meaningful scores to responses. (3) Reference-guided grading: The LLM is provided a reference answer to the problem when being asked to grade a response. This strategy is useful for complex problems (e.g., reasoning or math) in which even GPT-4 may struggle with generating a correct answer. In these cases, having direct access to a correct response may aid the grading process. “LLM-as-a-judge offers two key benefits: scalability and explainability. It reduces the need for human involvement, enabling scalable benchmarks and fast iterations.” - from [2] Using MT-bench, authors in [2] evaluate the level of agreement between LLM-as-a-judge and humans (58 expert human annotators), where we see that there is a high level of agreement between these strategies. Such a finding caused this evaluation strategy to become incredibly popular for LLMs—it is currently the most widely-used and effective alternative to human evaluation. However, LLM-as-a-judge does suffer from notable limititations (e.g., position bias, verbosity bias, self-enhancement bias, etc.) that should be considered when interpretting data.

📝 Announcing our demo paper at CIKM 2024 that introduces a novel tool for auditing LLMs using a multi-probe approach: 🔹"AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approach" 🔹 Authors: Maryam Amirizaniani, Elias Martin, Tanya Roosta, Ph.D., MFE, Aman Chadha, Chirag Shah 🔹 In collaboration with the University of Washington ➡️ 𝐌𝐮𝐥𝐭𝐢-𝐩𝐫𝐨𝐛𝐞 𝐀𝐩𝐩𝐫𝐨𝐚𝐜𝐡: AuditLLM evaluates LLM performance by generating multiple probes from a single question to identify inconsistencies in model responses. ➡️ 𝐎𝐩𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐌𝐨𝐝𝐞𝐬: The tool offers two modes: (i) Live mode for real-time auditing with immediate feedback, and (ii) Batch mode for comprehensive analysis using multiple queries. ➡️ 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐝 𝐑𝐞𝐥𝐢𝐚𝐛𝐢𝐥𝐢𝐭𝐲: By assessing semantic similarity in responses, AuditLLM helps identify potential biases, hallucinations, and other issues, contributing to the development of robust and reliable LLMs. 📄 Paper: https://lnkd.in/gZAT2xi8 #artificialintelligence #research