How AI can Advance Without Larger Models

🚀 The Future of AI Isn’t Just Bigger—It’s Smarter Advances in model distillation are reshaping how we think about frontier AI models. Traditionally, larger models meant better performance—requiring massive compute budgets and billions of parameters. But new research from DeepSeek, Stanford, and Washington shows that much of this “intelligence” can be compressed into smaller, cost-efficient models using distillation. 💡 What’s Changing? Instead of training models from scratch with astronomical budgets, distillation transfers knowledge from a large “teacher” model to a smaller “student” model—preserving performance while slashing costs and inference latency. Some cutting-edge models are now trained for under $50 in compute credits—a seismic shift for the AI industry. 💰 The Economic Shift This breakthrough changes the game: ✅ AI can now be deployed on resource-constrained devices ✅ Smaller companies & researchers gain access to state-of-the-art AI ✅ Competitive advantage shifts from sheer scale to efficiency & adaptation ⚖️ The IP & Geopolitical Battle Not everyone is thrilled. Big AI players like OpenAI argue that distillation threatens their investments—allowing competitors to replicate proprietary systems. Allegations that DeepSeek leveraged existing U.S. models have sparked heated debates on IP protection, fair use, and AI regulation. 🌍 Where This Leads As AI moves forward, the real frontier won’t be about who builds the biggest models—but who builds the smartest, most efficient ones. Expect a shift toward: 🔹 Task-specific fine-tuning over brute-force scaling 🔹 Sustainable, accessible #AI for a broader audience 🔹 A more level playing field for innovation Stay tuned for a detailed breakdown in the next Generative AI with Varun newsletter. 🧐

Just discovered a fascinating new paper: "ToM-RL: Reinforcement Learning Unlocks Theory of Mind in Small LLMs" While the buzz around large language models often focuses on scale, this research demonstrates that even small LLMs (0.5B-7B parameters) can develop impressive Theory of Mind (ToM) capabilities through targeted reinforcement learning. Theory of Mind—the ability to understand others' mental states and beliefs—is a cornerstone of human social intelligence that has been challenging to replicate in AI systems. The researchers found that with just 3,200 training examples, their RL-trained 7B model achieved 84.50% accuracy on the Hi-ToM benchmark, surpassing models like GPT-4o and DeepSeek-v3 despite having significantly fewer parameters! Key findings: 1. Small LLMs can effectively solve ToM reasoning tasks when trained with appropriate reinforcement signals 2. Moderate-sized datasets (3.2K questions) are sufficient for good performance 3. Models under 3B parameters experience "reasoning collapse," while 7B models maintain stable performance through consistent belief tracking 4. RL-trained models show strong generalization to unseen ToM problems, novel textual presentations, and entirely different datasets The results show a striking performance gap between baseline and RL-trained models across all model sizes, with a 40% improvement on Hi-ToM benchmark tasks after training. What's particularly exciting is how the models generalize to higher-order ToM reasoning. For example, they can answer questions like "Where does Alice think Bob believes Charlie thinks Diana thinks the ball is?" even when trained only on simpler examples. This research bridges the gap between structured problem-solving (math, coding) and nuanced social inference, potentially opening new pathways to developing more socially aware AI systems without requiring massive model sizes. #ArtificialIntelligence #MachineLearning #ReinforcementLearning #LLM #TheoryOfMind #SocialIntelligence #AIResearch Paper link: https://lnkd.in/eCj2sCUw

I recently delved into some intriguing research about the often-overlooked potential of Small Language Models (SLMs). While LLMs usually grab the headlines with their impressive capabilities, studies on SLMs fascinate me because they challenge the “bigger is better” mindset. They highlight scenarios where smaller, specialized models not only hold their own but actually outperform their larger counterparts. Here are some key insights from the research: 𝟏. 𝐑𝐞𝐚𝐥-𝐓𝐢𝐦𝐞, 𝐏𝐫𝐢𝐯𝐚𝐜𝐲-𝐅𝐨𝐜𝐮𝐬𝐞𝐝 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬: SLMs excel in situations where data privacy and low latency are critical. Imagine mobile apps that need to process personal data locally or customer support bots requiring instant, accurate responses. SLMs can deliver high-quality results without sending sensitive information to the cloud, thus enhancing data security and reducing response times. 𝟐. 𝐒𝐩𝐞𝐜𝐢𝐚𝐥𝐢𝐳𝐞𝐝, 𝐃𝐨𝐦𝐚𝐢𝐧-𝐒𝐩𝐞𝐜𝐢𝐟𝐢𝐜 𝐓𝐚𝐬𝐤𝐬: In industries like healthcare, finance, and law, accuracy and relevance are paramount. SLMs can be fine-tuned on targeted datasets, often outperforming general LLMs for specific tasks while using a fraction of the computational resources. For example, an SLM trained on medical terminology can provide precise and actionable insights without the overhead of a massive model. 𝟑. 𝐀𝐝𝐯𝐚𝐧𝐜𝐞𝐝 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬 𝐟𝐨𝐫 𝐋𝐢𝐠𝐡𝐭𝐰𝐞𝐢𝐠𝐡𝐭 𝐀𝐈: SLMs leverage sophisticated methods to maintain high performance despite their smaller size: • Pruning: Eliminates redundant parameters to streamline the model. • Knowledge Distillation: Transfers essential knowledge from larger models to smaller ones, capturing the “best of both worlds.” • Quantization: Reduces memory usage by lowering the precision of non-critical parameters without sacrificing accuracy. These techniques enable SLMs to run efficiently on edge devices where memory and processing power are limited. Despite these advantages, the industry often defaults to LLMs due to a few prevalent mindsets: • “Bigger is Better” Mentality: There’s a common belief that larger models are inherently superior, even when an SLM could perform just as well or better for specific tasks. • Familiarity Bias: Teams accustomed to working with LLMs may overlook the advanced techniques that make SLMs so effective. • One-Size-Fits-All Approach: The allure of a universal solution often overshadows the benefits of a tailored model. Perhaps it’s time to rethink our approach and adopt a “right model for the right task” mindset. By making AI faster, more accessible, and more resource-efficient, SLMs open doors across industries that previously found LLMs too costly or impractical. What are your thoughts on the role of SLMs in the future of AI? Have you encountered situations where a smaller model outperformed a larger one? I’d love to hear your experiences and insights.

We need to talk about this paper. Ever since reasoning models became viable, we’ve all wondered—why do they have to reason in natural language? It’s inefficient and isn’t necessarily how humans think. Much of human reasoning happens internally, with complex thought processes unfolding before a single word is spoken. What if AI had its own reasoning space beyond text? This paper shows that AI can actually reason in latent space. Instead of generating explicit chains of thought, it iterates within a recurrent depth block, allowing it to refine its internal state dynamically before emitting an output. This means test-time compute can scale adaptively. e.g. simple tasks require fewer iterations, while complex reasoning tasks can benefit from deeper computation. Why is this a big deal? 1. No need for specialized CoT datasets – Unlike traditional chain-of-thought models, which require explicit multi-step reasoning data, this approach learns from standard training data and generalizes to reasoning tasks naturally. 2. More efficient – Instead of processing long sequences in large context windows, reasoning is compressed into latent space which significantly reduces memory and compute costs. 3. Adaptive computation – The model can dynamically allocate more compute to harder problems. This results in scalability in reasoning without increasing model size. And the results are remarkable: a 3.5B parameter model trained with this approach performs on par with 50B parameter models, simply by reasoning more effectively. Benchmarks like ARC and GSM8K show that test-time recurrence dramatically boosts performance. paper: https://lnkd.in/ejJqufvv code: https://lnkd.in/eDWFu4zJ model: https://lnkd.in/eZ4tWEky

5 Reasons why Jensen is hyping GPU demands! Jensen Huang's statement that next-generation AI will require 100 times more compute is an oversimplification. While it's true that reasoning-based models like DeepSeek’s R1, OpenAI’s GPT-4, and xAI’s Grok 3 demand more computational resources due to step-by-step logical processing, the trajectory of AI model development suggests a countertrend. 1. The Future of AI is Leaner and More Efficient Mixture of Experts (MoE) Models Will Reduce Compute Load. MoE models selectively activate only a subset of parameters for each inference task, significantly reducing computational overhead. This approach optimizes resource allocation rather than scaling computation linearly.Future AI architectures will prioritize specialization over brute force compute, making AI inference more efficient. 2. Neuromorphic Processing Units (NPUs) Will Enable Efficiency NPUs, designed for AI workloads, provide energy-efficient processing, unlike traditional GPUs, which are general-purpose accelerators. These processors mimic biological neural efficiency, reducing power consumption while maintaining or improving performance. Companies like IBM (TrueNorth), Intel (Loihi), and BrainChip (Akida) are already working on NPUs that deliver orders of magnitude better efficiency than GPUs for reasoning tasks. 3. Recent breakthroughs in transformer-based optimizations, pruning, quantization, and sparsity techniques have demonstrated that AI models can deliver the same performance with significantly less computation. Meta’s Llama 3, OpenAI’s GPT-5, and other future models will likely prioritize efficiency over raw compute scaling. 4.Memory and Compute Optimization Will Play a Key Role Technologies like flash-based memory computing, optical computing, and edge AI will reduce the dependency on massive data center-scale computation. 5. AI is already moving towards on-device inference, where models run efficiently on smartphones and embedded systems instead of relying on large-scale cloud GPUs. While reasoning tasks inherently require more structured computation, the statement that 100 times more compute is necessary ignores the rapid advancements in MoE models, NPUs, and algorithmic optimizations that will lead to more efficient, leaner AI models in the future. The industry trend is towards higher intelligence per watt, not just raw compute scaling. Thus, Huang’s statement aligns with Nvidia’s business interests (selling more GPUs) rather than reflecting the full spectrum of AI efficiency advancements.

The future of AI isn't just about bigger models. It's about smarter, smaller, and more private ones. And a new paper from NVIDIA just threw a massive log on that fire. 🔥 For years, I've been championing the power of Small Language Models (SLMs). It’s a cornerstone of the work I led at Google, which resulted in the release of Gemma, and it’s a principle I’ve guided many companies on. The idea is simple but revolutionary: bring AI local. Why does this matter so much? 👉 Privacy by Design: When an AI model runs on your device, your data stays with you. No more sending sensitive information to the cloud. This is a game-changer for both personal and enterprise applications. 👉 Blazing Performance: Forget latency. On-device SLMs offer real-time responses, which are critical for creating seamless and responsive agentic AI systems. 👉 Effortless Fine-Tuning: SLMs can be rapidly and inexpensively adapted to specialized tasks. This agility means you can build highly effective, expert AI agents for specific needs instead of relying on a one-size-fits-all approach. NVIDIA's latest research, "Small Language Models are the Future of Agentic AI," validates this vision entirely. They argue that for the majority of tasks performed by AI agents—which are often repetitive and specialized—SLMs are not just sufficient, they are "inherently more suitable, and necessarily more economical." Link: https://lnkd.in/gVnuZHqG This isn't just a niche opinion anymore. With NVIDIA putting its weight behind this and even OpenAI releasing open-weight models like GPT-OSS, the trend is undeniable. The era of giant, centralized AI is making way for a more distributed, efficient, and private future. This is more than a technical shift; it's a strategic one. Companies that recognize this will have a massive competitive advantage. Want to understand how to leverage this for your business? ➡️ Follow me for more insights into the future of AI. ➡️ DM me to discuss how my advisory services can help you navigate this transition and build a powerful, private AI strategy. And if you want to get hands-on, stay tuned for my upcoming courses on building agentic AI using Gemma for local, private, and powerful agents! #AI #AgenticAI #SLM #Gemma #FutureOfAI

I spend a lot of time with technical founders building AI companies. Many assume that if we just make models bigger and feed them more data, we'll eventually reach true intelligence. I see a different reality. The fundamental limits of transformer architecture run deeper than most founders realize. Transformer models face three architectural barriers that no amount of scale can solve: 1️⃣ The Edge Case Wall An example in autonomous vehicles: Every time you think you've handled all scenarios, reality throws a new one: a child chasing a ball, construction patterns you've never seen, extreme weather conditions. The architecture itself can't generalize to truly novel situations, no matter how much data you feed it. 2️⃣ The Pattern Matching Trap Our portfolio companies building enterprise AI tools hit this constantly. Current models can mimic patterns brilliantly but struggle to reason about new scenarios. It's like having a highly skilled copywriter who can't generate original insights. The limitation isn't in the training—it's baked into how transformers work. 3️⃣ The Semantic Gap LMs process text without truly understanding meaning. We see this clearly in technical domains like software development. Models can generate syntactically perfect code but often miss fundamental logic because they don't grasp what the code actually does. This creates a massive opportunity for technical founders willing to rethink AI architecture from first principles. Some promising directions I'm tracking: → World models that understand causality and physical interaction → Architectures designed for reasoning during inference rather than training → Systems that combine multiple specialized models rather than one large generalist Founders: While others chase marginal improvements through scale, focus on solving the fundamental problems to build the next $100B+ business (and I'll be your first check ;))

🚀 A Clear Path to Generalized AI? The new models from DeepSeek seem to have changed the game in AI. But people are asking, were these just old tricks, or real innovations? From their papers, it’s clear to me that they pushed the boundaries of efficiency, lowering training and inference costs, while stepping on existing research. You can argue if those were moderate or big advances, however I think that's missing the point. Because the most mind-blowing part is something else that everyone needs to understand: They created a “thinking” model (Chain-of-Thought / CoT) using only Reinforcement Learning (RL), thus removing humans (as evaluators/trainers) from the process completely. To be clear, we don't know how OpenAI trained their own reasoning models (o1 and o3) since OpenAI stopped being "open" some time ago. But it was broadly assumed that massive amounts of human data is needed, until now. So that's big news! 🔥 But let me elaborate on why this is a huge deal 👇 1️⃣ Thinking Models Are Powerful—But Costly The best models today (like OpenAI’s o1 and o3) use CoT reasoning to tackle tough problems (math, logic, coding). This means that the spend a lot of time "thinking" the strategy for solving a problem (like humans do for hard questions). This allows them to achieve outstanding scores in tough AI benchmarks. 💰 But training them is expensive and doesn't scale—they need human-generated data showing step-by-step reasoning (Supervised Fine-Tuning / SFT). 2️⃣ DeepSeek Did It Without Human Data They trained a reasoning model from scratch using only RL—no human-generated training data. Here’s how: (a) Start with a standard model (non-reasoning, like early ChatGPT but bigger e.g. ChatGPT 4 or DeepSeek V3). (b) Pick tasks with clear right/wrong answers (e.g., coding—easy for a machine to check if the code runs successfully). (c) Ask the model to explain its reasoning while solving the problem. Initially, it’s very bad at this. (d) Reward correct answers, penalize wrong ones. The model adjusts and improves. (e) Repeat the cycle. 3️⃣ Reasoning Emerged Naturally What’s crazy? 🔹 The model wasn’t explicitly trained to give long, step-by-step answers. 🔹 It figured out that thinking longer = better results. 🔹 Over time, it naturally evolved into a CoT reasoning model—just through RL. Check out the chart below that shows how long DeepSeek R1 thinks before answering a question as its training progresses... it's quite amazing to see such behavior emerge on its own! 4️⃣ Why This Changes Everything 🚀 No need for human-generated training data → AI can self-improve at scale. 🚀 More RL = More thinking = More intelligence. 🚀 Faster breakthroughs in solving human-level reasoning tasks. The Bottom Line With humans out of the loop, this direct path to stronger AI is unfolding. We should expect massive improvements in problem-solving, and general intelligence soon, from all major labs. Stay tuned. 👀

As we've seen recently with the release of DeepSeek, there is substantial room for improvement in large scale foundation models, both in terms of architectural efficiency and unsupervised training techniques. While the discussion has been mostly about LLMs, there is also a strong need for improvement to the scalability of generative AI in other domains such as video and multi-sensor world models. In the last several months we have released multiple foundation models for video and multi-sensor generative simulation for the autonomous driving space: VidGen-1 and 2, WorldGen-1 and GenSim-2. These models were developed fully in-house (and not fine-tuned from any open-source models) using only ~100 H100 GPUs (inclusive of all the R&D and final training runs), which is a tiny percentage of the typical compute budgets associated with video foundation model development (thousands to tens of thousands of H100 GPUs). How did we achieve industry leading foundation models with much less compute? We combined DNN architecture innovation with advanced unsupervised learning techniques. By leveraging our Deep Teaching technology and improvements to generative AI DNN architectures, we were able to use smaller parameter/more efficient models and to simultaneously accelerate the unsupervised learning process, leading to superior scaling laws compared to industry-typical methods, which means higher accuracy per compute dollar spent, both during training and inference. We have verified that these scaling law advantages persist at larger scales of compute/data, and look forward to keep pushing the frontier of world models for autonomous driving and robotics by scaling up. In essence, combining Deep Teaching with generative AI architecture innovation, leads to a highly scalable form of generative AI for simulation.

For a long time, the common belief in AI has been “bigger is better”—more parameters, more power. But the truth is, we’re hitting a point where just stacking on more “size” isn’t the game-changer we hoped. Our roughly 140B-parameter model can already outperform some massive trillion-parameter systems from tech giants like Google or GPT-4’s variants. How is that possible? The key isn’t brute force. It’s about being smarter with what we have—optimizing the way we distribute computing power and using more sophisticated reasoning strategies. Think of it like a small, well-trained team that outperforms a huge army because they coordinate better and use their knowledge more effectively. What’s next? By 2025, I believe we’ll see a new wave of lean, nimble AI models that can actually “teach” themselves over time. These will be “self-evolving” and more adaptable—less about sheer size, and more about becoming truly self-improving. We’re entering an era where AI won’t just be bigger. It will be more intelligent, more dynamic, and more resourceful, paving the way for transformative breakthroughs across every industry. It’s time to think beyond raw scale and get ready for smarter, leaner, self-evolving AI.