Artificial Intelligence

Surgical robots cost $2 million. Beijing just built one for $200,000. Watch it peel a quail egg: Shell removed. Inner membrane intact. Submillimeter accuracy that matches da Vinci at 90% less cost. Think about that. Most hospitals can't afford surgical robots. Rural clinics? Forget it. Patients travel hundreds of miles for robotic surgery or settle for traditional operations with higher risks. Beijing's Surgerii Robotics just broke that equation. Traditional Surgical Robotics: ↳ $2 million purchase price ↳ $200,000 annual maintenance ↳ Only major hospitals qualify ↳ Patients travel or wait Chinese Innovation Reality: ↳ $200,000 total cost ↳ Same precision standards ↳ Reaches district hospitals ↳ Surgery comes to patients But here's what stopped me cold: Professor Samuel Au left da Vinci to build a network of surgical robots. Engineers from Medtronic and GE walked away from Silicon Valley salaries to build this. They're not chasing profit margins. They're chasing one vision: "Every hospital should have one." The egg demonstration proves what matters: Precision doesn't require premium pricing. The robot's multi-backbone continuum mechanisms deliver the same submillimeter accuracy whether peeling eggs or operating on hearts. What This Enables: ↳ Thoracic surgery in rural hospitals ↳ Urological procedures locally ↳ Reduced surgical trauma everywhere ↳ Surgeon shortage solutions The Multiplication Effect: 1 affordable robot = 10 hospitals equipped 100 deployed = provincial healthcare transformed 1,000 units = surgical access democratized At scale = geography stops determining survival Traditional robotics kept precision exclusive. Surgerii makes it accessible. We're not watching price competition. We're watching healthcare democratisation. Because that farmer needing heart surgery shouldn't die waiting for a $2 million robot his hospital will never afford. Follow me, Dr. Martha Boeckenfeld for innovations that put patients before profit margins. ♻️ Share if surgical precision should be accessible, not exclusive. #healthcare #innovation #precisionmedicine

🚨 BREAKING: An extremely important lawsuit in the intersection of PRIVACY and AI was filed against Otter over its AI meeting assistant's lack of CONSENT from meeting participants. If you use meeting assistants, read this: Otter, the AI company being sued, offers an AI-powered service that, like many in this business niche, can transcribe and record the content of private conversations between its users and meeting participants (who are often NOT users and do not know that they are being recorded). Various privacy laws in the U.S. and beyond require that, in such cases, consent from meeting participants is obtained. The lawsuit specifically mentions: - The Electronic Communications Privacy Act; - The Computer Fraud and Abuse Act; - The California Invasion of Privacy Act; - California’s Comprehensive Computer Data and Fraud Access Act; - The California common law torts of intrusion upon seclusion and conversion; - The California Unfair Competition Law; As more and more people use AI agents, AI meeting assistants, and all sorts of AI-powered tools to "improve productivity," privacy aspects are often forgotten (in yet another manifestation of AI exceptionalism). In this case, according to the lawsuit, the company has explicitly stated that it trains its AI models on recordings and transcriptions made using its meeting assistant. The main allegation is that Otter obtains consent only from its account holders but not from other meeting participants. It asks users to make sure other participants consent, shifting the privacy responsibility. As many of you know, this practice is common, and various AI companies shift the privacy responsibility to users, who often ignore (or don't know) what national and state laws actually require. So if you use meeting assistants, you should know that it's UNETHICAL and in many places also ILLEGAL to record or transcribe meeting participants without obtaining their consent. Additionally, it's important to have in mind that AI companies might use this data (which often contains personal information) to train AI, and there could be leaks and other privacy risks involved. - 👉 Link to the lawsuit below. 👉 Never miss my curations and analyses on AI's legal and ethical challenges: join my newsletter's 74,000+ subscribers. 👉 To learn more about the intersection of privacy and AI (and many other topics), join the 24th cohort of my AI Governance Training in October.

𝗢𝗻𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗠𝗢𝗦𝗧 𝗱𝗶𝘀𝗰𝘂𝘀𝘀𝗲𝗱 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻: 𝗛𝗼𝘄 𝘁𝗼 𝗽𝗶𝗰𝗸 𝘁𝗵𝗲 𝗿𝗶𝗴𝗵𝘁 𝗟𝗟𝗠 𝗳𝗼𝗿 𝘆𝗼𝘂𝗿 𝘂𝘀𝗲 𝗰𝗮𝘀𝗲? The LLM landscape is booming and choosing the right LLM is now a business decision, not just a tech choice. One-size-fits-all? Forget it. Nearly all enterprises today rely on different models for different use cases and/or industry-specific fine-tuned models. There’s no universal “best” model — only the best fit for a given task. The latest LLM landscape (see below) shows how models stack up in capability (MMLU score), parameter size and accessibility — and the differences REALLY matter.  𝗟𝗲𝘁'𝘀 𝗯𝗿𝗲𝗮𝗸 𝗶𝘁 𝗱𝗼𝘄𝗻: ⬇️ 1️⃣ 𝗚𝗲𝗻𝗲𝗿𝗮𝗹𝗶𝘀𝘁 𝘃𝘀. 𝗦𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘀𝘁: - Need a broad, powerful AI? GPT-4, Claude Opus, Gemini 1.5 Pro — great for general reasoning and diverse applications.   - Need domain expertise? E.g. IBM Granite or Mistral models (Lightweight & Fast) can be an excellent choice — tailored for specific industries.  2️⃣ 𝗕𝗶𝗴 𝘃𝘀. 𝗦𝗹𝗶𝗺:  - Powerful, large models (GPT-4, Claude Opus, Gemini 1.5 Pro) = great reasoning, but expensive and slow. - Slim, efficient models (Mistral 7B, LLaMA 3, RWWK models) = faster, cheaper, easier to fine-tune. Perfect for on-device, edge AI, or latency-sensitive applications.  3️⃣ 𝗢𝗽𝗲𝗻 𝘃𝘀. 𝗖𝗹𝗼𝘀𝗲𝗱   - Need full control? Open-source models (LLaMA 3, Mistral, Llama) give you transparency and customization.   - Want cutting-edge performance? Closed models (GPT-4, Gemini, Claude) still lead in general intelligence.  𝗧𝗵𝗲 𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆? There is no "best" model — only the best one for your use case, but it's key to understand the differences to make an informed decision: - Running AI in production? Go slim, go fast. - Need state-of-the-art reasoning? Go big, go deep. - Building industry-specific AI? Go specialized and save some money with SLMs.  I love seeing how the AI and LLM stack is evolving, offering multiple directions depending on your specific use case. Source of the picture: informationisbeautiful.net

Last week, a customer said something that stopped me in my tracks: “Our data is what makes us unique. If we share it with an AI model, it may play against us.” This customer recognizes the transformative power of AI. They understand that their data holds the key to unlocking that potential. But they also see risks alongside the opportunities—and those risks can’t be ignored. The truth is, technology is advancing faster than many businesses feel ready to adopt it. Bridging that gap between innovation and trust will be critical for unlocking AI’s full potential. So, how do we do that? It comes down understanding, acknowledging and addressing the barriers to AI adoption facing SMBs today: 1. Inflated expectations Companies are promised that AI will revolutionize their business. But when they adopt new AI tools, the reality falls short. Many use cases feel novel, not necessary. And that leads to low repeat usage and high skepticism. For scaling companies with limited resources and big ambitions, AI needs to deliver real value – not just hype. 2. Complex setups Many AI solutions are too complex, requiring armies of consultants to build and train custom tools. That might be ok if you’re a large enterprise. But for everyone else it’s a barrier to getting started, let alone driving adoption. SMBs need AI that works out of the box and integrates seamlessly into the flow of work – from the start. 3. Data privacy concerns Remember the quote I shared earlier? SMBs worry their proprietary data could be exposed and even used against them by competitors. Sharing data with AI tools feels too risky (especially tools that rely on third-party platforms). And that’s a barrier to usage. AI adoption starts with trust, and SMBs need absolute confidence that their data is secure – no exceptions. If 2024 was the year when SMBs saw AI’s potential from afar, 2025 will be the year when they unlock that potential for themselves. That starts by tackling barriers to AI adoption with products that provide immediate value, not inflated hype. Products that offer simplicity, not complexity (or consultants!). Products with security that’s rigorous, not risky. That’s what we’re building at HubSpot, and I’m excited to see what scaling companies do with the full potential of AI at their fingertips this year!

Large language models (LLMs) are typically optimized to answer peoples’ questions. But there is a trend toward models also being optimized to fit into agentic workflows. This will give a huge boost to agentic performance! Following ChatGPT’s breakaway success at answering questions, a lot of LLM development focused on providing a good consumer experience. So LLMs were tuned to answer questions (“Why did Shakespeare write Macbeth?”) or follow human-provided instructions (“Explain why Shakespeare wrote Macbeth”). A large fraction of the datasets for instruction tuning guide models to provide more helpful responses to human-written questions and instructions of the sort one might ask a consumer-facing LLM like those offered by the web interfaces of ChatGPT, Claude, or Gemini. But agentic workloads call on different behaviors. Rather than directly generating responses for consumers, AI software may use a model in part of an iterative workflow to reflect on its own output, use tools, write plans, and collaborate in a multi-agent setting. Major model makers are increasingly optimizing models to be used in AI agents as well. Take tool use (or function calling). If an LLM is asked about the current weather, it won’t be able to derive the information needed from its training data. Instead, it might generate a request for an API call to get that information. Even before GPT-4 natively supported function calls, application developers were already using LLMs to generate function calls, but by writing more complex prompts (such as variations of ReAct prompts) that tell the LLM what functions are available and then have the LLM generate a string that a separate software routine parses (perhaps with regular expressions) to figure out if it wants to call a function. Generating such calls became much more reliable after GPT-4 and then many other models natively supported function calling. Today, LLMs can decide to call functions to search for information for retrieval augmented generation (RAG), execute code, send emails, place orders online, and much more. Recently, Anthropic released a version of its model that is capable of computer use, using mouse-clicks and keystrokes to operate a computer (usually a virtual machine). I’ve enjoyed playing with the demo. While other teams have been prompting LLMs to use computers to build a new generation of RPA (robotic process automation) applications, native support for computer use by a major LLM provider is a great step forward. This will help many developers! [Reached length limit; full text: https://lnkd.in/gHmiM3Tx ]

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data.  2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro  -> RoboCasa produces N (varying visuals)  -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are creating tools to enable everyone in the ecosystem to scale up with us: - RoboCasa: our generative simulation framework (Yuke Zhu). It's fully open-source! Here you go: http://robocasa.ai - MimicGen: our generative action framework (Ajay Mandlekar). The code is open-source for robot arms, but we will have another version for humanoid and 5-finger hands: https://lnkd.in/gsRArQXy - We are building a state-of-the-art Apple Vision Pro -> humanoid robot "Avatar" stack. Xiaolong Wang group’s open-source libraries laid the foundation: https://lnkd.in/gUYye7yt - Watch Jensen's keynote yesterday. He cannot hide his excitement about Project GR00T and robot foundation models! https://lnkd.in/g3hZteCG Finally, GEAR lab is hiring! We want the best roboticists in the world to join us on this moon-landing mission to solve physical AGI: https://lnkd.in/gTancpNK

There is a push to use Model Predictive Control (MPC) instead of Reinforcement Learning (RL) in LLMs. MPC is not as common in AI but is well-known in robotics. Here is a simple explanation. Model Predictive Control (MPC): • Model-Based: MPC relies on an explicit model of the system. This model is used to predict how the system will respond to different control inputs. • Optimization in Real-Time: At each time step, MPC solves an optimization problem to find the best sequence of control actions for the future, based on the current state and model predictions. • Constraints: MPC can handle constraints directly in its optimization problem, which is crucial for systems with operational limits. • Predictive Horizon: It uses a "rolling horizon" where future states are predicted and optimized over a time window, but only the first action is implemented. • Feedback: Incorporates feedback by updating the system's state at each step, allowing for adjustments to the control strategy based on actual outcomes. Reinforcement Learning (RL): • Model-Free: RL typically does not require an explicit model of the environment. Instead, it learns from interaction, through trial and error. • Learning from Experience: An RL agent learns by exploring the environment, receiving rewards or penalties for actions taken, and adjusting its policy (strategy) over time. • Policy or Value Function: RL either learns a policy (what action to take in each state) or a value function (how good it is to be in a particular state or take an action from that state). • Long-Term Optimization: RL aims to maximize cumulative reward over time, which might not be immediately apparent in short-term actions. • Exploration vs. Exploitation: RL agents often need to balance between exploiting known good actions and exploring new actions to potentially find better strategies. Was this useful?

In the evolving landscape of AI, one thing is clear: it's not about elimination ➡️ but augmentation. As AI technologies grow, human capital will play an even more pivotal role. Yet, a critical talent gap remains. According to Lenovo's Global Study of CIOs, 97% fear their organization's ambitions are at risk without talent equipped with the right AI-focused skills. This underscores the strategic importance of recruiting and retaining AI-skilled professionals for many organizations. The concept of 'Augmented Intelligence' is gaining traction. Here, AI is seen as a tool to enhance and amplify human capabilities, not replace them. This approach ensures people remain at the heart of innovation and decision-making, leveraging AI for superior results. Understanding this balance between automation and augmentation is crucial. Automation can handle repetitive tasks, freeing humans to focus on creative and complex problem-solving. Those who can work alongside AI or in roles requiring human ingenuity will become increasingly valuable. To close the talent gap, it's time to redefine how AI and human intelligence work together – united, they are a powerful force for the future. 🌟 #AI #AugmentedIntelligence #Innovation #TalentGap

If you’re an AI engineer trying to understand and build with GenAI, RAG (Retrieval-Augmented Generation) is one of the most essential components to master. It’s the backbone of any LLM system that needs fresh, accurate, and context-aware outputs. Let’s break down how RAG works, step by step, from an engineering lens, not a hype one: 🧠 How RAG Works (Under the Hood) 1. Embed your knowledge base → Start with unstructured sources - docs, PDFs, internal wikis, etc. → Convert them into semantic vector representations using embedding models (e.g., OpenAI, Cohere, or HuggingFace models) → Output: N-dimensional vectors that preserve meaning across contexts 2. Store in a vector database → Use a vector store like Pinecone, Weaviate, or FAISS → Index embeddings to enable fast similarity search (cosine, dot-product, etc.) 3. Query comes in - embed that too → The user prompt is embedded using the same embedding model → Perform a top-k nearest neighbor search to fetch the most relevant document chunks 4. Context injection → Combine retrieved chunks with the user query → Format this into a structured prompt for the generation model (e.g., Mistral, Claude, Llama) 5. Generate the final output → LLM uses both the query and retrieved context to generate a grounded, context-rich response → Minimizes hallucinations and improves factuality at inference time 📚 What changes with RAG? Without RAG: 🧠 “I don’t have data on that.” With RAG: 🤖 “Based on [retrieved source], here’s what’s currently known…” Same model, drastically improved quality. 🔍 Why this matters You need RAG when: → Your data changes daily (support tickets, news, policies) → You can’t afford hallucinations (legal, finance, compliance) → You want your LLMs to access your private knowledge base without retraining It’s the most flexible, production-grade approach to bridge static models with dynamic information. 🛠️ Arvind and I are kicking off a hands-on workshop on RAG This first session is designed for beginner to intermediate practitioners who want to move beyond theory and actually build. Here’s what you’ll learn: → How RAG enhances LLMs with real-time, contextual data → Core concepts: vector DBs, indexing, reranking, fusion → Build a working RAG pipeline using LangChain + Pinecone → Explore no-code/low-code setups and real-world use cases If you're serious about building with LLMs, this is where you start. 📅 Save your seat and join us live: https://lnkd.in/gS_B7_7d

What's new and noteworthy in Google's newly released Gemma 2 LLMs? The main theme is that they explore techniques without necessarily increasing the size of training datasets but rather focus on developing relatively small and efficient LLMs. In particular, they blend three main architecture and training choices to create the 2B and 9B parameter models: 1) Sliding window attention (e.g., as popularized by Mistral): This technique uses a fixed-sized attention block that allows a current token to attend to only a specific number of previous tokens instead of all previous tokens, as illustrated in the figure below. 2) Group-query attention (like in Llama 2 and 3): This can be regarded as a more generalized form of multi-query attention. The motivation behind this is to reduce the number of trainable parameters by sharing the same Keys and Values heads for multiple Query heads, thereby lowering computational requirements. 3) Knowledge distillation (as in MiniLLM): The general idea is to transfer knowledge from a larger model (the teacher) to a smaller model (the student). Here, they trained a 27B (teacher) model from scratch and then trained the smaller 2B and 9B (student) models on the outputs of the larger teacher model. The 27B model doesn't use knowledge distillation but was trained from scratch to serve as a "teacher" for the smaller models. There's also an interesting section on "logit capping," a technique I haven't seen used before. Essentially, it is a form of min-max normalizing and clipping of the logit values to keep them within a certain range. I presume this is to improve stability and gradient flow during training. Additionally, they leverage model merging techniques to combine models from multiple runs with different hyperparameters, although there isn't much detail about that in the paper. In terms of modeling performance, Gemma 2 is almost as good as the 3x larger Llama 3 70B, and it beats the old Qwen 1.5 32B model. It would be interesting to see a comparison with the more recent Qwen 2 model. Personally, a highlight is that the Gemma 2 report includes ablation studies for some of its architectural choices. This was once a given in academic research but is increasingly rare for LLM research. And here's a link to the Gemma 2 technical report for additional details: https://lnkd.in/gAe4yewy