Fact-Checking Procedures

As a first-time founder, here’s one lesson that hit me fast: Bad information is more dangerous than no information. In startup world, you move fast and make calls with limited data. But in a landscape full of AI noise, biased takes, and polished BS, even knowing what’s real is a challenge. I've always questioned the narrative—but now, building a company, the cost of trusting the wrong “truth” is way higher. So, we've developed systems to cut through the noise. Here are three simple but powerful frameworks I've learned over the years that I lean on constantly: 🔍 CRAAP Test - From the academic world, but still gold: Currency – Is this info actually up to date? (Especially in tech, yesterday’s truth is today’s myth.) Relevance – Does it apply to my situation, or just sound good? Authority – Who’s saying it? Have they done it, or are they just loud? Accuracy – Can I fact-check this or cross-reference it? Purpose – Is there an agenda, a sale, or a spin? 🧐 RAVEN Method - A credibility gut-check: Reputation – Does this person/source have a track record of being right? Ability to Observe – Are they speaking from experience or just quoting others? Vested Interest – What do they gain from me believing this? Expertise – Are they qualified in this space, or just internet famous? Neutrality – Are they showing multiple sides or pushing one narrative? 🕵️ SIFT - The quick digital sanity check: Stop – Before reacting or sharing, take a breath. Investigate the source – Who created this and why? Find better coverage – Do other trusted sources say the same? Trace claims back – Can you find the original data or context? Startups already have enough risk. No need to add “believed the wrong blog post” to the list. Fellow founders: how do you separate signal from noise? Would love to hear what’s working for you. Drop it below 👇

Article from NY Times: More than two years after ChatGPT's introduction, organizations and individuals are using AI systems for an increasingly wide range of tasks. However, ensuring these systems provide accurate information remains an unsolved challenge. Surprisingly, the newest and most powerful "reasoning systems" from companies like OpenAI, Google, and Chinese startup DeepSeek are generating more errors rather than fewer. While their mathematical abilities have improved, their factual reliability has declined, with hallucination rates higher in certain tests. The root of this problem lies in how modern AI systems function. They learn by analyzing enormous amounts of digital data and use mathematical probabilities to predict the best response, rather than following strict human-defined rules about truth. As Amr Awadallah, CEO of Vectara and former Google executive, explained: "Despite our best efforts, they will always hallucinate. That will never go away." This persistent limitation raises concerns about reliability as these systems become increasingly integrated into business operations and everyday tasks. 6 Practical Tips for Ensuring AI Accuracy 1) Always cross-check every key fact, name, number, quote, and date from AI-generated content against multiple reliable sources before accepting it as true. 2) Be skeptical of implausible claims and consider switching tools if an AI consistently produces outlandish or suspicious information. 3) Use specialized fact-checking tools to efficiently verify claims without having to conduct extensive research yourself. 4) Consult subject matter experts for specialized topics where AI may lack nuanced understanding, especially in fields like medicine, law, or engineering. 5) Remember that AI tools cannot really distinguish truth from fiction and rely on training data that may be outdated or contain inaccuracies. 6)Always perform a final human review of AI-generated content to catch spelling errors, confusing wording, and any remaining factual inaccuracies. https://lnkd.in/gqrXWtQZ

A lesson for me and maybe for you. 👇 In cybersecurity we talk a lot about zero trust — but what we don’t talk enough about is about applying that mindset to information itself. Recently, I got caught out. Not by malware. Not by a phishing email. But by information that looked credible, and was shared by a trusted cybersecurity source. Sadly, it turned out to be inaccurate and misinformed. I don’t blame this person. As I said, it was a timely reminder to do better and to understand that: ✅ Trust is not a substitute for verification ✅ Cognitive bias affects all of us — even those trained to detect deception ✅ We all need to slow down and check. So, here’s my curated list of tools and resources to help spot misinformation, scams, and dodgy websites. I highly recommend taking a look — and please feel free to add others you trust in the comments. I’ll be checking them out! 😆 A course in how to find reliable info online: https://lnkd.in/e4rG8sfb Fact checker tools: https://www.factcheck.org/ https://lnkd.in/eUKBcRB6 StopagandaPlus (browser extension) https://lnkd.in/eJui5ijZ Tools like Full Fact, ClaimBuster, and Chequeado are at the forefront of automated fact checking. They cross-reference claims against databases of verified information, flagging potential falsehoods in near real time. However, they’re not infallible. These systems struggle with context, nuance, and rapidly evolving situations. They’re best used as a first line of defence, not as the final arbiter of truth. Check a website & find out how likely it is to be legitimate. Just put the url in and it will tell you: https://lnkd.in/eDSjP3S7 Ask Silver to check to see if a message is a scam. Upload a screenshot on WhatsApp and it will tell you & report it to the right authorities : https://lnkd.in/evG545Nn Virus Total (similar to check a website) https://lnkd.in/eYyhWMNU Can you detect these deepfakes? https://lnkd.in/ejf2c95U https://lnkd.in/e5etYRET ⸻ No matter how experienced you are, never let trust replace due diligence. Disinformation (fake news, deliberate spreading usually for a political agenda) and misinformation (mistake/ misinformed) are rife and scaling thanks to AI. Even the most well-intentioned sources can get it wrong. Stay curious, stay cautious, and keep learning. Got more tools or techniques you use to verify info? Share them below — let’s build better digital habits together. 💬👇 #CyberSecurity #Misinformation #MediaLiteracy #FactChecking #DigitalHygiene #CriticalThinking #ZeroTrust #Scams #OnlineSafety

AI factual accuracy is a core concern in high-stakes domains, not just theoretically, but in real-world conversations I have. This paper proposes atomic fact-checking: a precision method that breaks long-form LLM outputs into the smallest verifiable claims, and checks each one against an authoritative corpus before reconstructing a reliable, traceable answer. The study focuses on medical Q&A, and shows this method outperforms standard RAG systems across multiple benchmarks: - Up to 40% improvement in real-world clinical responses. - 50% hallucination detection, with 0% false positives in test sets. - Statistically significant gains across 11 LLMs on the AMEGA benchmark - with the greatest uplift in smaller models like Llama 3.2 3B. 5-step pipeline: - Generate an initial RAG-based answer. - Decompose it into atomic facts. - Verify each fact independently against a vetted vector DB. - Rewrite incorrect facts in a correction loop. - Reconstruct the final answer with fact-level traceability. While the results are promising, the limitations are worth noting: The system can only verify against what’s in the corpus, it doesn't assess general world knowledge or perform independent reasoning. Every step depends on LLM output, introducing the risk of error propagation across the pipeline. In some cases (up to 6%), fact-checking slightly degraded answer quality due to retrieval noise or correction-side hallucinations. It improves factual accuracy, but not reasoning, insight generation, or conceptual abstraction. While this study was rooted in oncology, the method is domain-agnostic and applicable wherever trust and traceability are non-negotiable: - Legal (case law, regulations) - Finance (audit standards, compliance) - Cybersecurity (NIST, MITRE) - Engineering (ISO, safety manuals) - Scientific R&D (citations, reproducibility) - Governance & risk (internal policy, external standards) This represents a modular trust layer - part of an architectural shift away from monolithic, all-knowing models toward composable systems where credibility is constructed, not assumed. It’s especially powerful for smaller, domain-specific models - the kind you can run on-prem, fine-tune to specialised corpora, and trust to stay within scope. In that architecture, the model doesn’t have to know everything. It just needs to say what it knows - and prove it. The direction of travel feels right to me.

"From the very top of Mount Sinai, I bring you these ten . . . cybersecurity regulations." In IT/cybersecurity, the "single source of truth" (SSoT) refers to the authoritative data source, representing the official record of an organization. The broader concept of the SSoT, however, can be helpful in fighting misinformation and disinformation: 1. OBTAIN THE ORIGINAL SOURCE DOCUMENT: Much of the news we hear can be tracked down to a SSoT--an original source document. The original source document can be a judicial opinion, text of a regulation, government or corporate press release, a scientific study, or an audio/video file. 2. FIND IT ON AN OFFICIAL SOURCE: The challenge these days is that with deep fakes, it is hard to know whether you have the SSoT or a fake. Thus, obtain a copy of the SSoT on an official source. For example, judicial opinions can be found on the court website or ECF Pacer. Legislation and proposed legislation can be found on Congress' website. Press releases are available on the issuing agency or organization's website. Scientific studies are usually available (for a fee) on the publishing journal's website or the sponsoring university's website. If you cannot find the SSoT on an official website, consider finding it through a "reliable" news source--one that independently and credibly fact checks its sources, and let's its audience know when it has not done that (e.g., WSJ, NYT, etc.). 3. READ IT YOURSELF: Once you obtain the SSoT, read it yourself, rather than relying on someone's characterization of the document or an AI summary of it. AI regularly hallucinates and mischaracterizes documents and humans often have their own spin or interpretation. See https://lnkd.in/eypgWCnd. 4. CONTEXT MATTERS: Just because you have read the SSoT doesn't mean it is accurate. First, consider what sources the SSoT cites. Are their sources cited at all? Are those sources reliable? Can you review the cited sources themselves? Also, consider who authored the SSoT. Is the author credible? Does the author have a reputation for accuracy and reliability? Consider Googling the name of the document to see whether there is controversy over its authenticity. 5. WHAT IS NOT SAID: When you are reviewing the SSoT, remember that what is NOT said in the SSoT is just as important than what is said. It is not uncommon for people (and perhaps as a result, AI) to make their own inferences and inject their own opinions into their discussion of a topic, when that inference or opinion is not a part of the original SSoT at all, and may be fair or unfair under the circumstances. Deep fakes are a significant problem but the truth is out there. We all bear the responsibility to find it.