Subscribe
Share
Share
Embed
Truth Seekers: Where Data Meets Reality
Tired of sensational headlines and conflicting health advice? Join Alex Barrett and Bill Morrison as they cut through the noise to uncover what scientific research actually says about the claims flooding your social media feed.
Each week, Alex and Bill tackle a different health, nutrition, or wellness claim that everyone's talking about. From "blue light ruins your sleep" to "seed oils are toxic," they dig into the actual studies, examine the methodologies, and translate the data into plain English.
No agenda. No sponsors to please. No credentials to fake. Just two people committed to finding out what's really true by going straight to the source—the research itself.
Perfect for anyone who's skeptical of influencer health advice but doesn't have time to read every scientific study themselves. New episodes drop regularly, delivering clarity in a world full of clickbait.
Question everything. Verify with data. Find the truth.
Disclaimer: Truth Seekers provides educational content based on published research. Nothing in this podcast should be considered medical, financial, or professional advice. Always consult qualified professionals for decisions affecting your health and wellbeing.
**The Sleep Test That Predicts Everything (Except It Doesn't)**
Alex: Right, so apparently Stanford researchers have built an AI that can predict over a hundred diseases—dementia, cancer, heart attacks—from just one night's sleep. Who wouldn't want that, honestly?
Bill: If this actually works, this is the biggest breakthrough in preventive medicine in decades.
Alex: Early detection, years before symptoms show up, all from tracking your sleep. It sounds brilliant.
Bill: Published in Nature Medicine in January 2026. Stanford researchers, peer-reviewed, huge dataset.
Alex: Which is why it's everywhere. But here's what I'm actually wondering—if this is real, why isn't every doctor ordering sleep tests tomorrow?
Bill: Okay, yeah. That's the right question.
Alex: Because the headlines are incredibly vague about the actual methodology.
Bill: So let me walk through what they actually did. They trained this AI model called SleepFM on sleep recordings from over 65,000 people. That's 585,000 hours of sleep data—polysomnography recordings, which measure brain waves, heart rhythm, breathing, muscle activity, all of it.
Alex: That's proper sleep lab stuff, not just a fitness tracker on your wrist.
Bill: Right, full medical-grade monitoring. And they paired those recordings with electronic health records to see who developed which diseases later. The model achieved what's called a C-Index—
Alex: Mmm.
Bill: —of 0.80 to 0.85 for conditions like dementia, heart attacks, even certain cancers.
Alex: Those numbers sound impressive. 0.85 out of 1, that's like 85% accurate, yeah?
Bill: Hold that thought. But before we get to the statistics, there's something crucial about WHO these 65,000 people actually were.
Alex: Go on.
Bill: They were all patients who came to sleep clinics. People who were referred for sleep studies because they had suspected sleep disorders or other medical conditions that required overnight monitoring.
Alex: Wait, so these weren't just random healthy people?
Bill: Not even close. And the researchers themselves acknowledge this directly in the paper. They write, and I'm quoting here, "our cohort is not representative of the general population, as people without sleep complaints or those with limited access to specialized sleep clinics are underrepresented."
Alex: That's quite a caveat to bury in the discussion section when your headline is claiming this works for everyone.
Bill: This is what we call selection bias, and it's a massive problem for generalization.
Alex: Because if you're training an AI on people who are already sick or at high risk, of course it's going to be good at predicting sickness. That's not the same as it working for someone like me who just wants to know if their sleep patterns mean anything.
Bill: That's exactly it. These are people with higher baseline disease rates, more comorbidities, more healthcare access—they're wealthier on average because they can afford overnight sleep lab stays.
Alex: Right.
Bill: When you apply a model trained on this population to healthy people with low disease rates, your predictive value is going to plummet.
Alex: Hang on. Didn't we—I feel like we've seen this exact problem before. Was it the Apple Watch thing?
Bill: Oh, the heart disease detection study?
Alex: Yeah, where they trained the AI on hospital patients and everyone got excited about the accuracy numbers, but then it turned out those numbers didn't mean what the headlines said.
Bill: Right, right. That was—what was it, the NPV issue? Same basic problem. Training on sick people, impressive-sounding metrics, but it doesn't translate.
Alex: It's the same pattern.
Bill: It really is. And the really frustrating part here is that the science is actually good. The researchers are being honest about their limitations. It's the translation into headlines that's the problem.
Alex: This reminds me so much of when I was working in journalism. You'd see a study on a specific population, and the headline would just strip all that context away. "Scientists say" becomes "this applies to everyone."
Bill: Okay, so selection bias is problem number one. But you asked about the statistics earlier—the 85% thing.
Alex: Yeah, what's going on with that?
Bill: So this is where my data science background gets excited, because this is a textbook case of metric misinterpretation.
Alex: Oh no, you're doing the thing where numbers get fun.
Bill: Just bear with me. When the headlines say "80% accurate," they're referring to a C-Index of 0.80. But a C-Index does not measure accuracy the way most people think.
Alex: What does it measure then?
Bill: It measures ranking. Specifically, if you take two random people from the study and one of them eventually develops a disease, the C-Index tells you how often the model correctly predicts which person will get sick first.
Alex: Okay.
Bill: So it's not saying "80% of predictions are correct," it's saying "80% of the time, we can correctly rank who's higher risk."
Alex: So it's not saying "80% of predictions are correct," it's saying "80% of the time, we can correctly rank who's higher risk."
Bill: Exactly.
Alex: But that's not the same thing at all.
Bill: No. And that's useful for risk stratification in clinical settings, but it's not the same as diagnostic accuracy. It doesn't tell you sensitivity, specificity, or positive predictive value—the things that matter when you're talking about screening the general population.
Alex: What would happen if you actually tried to use this as a screening tool for healthy people?
Bill: The positive predictive value—meaning the chance that a "high risk" prediction actually means you'll get the disease—would drop significantly because the baseline disease rate in healthy populations is so much lower than in sleep clinic patients.
Alex: So you'd have loads of false alarms.
Bill: Exactly. And here's the other thing about C-Index scores—independent reviews in medical literature show that 0.75 to 0.8 is considered "reasonable discrimination," not exceptional.
Alex: Huh.
Bill: We've been using similar scores for cancer treatment response models for years. This isn't a breakthrough number.
Alex: But it sounds like a breakthrough when you say "80% accurate" in a headline.
Bill: That's the magic of metric selection.
Alex: This is making me quite annoyed, actually. Because there are people out there with family histories of dementia or heart disease who are seeing these headlines and thinking, "Finally, there's a way to know if I'm at risk."
Bill: And that's not wrong to want.
Alex: No, of course not.
Bill: Early detection could genuinely save lives. The problem is this tool doesn't deliver what the headlines promise, at least not for the general population.
Alex: What about the "years in advance" claim? That's everywhere in the coverage.
Bill: So the study required at least seven days between the sleep recording and the disease diagnosis—that's to prevent trivial same-day associations. But the follow-up ranged from seven days to 25 years.
Alex: That's quite a range.
Bill: And they don't report the average or median follow-up time. We don't know if most disease diagnoses happened within months or actually years later.
Alex: Wait, they don't report that?
Bill: No. Plus, the paper shows that performance degrades on more recent patient data, which suggests the model might not generalize well over time.
Alex: Okay, but even if we set aside the timeframe issue, there's still the causation problem, right? Like, we don't know if bad sleep causes disease or if early disease causes bad sleep.
Bill: Exactly. The researchers are explicit about this—they state the model can only show correlations, not causal relationships. You could have abnormal sleep patterns because you're already in the early stages of a disease, or both could stem from shared risk factors like obesity or age.
Alex: Age is in the model, right?
Bill: Yep, they add age and sex to the model's predictions. And age alone is a strong predictor of most diseases. So we don't actually know how much predictive power is coming from the sleep data versus just demographic information.
Alex: That's a massive confound.
Bill: It is.
Alex: Actually, wait. I want to push back on something here.
Bill: Okay.
Alex: You're saying age is a confound, and yeah, obviously older people get more diseases. But isn't that always going to be true in longitudinal health studies? Like, you can't just not include age in your model.
Bill: No, you're right. You need age in the model.
Alex: So is the problem that they included it, or that we don't know how much the sleep data adds on top of age?
Bill: The latter. And actually, that's a fair point. The issue is they don't report how much additional predictive value the sleep data provides beyond just knowing someone's age and sex.
Alex: Right, okay.
Bill: When I was doing A/B testing, we'd always report the incremental lift. Like, here's the baseline performance with just demographics, here's what our fancy model adds. They don't really do that here.
Alex: So we can't tell if this is genuinely useful or if it's just a complicated way of saying "older people get sick more."
Bill: Exactly. Though to be fair to the researchers, they'd probably agree with everything we're saying. This is a legitimate research contribution about building foundation models for physiological data.
Alex: Mmm.
Bill: The problem is how it gets packaged for public consumption.
Alex: So what's actually valid here? Because it can't all be hype.
Bill: The technical achievement is real. Training a foundation model on this much unlabeled sleep data using modern deep learning techniques is genuinely innovative. And for specific high-risk populations—people already being evaluated at a sleep clinic—this could actually be useful for clinical risk stratification.
Alex: So within its actual context, it has value.
Bill: Absolutely. And we've known for decades that sleep matters for heart disease, brain health, and mortality. This study reinforces those connections across multiple conditions.
Alex: Right.
Bill: For diseases like Parkinson's, where they got a C-Index of 0.89, there might be real clinical applications.
Alex: But it's not a universal screening tool that works from one night's sleep for anyone.
Bill: Not even close. And this is why understanding selection bias matters. A model trained on sick people looks amazing at predicting sickness in sick people. That doesn't mean it works for healthy people.
Alex: What should people actually take away from this?
Bill: First, when you see AI headlines with big numbers—"predicts 130 diseases!"—dig into the details. In this case, 130 diseases had C-Index scores above 0.75, which sounds impressive until you realize they tested over a thousand disease categories and only reported the ones that cleared a threshold.
Alex: Oh, that's clever.
Bill: And when you see accuracy claims, ask what metric they're actually using and what population it applies to.
Alex: The real question people should ask is: does this work for people like me? Not just "does this work?"
Bill: That's perfectly put. And look, this isn't bad science. It's good science packaged in misleading headlines. The researchers were honest about limitations. The media just didn't emphasize them.
Alex: Which brings us back to where we always end up—reading past the headline, finding the actual study, looking at who was studied and how.
Bill: And being skeptical of breakthrough claims, especially in AI and health.
Alex: Though I will say, the idea of foundation models learning from physiological data is genuinely exciting. If we can do this responsibly, with representative populations and honest communication about limitations, there's real potential here.
Bill: Agreed. The future of AI in medicine is promising. We just need to stop overselling what we've actually achieved today.
Alex: So if you see a headline claiming AI can predict your health from your sleep, the answer is: maybe, if you're already at a sleep clinic being evaluated for health problems. For everyone else, get your sleep sorted because we know it matters, but don't expect a crystal ball.
Bill: And if your doctor actually recommends a sleep study based on symptoms or risk factors, take it seriously. These tools have value in the right context.
Alex: Just not the context the headlines are selling.
Bill: Exactly.