What's up everyone? Here we are, again, we're on episode 5.5, I had to bring back my favorite second favorite co host, Dr. Akerman, because what we're going to talk about today is really interesting. It's the thing that everybody has been discussing in the news. It's how do we test people? Everyone needs to be tested. And then people go, Oh, well, there's these bad tests out there. Oh, with a sensitivity specificity. Nobody's explaining any of it. So as it turns out, Dr. Akerman is really smart at this stuff. And we've been working with a company because we want to introduce rapid testing here in North Texas. And Dr. Akerman has actually helped this company a ton in the type of tests that they're doing, how to explain it. And we're talking sensitivity, specificity, negative predictive value, positive predictive value stuff that I struggle with a ton, well as it turns out, the other guy on the side of the screen here is super smart at it. So Dr. Akerman, can you comment on that real quick?

Yeah. First of all, Make sure you don't burn any bridges with your first favorite co hosts.

Yeah, right.

And, and you're right. I mean, we've pretty much turn on the TV go on the internet, and we're talking about, you know, testing, what kind of testing, is it available, how much do we need? And then you kind of hear from the other side that there may be tons of tests, but too many, and they're not as good. And, you know, you sort of take that all at face value. But what does that mean that the test is no good, what makes it a good or a bad test? And I think that's where, you know, maybe we can do a better job of explaining it to people. And you're right, it's it's super confusing. So...

We did Episode 4.5, where we talked about different the mcas and the antibody testing and things. So we kind of hinted at it. But now we're getting to this point where we're saying, Yeah, but what does all that mean? Nobody's talking about it. You You and I got in a discussion John Oliver. You watched a show on that. 

Yeah. So you know, I don't want to sort of rehash I don't think we're going to do it justice. John Oliver with his extensive medical degree, did an excellent job last week explaining some of the differences between the tests and some of the pitfalls. But I think where we can add a little value to the discussion is to discuss why. Why is it that the test you might go get at your local, you know, minuteclinic, or doctor's office or anything, why that might not offer you the value that you're really looking for.

Yeah, and we can we talked before about the pros and cons of that. We don't know that. But what we're going to talk about today is why that's why getting a test may mean something different if you're in New York or if you're in Plano, Texas,

right? And you want to know the characteristics of the test the characteristics of the disease process, where you are, and what you're trying to get out of the test. And that will determine whether or not it number one makes sense for you to get tested, and how you can get a better result.

Yeah, so we've been seeing that the FDA is saying that you have to apply for this and then you have to achieve a certain. Let's start with the basics, a certain sensitivity and specificity, sensitivity, my friend.

So actually, what I want to do is I'm gonna I'm gonna share my screen with a couple slides here. And, you know, I want to I want to go over some important definitions. There's tons of stuff and you don't need a degree in statistics to figure this out. But it can be confusing. So I think if we sort of write it down, and might it might hit home a little bit better. So

I'm still impressed that you actually left New York I really because you seem like somebody who would be teaching at NYU or Mount Sinai or something like that. So, thank you for joining our group.

I wanted to have more time for podcasts.

There we go.

So a couple six definitions that I want to I want to go over here. And the first you alluded to sensitivity and specificity. The best way I would explain sensitivity is what percentage of patients that actually have the disease will test positive. Right? So the true positives coming out of the, of the test. And the specificity is the opposite. It's the percentage of the patients that don't have the disease and will test negative. So the patients who appropriately test negative for this disease process. Those are intrinsic qualities of the test itself. So you can apply the specificity and the sensitivity to any different population because the test characteristics don't change. What does change and this is the difference is the positive and the negative predictive values. So the positive predictive value, are the percentage of those positive tests that are accurately positive, and the negative predictive value is the percentage of negative tests that are accurately negative to say it another way, when I get a test, if it tells me I'm positive, what are the chances that it's right? That's what I really care about. I want to know If I think I have strep, and I go get a strep test, and it's positive, should I be taking antibiotics? And will I get better? That's what we're really asking. And it is different than the sensitivity and specificity, which I will clarify in a second. The other two really important definitions are the incidence of disease and the prevalence of the disease because these are two different things. The easiest way to differentiate is the incidence are the proportion of people who actively currently have the disease. The prevalence is how many people were affected, not necessarily currently infected. So when you talk about it in terms of the test that we're talking about the PCR test looking for active viral replication, that's looking at the incidence of disease, who's got it now. But the antibody testing, looks at who's got it or who had it. And that's more a test for the prevalence of the disease.

Well explained. Okay, yeah.

All right. So the classic way we look at this is using this, it's called a two by two table or a two by two plot. And you basically up on the, in the in the top bar here, that's the disease or the process that you're looking at whether you have positive or negatives. And then along the margin here, you're talking about the test characteristics, a positive or a negative test. So when you want to calculate the sensitivity, you're looking at this row right here. So the people who have true positives over the total number of positives, so 80, excuse me, if you've got 100% of people ten, let's say people who have a positive test and only eight of them actually have the disease. That means eight out of ten or 80% sensitivity for that test. When you look on the, the the right column here, that's where you're gonna figure out your specificity. That's your negative side. It's the total number of true negatives, people who actually don't have the disease and tested negative over all the negative tests that's going to tell you your specificity.

So I remember...

Sorry?

Go ahead. No, I was just gonna dumb it down to a level that this...I just remember that when I had somebody explained it to me that think of sensitivity and the specificity. If you were to look at these graphs, think of it like a car alarm, that if you make everything so sensitive, so that you could catch the person trying to steal your car. That means that a horn or a loud noise will set it off. If you make it super specific, that it only means that one method of a car being stolen. And so you have this issue of you have this trade off where you have the you want a car alarm that's super sensitive and will catch everything but you're going to get false positives all the time or do you want a specific test that only catches but every once in a while, they're gonna be able to steal your car a different way.

Right. And the ideal test is going to be high at both. It's gonna be really great at ruling people in and great at ruling people out. And the question that often comes up is how do you know the sensitivity and specificity? And the answer is you have to validate every study against the gold standard test ones where you know the numbers and see how it performs. The positive and negative predictive values are different. That's looking across the rows. So if you have the percentage of positive tests accurately positive, now you're looking at of all the tests you did and all the positives you got, how many were right, how many actually are getting positive for a person who has a disease as opposed to missing it, right. So to say it a different way, the percentage of positive tests that are accurately positive, because it's how many people in there have it and showed it versus how many have it and you missed them. The negative predictive value is the bottom row, kind of similar, how many of the people who the tests that are negative, actually are negative, and you've missed and got these false positive results and people who actually don't have the disease. 

That's so fascinating, because once you got me thinking about this, and I've learned this for every board exam, and then I've kind of forgotten it, and then I have not applied it. And then I started thinking about this, you got me thinking, we need to be using this type of logic in the area they're at with the incidence prevalence. It's got me kind of rethinking how to look at this whole thing.

Yeah, and it doesn't this is not something that's unique to COVID testing. This is true of every single test that we do. If you get flu swab if you get a strep test, a culture, everything...

Oh a UA for a UTI, I mean you name it, we're you're gonna use this on every single test. as clinicians we tend to move away from this a little bit because well, we're just treating symptoms and we're kind of moving on but this is such an it's such a fun exercise and if you're...if anybody's listening to this, and they've got a child, niece, nephew, friend, anybody who's in med school, this is a great way to learn it. This is not how they teach you in med school, they teach it way harder. That's why it never sinks in.

Yeah, well, I've had time to think about it and marinate for a bit. So I just want to finish this without leaving too many slides here. But what I want to do is I want to show now how the disease prevalence how much disease there is in the population that you're testing will affect the positive and negative predictive values. Okay. So to say it a different way, the more prevalent your disease is, the more chance there is of having a true positive, and therefore your positive predictive value will go up. Whereas when you have a lower incidence of disease, it's more likely that the random person you test is negative, so that's going to favor having a better negative predictive value. Okay. So as I said in the beginning sensitivity and specificity, that doesn't change that's inherent to the test. So if I tell you that there's a 10% incidence of disease, and an 80% sensitivity of the test that we're applying, eight people will test positive who have the disease, two who have the disease will be missed and test negative. For the specificity I'm using in this test, an arbitrary value of 94% I'm just telling you that that's the parameter of this test. 90 people because it's a 10% incidence disease 90% don't have it. So 85 will test negative based on a 94% specificity. And five will test positive, even though they don't have it, they're false positives. So looking across here, eight out of 13 people will test positive and have the disease that's a positive predictive value is 62%. That means if you go today, and you go get tested, and it's positive, almost a flip of a coin, if that test is accurate for you, whereas it does a great job of telling you you don't have it if it's negative, so you can feel a lot more confident in that negative study than you can in the positives. When you flip this around, and you increase the disease incidence, and instead you got 20% of people now, the sensitivity and specificity didn't change, you're using the same exact test. So I know that the sensitivity and specificity are 80 and 94 doesn't make a difference.

Just to clarify. So to get the sensitivity and specificity it's usually Done where they have the known disease negative known disease positive, they go to a clinical lab and they run it. And then they can show, okay, because that's what's going on right now. 

Right, they compare it to a gold standard. And that's one of the issues here. We don't have gold standards, but we've got good ideas. And we know that people who are positive with the PCR testing, that's the time to check people because that becomes our gold standard. And those have a very, very high sensitivity and specificity. The assays that we've been using for PCRs, have sensitivity and specificity of 98 and 99%.

Of the test, but then we can talk about technique, which is a whole different thing.

Which changes, and can lower your sensitivity...part of the problem. So here in this in this example, we're gonna say that 20% of the people that you're testing have the disease process. So now, up here 20% 20 people out of 100 habit 80 don't so now 80% of them 2016 20% is for on the flip side here 75 test negative don't have it five do. So now when you calculate the positive predictive value, because more people around you have it, there's more of a chance a positive will be right. 16 out of 21 increases your positive predictive value to 76%. But the flip side to that, since more people have the disease, there's a slightly lower chance that a negative will catch you appropriately. So now 75 out of 79 give you a negative predictive value of 95%. So to summarize that one more time: the higher the prevalence of the disease, the higher the positive predictive value, the better a positive test indicates true disease. In a lower incidence population, there's less disease there. The test is much better at giving you a true negative result and ruling the test the ruling the disease out for you.

Got it. 

Hope that makes sense. I mean, I think that...

Let's, let's play around with a little bit I went ahead and pulled up an Excel spreadsheet. And if you'll let me share real quick I'll do the same thing. Let's play with this. Let's say that there's 100,000 person population. And we can change these parameters. Let's say that it's infected by SARS CoV. There's 2000 not infective. So the prevalence we are now in where are we...Plano? What do you think? What do you think our prevalence will end up being?

So tough to say but there's actually data in Dallas, and our prevalence is somewhere between 11 and a half and 12%?

Oh, really? Yeah, that is way higher than when I was thinking. 

It's published data by the Dallas department of health.

Okay, so who let's say that there's a 1% prevalence, then what that means is just like you were explaining, if we have a sensitivity of 98%, and a specificity of 99,

So a very ideal test.

It is the ideal test. And so if you have this, the positive predictive value is literally a flip of a coin. That is, with the best test you can have. That is fascinating with what you just showed. And then of course, the negative predictive value, just like you explained is 100%. So let's say and you're part of the solution, because as we talked about last time, and as we've been working with these companies, we want to be part of the solution. We want to be able to do rapid point of care, IgG IgM tests, and we want to get that data and we want to show it but let's say that here in the state of Texas we are at 12%. That positive predictive value goes to 93% So the more that we know about how many people are infected, the more that we can help these companies with their testing, and the more that we can say things. So I want you to explain this. You have a patient that says, Dr. Akerman, I watched your podcast. And let's assume that we're going to split the difference or whatever, somewhere around 5%. So I got a positive test. What would you tell your coworker, patient friend that did this looking at that?

So which tests are we talking about?

Oh, we're talking about, I thought I had COVID, two months ago, and I tested IgG positive.

Right. So if you've got an antibody test, that's positive, assuming all these characteristics, I'd say there is a very good likelihood you in fact had the disease but if you tested negative, I will tell you almost certainly you haven't had it or been exposed.

So let's get back to those tests there. I'll stop my share. The value of doing antibody tests. Everybody on the news is talking, we need to get tested, we need to get tested. nobody's talking what type of tests, the FDA tries to say, sensitivity specificity, you just showed very clearly that it really depends on where you're getting it done. So we'll be able to gather some of this data. And as we start doing this, what are you going to tell your patients that do test positive as we're gathering the data? So that's, that's an ideal. We can play with the numbers. We don't know what the numbers are yet in each segment. So what would you say to that just so that everybody is very clear about what an antibody test is.

So right now, the only thing that we can tell you if you're If you have antibodies is whether or not you've been exposed. The assumption is that there's immunity because that's how immunity works for all kinds of viruses. But the The X Factor here, is that what that means for each virus is different. How much do you need for immunity? How long does that immunity last? Can you lose that? Will it wane over time? And these are all the questions that we can't yet answer. So if you're going to get tested, it doesn't help you so much at the moment, because we don't know what it means for immunity, but it helps the population because it helps us better define how prevalent the disease is and that starts helping us then figure out well, when are we going to get closer to herd immunity because we know a certain a certain amount of the population needs to have been exposed. Right?

Can you just define herd immunity really quick because that gets thrown around a whole lot by the media. But the reality is herd immunity is a very definitive or it's not a definitive thing. It's a theoretical thing. But...

Right, so if you have the disease, and you're walking around, and all these other people around you that you could potentially, infect, that's not going to be great, right? And that's kind of what happened in the beginning, right? People were walking around, they didn't know that they were infected themselves, and therefore able to infect other people. But if enough people walking around you are for lack of a better term at the moment immune so they can't get infected, then the fact that you have the infection isn't as big a deal. And that's what herd immunity is that if enough people around you are immune, your ability to affect the population as a whole goes down. And that's the idea behind immunity and vaccination to create this herd immunity.

Yeah, think of it like you're at a bowling alley, there's 12 lanes, 12 people the lane one person has the virus, they give it to lane two, lane two gives it to lane three, lane three gives it to somebody who has been vaccinated. It stops. Lanes 4, 5, 6, 7, 8, 9, 10, 11, 12 do not get this. So that's that's what the herd immunity is. I don't know what the it's based on R0 number, the number of exposed, I think I read someplace, it's if we can get 70% of the US either vaccinated or infected. It basically is wipes out this disease.

I'll admit I've heard the same number but I don't know. I don't know the data behind it. But I have seen that quoted.

Now you got me questioning data all the time. Now, you know, I'm looking at it's oh my gosh, I'm getting I actually I'm gonna take a break Eric and I worked with Eric today and I was like, dude, can we do a non COVID podcast? Can we do I don't really care what we talked about. Eric brought up, Mike Tyson looks like he's training again. And at 53 he looks like he could just wreck shop. So I'm like, let's talk boxing or something, anything but COVID right now, because the data is just bruhhh, all these non peer reviewed articles that then get chewed up and shredded by statisticians and other doctors. It's, it's hard. It's really hard

It is and it's confusing. And, you know, I think when it comes to all this COVID testing, you know, some of the things you got to remember, as a patient or someone who's looking to get tested, there's tons of tests available now, tons, and all of them have different sensitivities and specificities and because of that, their ability to tell you whether you do or don't have the disease or in the antibody testing, whether they whether you have or haven't been exposed. are gonna vary, right? And many of those tests were recalled already by the FDA and the FDA is starting to crack down on it because they, they sort of opened the gates a little too wide.  And because we don't know yet the incidence and prevalence of the disease, it makes it a little more challenging to calculate those positive and negative predictive values. And we know that geography plays a role in this because the incidence changes, right? While it definitely is higher in Dallas than we initially thought, we haven't thankfully gotten anywhere near what the incidence is in New York where in some areas it's gotten over 20%. So the same test might might tell you different things depending on whether you did it in Dallas or whether you did it did it in New York.

And you know, you said it best right there that if we do this when we get our tests in because you're an integral part of this, and you're gonna, you're gonna help us try and figure this out tests that we're looking at, will upload to the CDC to help them have real time history. We're gonna have a snapshot of our area, it would be so cool to we're all in this together kind of attitude. Well, let's bring the data in together. I did have a very interesting discussion which you and I have talked about with the manager at our surgery center today, Chris, and he was like, but what do you do if you start testing employees does do you have to shut up? I mean, then all of a sudden get into these ethical, theoretical, where do we go blah blah blah, like you just just go down rabbit holes. All I know is we got to start getting some data and we can start doing it between you and I we can get our patients involved we can get people coming in if you want to get a finger prick and find out if your IgG positive. I know I will feel more comfortable going to the hospital I know I'll feel more comfortable hugging my wife. If we're both IgG positive whether I'm wrong we're just going to learn over time but there's no way to test it.

Right and and some of this, you know, information that we're going to gather now, you know, will be available later that we can look back at and reanalyze. So we've got a, we've got to build that data set. And that's kind of the pitch that I would make to our patients and the community at large. You know, if someone says to me right now, should I get tested for antibodies? You know, is there value to it? I would say there is you just have to understand what that value is. And right now, there's a little bit less of a value personally, although there's definitely some, like you just said, but there's a big community impact that we can we can have. 

Okay, so you always end up with some great little nugget. Say that one more time right now.

So, as it stands right now, you'll have some information for yourself. I can't tell you if you're immune. We can tell you if you've been exposed, which is on the mind of many people. I had that illness two months ago, they tested me for flu, it was negative. But now that I know that I know more about COVID why couldn't I have had COVID at that time? This can answer that question. But it can't tell me if I'm immune. But on a population level, it can start telling us what that prevalence of disease is, it then can feed back and make our tests better. Because now we know what the prevalence is. It has ripple effects,

This is usually the stuff that's discussed in an academic institution behind ivory tower. Now we're just like, we're all in it people let's all do this. That's, that's what's cool.

Yeah, and, you know, two months from now, three months from now, whatever it is, when we find out what immunity means, and what what thresholds we need for immunity, we might be able to look back at this now and say, Well, we've got all these patients that are already tested, and this is the threshold in this test. Maybe these patients are immune and now we can go back and tell them that, you know, sort of remains to be seen, but I would say that there's value in antibody testing, you just have to understand what that value is and I would also suggest that and we've discussed this as part of the parameter that we're going to employ. If a patient has a positive test, you know, partially because we don't know yet how strong that positive predictive value is, and also because IgG starts during the illness, so we don't know if you're a had a disease person or have a disease person, we're going to get, we're going to have all those people get follow up testing immediately for PCR, because if your PCR test is positive, you're actually in the disease starting to get over it. And you're going to you're going to self quarantine, and we're going to let you know and you're not going to expose anyone. And now you're going to know that you had it. And that's going to help you later on, not just now.

Yeah, yeah, I know that if I when these tests come in, and I and I get tested, and I'm IgM for instance, calling up the wife and kids and I'm saying see in about three weeks go get a hotel room. Not even coming not even coming home. So...

And, you know, I'll go one step ahead of you there and sort of shift gears just a little bit on my soapbox. But, you know, we had discussed a couple recent articles, you know, offline here, one of them being that article in Spain that tested all the health healthcare workers and one of the hospitals where they had really strict PPE. And they found an incidence of the disease that was almost identical to the general incidence they have in population. And it's interesting, because we both we both came to the same conclusion, as did your favorite co host, that it wasn't that the incidence was lower. It was that PPE works. And you know, you can have your own opinions. And I'm not saying there's right or wrong about the economy and all these other things that are going on. But there are ways to meld both of those ideas together. And as we start getting out of our homes and reopening the economy, and we're more available and we have more face time with people, we shouldn't lose sight of that. And these principles that that are working to decrease transmission should still be employed.

Yeah, a recent study came out where it looked at ER physicians in Utah and very similar, like the incidence was much lower than you would think. What we're doing works the hand washing the masks everything. So yeah, well, that's awesome. Well, I tell you what, this is a tough topic that nobody's talking about. I appreciate you taking the time to do the PowerPoint. You've got it sorted out in your brain to help thicker brains like me get it kind of figured out. Because when we started realizing that, yes, when I talk to my patients, this is the incident. Six months from now we can say, oh, look, as it turns out, Plano's at 15% or North Texas or whatever, we can say with a certain certainty, this is what it is then. So thanks for helping the companies that we're working with. Thanks for taking the time to do this. And yeah, this is an ever evolving thing. And I think that we're in it, we're part of it, and the beautiful people behind you, and you're seeing there, this is why we're doing a lot of this stuff. So

Yeah, I agree. And I would encourage anyone watching these videos, to, to ask us questions. You know, we were, we're guessing based on what we're hearing and what we're seeing what it is, that's confusing, and we can help explain and, you know, disseminate the information there. But if there's specific questions, let us know. That's what we're here for.

Absolutely. And as always, please like and share, go to Dr. Akerman's website, go to my website. We got our sponsor Atrantil. We know that a healthy gut leads to a healthy immune system so everyone stay safe. Dr. Akerman once again, thank you for a fantastic point five episode.

Thank you.