Complexity in Medicine and Medical Education

Simple? Complicated? Complex? Complexity in Medicine and Medical Education.

What is The Clinical Excellence Podcast?

The Clinical Excellent Podcast, sponsored by the Bucksbaum Institute for Clinical Excellence is a biweekly podcast hosted by Drs. Adam Cifu and Matthew Sorrentino. The podcast has three formats: discussions between doctors and patients, discussions with authors of research pertinent to improving clinical care and the doctor-patient relationship and discussions with physicians about challenges in the doctor-patient relationship or in the life of a physician.

[00:00:00] Dr. Cifu: On today's episode of The Clinical Excellence Podcast, we have James Woodruff talking about complexity in medical education.

[00:00:11] Dr. Woodruff: We are finding that we are spending tons of money, that we don't have good patient satisfaction, that we are losing our patients' trust, not only in an absolute sense, but a relative sense compared to other countries. And we need to rethink how it is we are making these decisions because to apply evidence blindly is not rigorous.

[00:00:32] Dr. Cifu: We're back with another episode of The Clinical Excellence Podcast sponsored by the Bucksbaum Institute. During this podcast, we discuss, dissect, and promote clinical excellence. We review research pertinent to clinical excellence. We invite experts to discuss topics that often challenge the physician-patient relationship, and we host conversations between patients and doctors.

I'm Adam Cifu, and today I'm joined by Dr. James Woodruff. Jim is a professor of medicine and a practicing general internist at the University of Chicago. Over the past two decades he has served both as a program director for our internal medicine residency program and as the Dean of Students in the Pritzker School of Medicine.

In addition to responsibilities for curriculum development and trainee support, Jim has pursued scholarship in medical professional development. Two areas of focus have been complexity and adaptive behavior in medicine. Thanks so much for joining me, Jim.

[00:01:30] Dr. Woodruff: Thanks Adam for having me.

[00:01:32] Dr. Cifu: I love this topic and I was so excited to get you on to get into it. Um, before we really get into things, since we're going to talk about complexity in medicine and how that impacts medical education and maybe even assessment in medical education, just to start, how do you think about tasks that are kind of simple, complicated and complex, sort of what's different about those?

[00:01:57] Dr. Woodruff: Yeah. I thank you for asking that question because all too often we pursue our work in our medical education without really thinking carefully about, um, what the nature of the problem is that we're dealing with. And it turns out that actually the level of complexity has great bearing on how we can successfully approach these problems.

I like to think about complexity on three levels. There's sort of simple problems, which are the least complex problems, an example of that would be a recipe like, uh, boiling an egg where if you just follow the recipe, you get a good outcome every time you boil that egg. Um, a higher level of complexity is where there are decision trees or specific calculations that you need to make in order to be successful. An example of that is putting a man on the moon and then bringing him back from the moon to the planet earth. A pretty daunting effort, but one that is very doable with a high degree of success or likelihood of success, as long as you have the right people on your team and the right science to help you accomplish that.

Um, the most complex category is what I would just call complex problems or complex systems. An example of this is like raising a child, and the thing about a child is that if you try to raise the child with a recipe, you're not likely to be successful. And even with evidence-informed decision trees, you may not necessarily get the outcome that you expect because systems like children are dynamic and behave in non-linear fashions because they have such a large number of elements and interactions that often evolve with time. And there are thresholds and inhibitory feedback cycles that really surprise you with the overall behavior of the system.

[00:03:49] Dr. Cifu: When I hear you talk about those things that I'm getting ahead of us and thinking about how this pertains to medicine, I sort of think of things within medicine that are simple, complicated, and complex, right? There are things that we do in medicine, not all of it is complex, right? There are some very simple things and there are some, you know, maybe procedures, uh, which are complicated but are not complex.

[00:04:14] Dr. Woodruff: There are but I think it's important to recognize that in a field that ultimately delivers care to people, fundamentally, the work that we do overall is complex. Now, it may be that we have developed tools that rely on recipes or decision trees, but those are all instrumental to a larger goal of problem-solving, which is complex.

[00:04:37] Dr. Cifu: Right. So, so I could imagine that if you're talking about, you know, treating a sensitive urinary tract infection, that's a simple task, but because you're doing it in a human being who you have to, um, you know, collaborate, interact with, um, that makes the task, you know, at least complicated, if not complex.

[00:04:57] Dr. Woodruff: Sure. I mean, the social situation of the patient, the physical capabilities of the patient, the mental capabilities of the patient may determine how you would approach that particular problem, in addition to the fact that patients may have preferences that they would like to express regarding their care.

[00:05:15] Dr. Cifu: So given that definition of complexity, and I love the raising a child metaphor, and I love when you talk about children not being linear, which is certainly the case as you picture a small child. How does one then think about teaching someone a complex task? And that may be just in general, or it may be that when we actually think about, you know, training a doctor, which is the complex task that I think you and I are sort of most interested in or have been most interested in.

[00:05:49] Dr. Woodruff: Yeah. So training someone to pursue complex problem solving is an altogether different responsibility, than training somebody to do purely simple or complicated tasks, and that's because the simplest categories, which are simple and complicated are problems and systems that behave predictably, whereas the complex problems don't behave predictably. And so knowledge by itself is never sufficient to manage complex problems. There needs to be, in addition to this focus on prediction, there needs to be a willingness to respond in an adaptive fashion to the circumstances as they evolve. And this involves a completely different sort of skillset than following or complying with evidence or guidelines.

And again, we're dealing in a situation where our ultimate goals are really complex issues. And it may be that we have simple or complicated tools that we're using to enhance the outcomes of that complex problem, but ultimately the unpredictable nature requires that we go beyond knowledge in order to be successful.

So the question of like, how do you train people to do that? Well, one thing about traditional curriculum is that it is very heavily weighted towards the more predictive skillsets, like fund of knowledge, predictive competencies, I should say. Fund of knowledge for example, or procedural skills, that type of thing, but a subset of the competencies which we really don't think of in a way that I think is most productive are the ones that are more adaptive, like systems-based practice and practice-based learning, we often convert those into sort of very standardized behaviors, when in fact the whole point of those competencies is to encourage people to develop an adaptive response to the larger system that they're working within and to learn as they sort of encounter challenges within the system.

So how does one actually accentuate that portion of those competencies? Well, it means actually training people in complex environments, typically giving them an opportunity to reflect on context, what worked, what didn't work, what was different than expected, given the knowledge, so that the next time they encounter that particular problem, they're better prepared to be responsive, and they may have some other ideas besides the traditional knowledge to address that issue. And in effect, they've gained some wisdom and then to facilitate that kind of learning, oftentimes there's a social context to help with that. There are mentors, there's a community of practice that facilitates that reflection and actually shares the wisdom that these other individuals have gained beyond the technical knowledge that is required to take care of patients.

[00:08:52] Dr. Cifu: So I see this as you are teaching people, you know, some skills that are simple, some that are complicated, and that's, you know, the knowledge as you talk about it, that's the clinical skills. But at the same time, you're sort of teaching the complexity by teaching them to be, you know, like a functioning, I don't know, professional human being, whatever we want to call it, and it sounds like your ideal then is, okay, you know, we're going to teach you the stuff that you need to know, we're going to teach you the clinical skills, but at the same time, we're going to make sure that you are kind of reacting to that information and figuring out how to apply that in the complex situations.

[00:09:34] Dr. Woodruff: Absolutely. One way to describe this that actually makes it a little bit clearer for everybody is to think of the traditional medical education curriculum as providing the trainee with tools, a screwdriver, a wrench, a chainsaw, but remember, just because you equip somebody with tools doesn't mean they can actually solve problems, um, because understanding how those tools are made and what are the operating parameters of those individual tools doesn't necessarily translate into the production of a tailor-made home, for example. And so that second part, which is actually teaching people how to use the tools in dynamic real world environments is the part that I think we don't pay enough attention to in our curriculum.

[00:10:24] Dr. Cifu: You provide chainsaws to the students at the white coat ceremony?

[00:10:27] Dr. Woodruff: Absolutely.

[00:10:28] Dr. Cifu: Good, good, good. So this is already, I think, I don't want to say confusing and I certainly don't want to say complicated or complex, but you know, you can imagine how daunting it is just thinking through it this way. And you know, we train a lot of great doctors and in a way we do it blindly because I think people are creative and we accept terrific people. What has always really confounded me is the assessment side of medical education and figuring out how we can assess our students and figure out, you know, who are the people who are really excelling or I guess predict that they will excel in the future in practice and on the other side of things, how can we pick up the students who, you know, maybe need a little bit more help, a little bit more work to make sure that they're going to do well, you know, once we send them out into residency or out in the world, and thinking about the kind of complexity of medicine that does really seem daunting. So how does that all kind of work together?

[00:11:33] Dr. Woodruff: Yeah, so you bring up a good point. As a program director, I often receive trainees into my program that had excellent grades from medical school and maybe, or even Alpha Omega Alpha, which is the honor society for the top students graduating from any particular medical school arrive in my program, and a subset of them actually would have difficulty taking care of patients.

And I think, again, that's a perfect example of equipping students with standardized tools, but not really helping them learn how to use those tools to navigate the complexity of real world care.

[00:12:13] Dr. Cifu: They may be students who've sort of perfected school, right? But not perfected, you know, I don't know, clinical occupation or clinical professionalism.

[00:12:23] Dr. Woodruff: And the problem is not just that they aren't able to navigate the complexity of the day-to-day work on the wards or in the clinic, but that actually a curriculum that measures their performance and incentivizes standardized performance in some ways changes their attitude towards what it means to be a professional, towards one that is really a very technical definition that is often very standardized, and what is the right answer and that there's one right answer. When in fact, we all know that a given circumstance may have multiple right answers depending upon the details of the context, including like what a patient or their family is willing to pursue in terms of therapy. And so those trainees have difficulty, and as a program director, my responsibility was to support them as they went through the really difficult first year of residency, which hopefully allows them to learn that actually being a professional means more than just the technical success.

So your question about how do you evaluate these folks sort of asks a really important question, how does one pick up these types of issues in medical school? And now I just want to state upfront that actually the technical skills are very important because we do want to equip our students with the best possible tools for success, but I would argue that those standardized measures, those evaluation tools that we use to evaluate standardized performance or standardized behaviors in students are at best formative and should be part of a more holistic approach to evaluating students, that includes real world venues and observation of people that are working closely with them in the same team so that the evaluators are not only content experts, but they are context experts. That is, that they understand what the family's concerns are, they understand the past medical history in detail, and so that they can actually determine what is authentically a good outcome for that situation, which is not ever going to be some standard outcome, it's going to be an outcome that's the best one for that particular patient. Equipped with that information, a mentor at the bedside is better able to provide feedback and assess the quality of performance of that individual trainee and reinforces adaptive behavior, as sort of the strategic element of performance in addition to technical performance as sort of the tactical part of the work that they do.

[00:15:01] Dr. Cifu: So let me challenge you a little bit on one thing because as you describe that, you know, it sounds like an amazing kind of educational setting and educational experience from the formative side of things.

You know, I would love to be a student in that setting with someone who's good at this, who I trust observing me and giving me really useful feedback on how I'm interacting with people, how I'm responding to situations, kind of how mindful I'm being in practice. But I guess as a program director, right, you would actually like those students to have some sort of summative assessment of this, right? That you can read and you can say, yes, you know, this student is someone who excels at this, this is a student who I'm going to have to work with maybe a little bit more. I think we all know, you know, certainly students now, and I'm sure students forever, you know, are very clear about like what's on the test, you know, what do I need to know? Um, and I can imagine students pushing back some when they get that sort of summative assessment, look, "You know, you got a B on this task." And they say, "Well, you know, what did I not do?" And describing what they didn't do in this kind of complex setting, boy, that's difficult, right?

[00:16:22] Dr. Woodruff: It is, but that's why it requires somebody who's part of the team who is standing by the trainee taking care of the patient.

And I think if you actually have that kind of relationship, then you're in the best position to not only provide good feedback, but for the trainee to really value what it is that... The feedback that you're giving to the trainee. I think it's important to think, you know, I'm somebody who loves philosophy and one of the things that I would just say about how we think about evaluation in medical education is that it's actually exporting values from science into a realm that is very different than science. It's actually the interaction of science with patients. And we tend to want to substitute values like reliability and replication of findings, objective observations. We tend to want to substitute those values for the values of the patient and outcomes that really are going to be most important to the patient.

And I think as physicians and medical educators, we need to actually find some balance between those two. We can't completely abandon this idea of technical expertise because indeed it is important for people to have technical expertise, those are important tools, but ultimately I think we are falling short in the other category, which is understanding that success in a patient interaction is not just about technical expertise. It's also about understanding the patient's values, understanding the patient's biologic and social context, and creating an intervention that addresses their needs.

[00:18:12] Dr. Cifu: There are two things I've thought about as you were discussing that. One is people who push back against evidence-based medicine often kind of criticize, you know, EBM practitioners as people who've, you know, adopted ideas from the social sciences with, I know, things like absolute risk reduction, number needed to treat, you know, randomized control trials which maybe don't really work in medicine where everybody you're taking care of is a little bit different. I also think of, you know, a metaphor which makes me crazy when people compare medicine like, you know, the medical establishment to the airline industry, right? And say, well, if we could just do things like them with their fifty different check boxes when they sit down in a cockpit, everything would be fine. That really came to mind as you talked, because I see, wow, you know, medicine is really quite different from that because you could do all the check boxes and still do terribly wrong if the patient's different from the ones that those check boxes were designed for.

[00:19:23] Dr. Woodruff: Yeah. I mean, the mistake that people make when they are making those types of decisions is what I would call a metaphysical misstep. It's actually believing that they're dealing with a simple or complicated problem and using strategies that are appropriate for those particular types of problems to deal with something that's altogether a different beast, which is the unpredictable and oftentimes very subjective nature of taking care of people. And I should say this is not just about patient preferences, this is also about context. And we actually see this play out and even our understanding of evidence when you talk about Bayesian sort of analysis which is basically an expression of, sort of the need for context in order to have even these well-studied tests be useful in individual patients. So this is not just about being touchy-feely with patients. This actually drives straight into actual hardcore medical decision making, and we tend to ignore that part of decision making. And I see it as ignoring it more and more as we move into the future of evidence-based medicine.

There was at one point a debate about what evidence-based medicine was. And the debate was whether it was using evidence with judgment versus simply just using evidence in the form of required guidelines. And I feel like especially in the age of the electronic medical record, we're moving into that more hardcore approach to evidence-based medicine without the use of judgment for reasons related to efficiency, et cetera, But we are finding that we are spending tons of money, that we don't have good patient satisfaction, that we are losing our patients' trust, not only in an absolute sense, but a relative sense compared to other countries. And we need to rethink how it is we are making these decisions because to apply evidence blindly is not rigorous.

[00:21:25] Dr. Cifu: So as I listen to you, you know, design an assessment strategy for students, your description is wonderful, but when I think to the last time I was on the inpatient service, I think of how hard that is to do well, even if you have sort of a perfectly functioning house staff team, um, you have patients who are often quite difficult to take care of. You have many of them and you have a medical student who, you know, has a lot on his or her plate as well. I wonder if there are, you know, just a couple of, I don't know, a couple of suggestions you can give to people to say, look, you know, the next time you are working with a medical student, whether it be on a hospital ward, in an operating room, in a clinic, um, you know, how can you do a good job, um, at helping students really excel in performance in the sort of settings that we're working in?

[00:22:24] Dr. Woodruff: Yeah, good question. Let me start with sort of a big picture point, which is that I recall when I was a junior faculty member that the model of teaching on the wards or at the bedside was very different than it is now. I think you're right, things have gotten quite busy in everybody's lives, and there is a focus on efficiency, which I guess, we should have anticipated, everybody wants to increase margin or to decrease losses, that makes perfect sense. But I think along the way we've actually sacrificed a model that actually worked better than the one that we have right now. And it's not just that we've lost something for medical education, I think we've lost something for patient care.

Um, and I think we need to seriously consider moving back to a model where the faculty member is more engaged in the day-to-day work of the team than they have become recently. Now, if you have a model where you're actually working with the team more often, in my opinion, not only will the patient care be better and the medical education be better, but I think our faculty are going to be happier because there is nothing worse from a medical education happiness perspective than simply having the responsibility of evaluating people without really having an opportunity to spend time with them or to gauge, you know, their performance. So how does one do that? Well, I think, if one has a bit more time, one actually does go to the bedside and I want people to think a little bit more about what it means to go to the bedside. How is it that actually one can make full use of that short period of time you have at the bedside?

Going back to what I think is important to training people to work in complexity, what was it about this case that actually changed your management? What was it that caused you to modify the evidence-based intervention? What was it that didn't work out the way you expected, given the evidence and the literature, how would you do it next time? And what would you watch out for in the patient's care that would cause you to maybe modify your approach yet again? How could you engage the family and the patient more effectively to avoid that problem that you experienced at that particular point in time? So we have to be asking the right questions at the bedside. It isn't just do your presentation and let me make sure you can actually do an H and P. Now, of course, that's one part of it, but then after that, and especially in the assessment and plan and then sort of the discussion after the presentation, we need to be using that to provide mentorship. Okay? Not just I'm observing you and evaluating you, but real mentorship so that they're growing and you are delving into their mental models.

Because evaluating complex problem solving is not just about behavioral outcomes, it's about mental models, it's about attitude, it's actually all the things that are built into the competencies that we never get at. So I think there are ways of doing this and some of it may require a bit more time, but not as much time as you might think.

[00:25:36] Dr. Cifu: I think the proof for me that you're onto something is when I think about students over the years who I've worked with, who I had to give, you know, a not terrific grade to or not terrific assessment to over the years, those conversations were much easier, you know, 15-20 years ago when I was more involved with the team, with the student, with the care of the patient, because it was so easy for me to talk about specific instances and how, you know, we as a team did and how the student working with the team performed. While now, you're right, when I'm a little bit more divorced from the goings on, you know, maybe I'm giving the wrong grade, but I think even when I am confident in my assessment, it's much harder to kind of convince the student that my assessment is accurate, that my assessment is helpful because I haven't been kind of on the ground as much with the patients and the student together.

[00:26:40] Dr. Woodruff: Absolutely. And one additional benefit of all this relates to a problem that we're all dealing with right now, which is how do you distinguish between the performances of all these students? We have so many students that are all getting honors and it's not surprising, right? If we have an evaluation rubric that focuses on standardized behaviors, and we've laid out expectations at the beginning of the course, actually, we should be surprised if the students don't accomplish those check boxes, right? I mean, that's what they've been doing as part of their education up to this point, but the way that you distinguish between these students is to move beyond the checkbox at that point, after you've dealt with the tools, now you're going to deal with the person and their professional approach to this problem. Tell me a little bit about what you were thinking about. What do you think would've changed your management? Was there anything else you noticed about this case that was really important in your decision-making? And you will find that you can distinguish much more easily between those students that are the so-called honor students versus the high pass students. And right now we're giving a lot of those high pass students honors because we're just saying that if you can meet these check boxes, then you get honors. So I think this helps with that problem too.

[00:27:53] Dr. Cifu: Great. Jim, thank you so very much for talking to us today. This is really, really helpful. And thank you for joining us for this episode of The Clinical Excellence Podcast. We are sponsored by the Bucksbaum Institute for Clinical Excellence at the University of Chicago. Please feel free to reach out to us with your thoughts and ideas on the Bucksbaum Institute Twitter page. The music for The Clinical Excellence Podcast is courtesy of Dr. Maylyn Martinez.