Subscribe
Share
Share
Embed
Publication: Mazzone, Elio; Puliatti, Stefano MD; Amato, Marco; Bunting, Brendan; Rocco, Bernardo; Montorsi, Francesco; Mottrie, Alexandre; Gallagher, Anthony G. PhD, DSc||. A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes. Annals of Surgery 274(2):p 281-289, August 2021. | DOI: 10.1097/SLA.0000000000004650
Publication: Puliatti, S., Rodriguez Peñaranda, N., Amato, M., De Groote, R., Farinha, R., Bunting, B., van Cleynenbreugel, B., Mottrie, A. and Gallagher, A.G. (2026), Randomised trial on the economic impact of proficiency-based progression vs conventional robotic surgical training. BJU Int, 137: 493-501. https://doi.org/10.1111/bju.70130 https://bjui-journals.onlinelibrary.wiley.com/doi/full/10.1111/bju.70130 — Cost-effectiveness analysis of PBP vs. conventional training. At 500 trainees/year: PBP ~€1.7M vs. conventional ~€3.5M; cost equivalence at just 25 trainees; 100% of PBP trainees reached proficiency vs. 58% conventional
Most training is sold on confidence. Show Me The Evidence is built on data.
In every episode we take a single study, clinical trial, or systematic review and work through what it found, how it was designed, and what it means for the way we teach and assess skill. We focus on metrics-based training and proficiency-based progression, the approach that asks learners to demonstrate measurable competence before moving on, and we trace its results across surgical, medical, and professional education.
This is a podcast for learning professionals and medical educators who want more than opinion. Expect plain-language breakdowns of the research, honest discussion of what the evidence does and does not support, and conversations with the people behind the studies.
If you make decisions about how people are trained, we think you deserve to see the evidence first.
So Tony, I want to start with, I suppose,
an uncomfortable question.
If you had surgery today
at a major teaching
hospital with a fully credentialed
surgeon,
what's the honest answer to the question,
how would you know they're actually
competent to operate on you?
I mean, the bottom line is
you wouldn't know.
I started working in this area
about 30 years ago,
and the assumption was that because somebody
was a consultant in a public hospital
and maybe they had a private practice, it
was an indication that they were good.
And then what I learned was
it's actually to do
with, okay, where did they
do their fellowship?
Where were they trained? Who
did they work under?
What courses did they do and so on?
But over the years, what I've learned,
it means absolutely nothing
because somebody can attend the course
and they can walk away,
and you've no idea what
they actually have
learned cognitively or what
they can actually do.
And it turns out that when we
assessed individuals
about what they can actually do
at the end of a course,
we were shocked.
I mean, the first few studies
that we did, we
thought there was something wrong
with the methodology
because what we were finding
was that large
numbers of the trainees could
do very little.
They knew very little about the procedure
and they could do very little safely.
And then on top of that, what we started
to learn in the early 2000s,
when we did a study with American
College of Surgeons,
where we assessed only attending consultant
level laparoscopic surgeons,
that we found that some of
them were performing
up to 20 standard deviations
away from the mean
in the wrong direction.
And again, we questioned,
you know, we must
have done something wrong.
This can't be right.
These are fully credentialed, you know, hospital
consultants who are attendings,
working mostly in university
teaching hospitals.
But over the years, what we've
learned is the
methodology was signed and the
results were robust.
What we weren't ready for was
to accept that some
of the individuals working in
our teaching hospitals,
in our general hospitals, they're really
not very good at what they do.
So when you go into a hospital
to have a procedure done,
the credentialing that tells
you something,
it tells them being through the process
of that country or that hospital.
But it doesn't tell you anything
about the skills level.
Probably the best way to find out about
the skills level of the surgeon
is to ask one of their colleagues who
will give you an honest answer,
or better still, ask the
theatre sister or
the nurse in charge of the
operating room.
They're more likely to know
who they wouldn't have
operating on them, their family
or their children,
because there's individuals that you
really don't want operating on you,
your family or your friends.
It's eye-opening.
Do you think it's fair to characterise the
current model of professional training
as one of the most consequential
unexamined
assumptions in modern health
institutions?
Yes, I think that's completely right.
We're relying on a training
paradigm that was
developed during the late 19th,
early 20th century.
I buy William Halstead at Johns
Hopkins Hospital.
Halstead, my reading of him,
I could be wrong,
my reading of him was he was a challenging
individual to work with.
He had very exacting standards,
and not everybody that
started their surgical training with
him actually completed.
He learned a lot of lessons from Europe,
so the probability is that he was quite
authoritarian in his approach
to the trainees.
But the thing is that the trainees had sufficient
procedure volume and supervision
by Halstead or his colleagues
to ensure that
they had the opportunity to
become a good surgeon.
And if they didn't meet the
skills levels that
Halstead or his colleagues expected,
they didn't finish.
The problem with that was it
was very subjective.
And today it's still very subjective.
A lot of how a trainee progresses depends
on the consultant that they're
working with,
the references that they give
them and so on.
And the whole process, there are far,
far too many holes in it.
And for the 21st century, with the
rate of change of technology,
with the complexity of the technology
that the surgeon,
and it's not just surgeons, it's cardiologists,
it's any of the proceduralists.
The technology is changing a pace and
that's going to continue.
The bottom line is that we need to
know what somebody can do,
sorry, what they know and what they can
do to some sort of performance level.
Because what we're 100% certain
about now is
that that knowledge and that
performance level,
coupled, you can't have one without
the other, that
that's almost certainly going
to impact on outcomes.
It's not a one-to-one correlation.
It's not a 100% predictor, but the data
clearly show that if somebody's
not very good,
or they're not very knowledgeable about the
procedure or task that they're doing,
they're much more likely to have a bad
outcome than somebody that knows
the procedure well
and is quite good at it.
And given what we're measuring
and training at
the moment, which is time in
seats, hours logged,
courses completed, these
are the currencies
of almost every training
program on Earth.
Why are they the wrong metrics and what
should we be measuring instead?
It's slightly worse than that because the
medicine and particularly
surgery has caught
onto the idea that they must
measure performance.
And now if you take a look
at the training
programs around the world, yes,
there's an emphasis on
procedure numbers.
Yes, there's an emphasis
on the variety of
procedures that they do and the mix
of procedures and so on.
But they've also, what's also
been included in
the assessment is Likert type
scale assessments.
These were originally developed to assess
attitudes to things, personality
attributes,
and so on.
They're totally unfit for
assessing technical
performance or cognitive performance,
but this
is what's been widely used.
So it gives the illusion
that performance is
being assessed, but they're
not being assessed.
I mean, these are entirely subjective and
there's no training on how to
use these instruments.
And when we assess the use of those
instruments, the correlation or the
inter-rater reliability
between two independent assessors,
it's very weak.
And if a trainee was to take a case
against the hospital or the
training program that
assessed them, the hospital
or the training
program wouldn't win because
the data is very
clear.
These instruments do not assess
performance reliably.
I think a lot of the training
programs that
we have, the assessment programs
and so on,
they're smoking mirrors.
I mean, the bottom line
is that around the
world, we now know that when
trainees finish
their residency, whether
it's in the United
States or whether it's in
the UK or Europe,
they're not of the same
performance level,
the same skill level as consultants
or attending
surgeons who completed 20 odd years ago.
And that's for a variety of reasons.
It's not the trainees fault,
I don't think.
I mean, there's reduced workers in Europe
because the assumption, well,
it used to be the assumption the more hours
you worked, the better you were.
That's not correct.
In the US, there was a reduced work hours
because of a fairly disastrous
case in New York,
Lebizian, where the residents in charge of
the case were actually blamed for it.
So the residents have less opportunity
to acquire their skills.
In the past, there would be significantly
more operating experience.
Today, the technology is
much more complex.
The consultants and the attendings
are less likely,
or less enthusiastic about allowing
the trainee to
use the technology to operate on
one of their patients.
And so the trainees are being pushed
from different quarters.
And the assumption is that training in the
skills lab will substitute for the
experience that they're losing
in the operating room.
And that's problematic as well, because
trainees go to a simulation centre
and they talk about playing on the simulator,
which I find appalling.
The simulator is a tool for
facilitating or
helping the trainee acquire
a level of skills.
But what the data clearly shows is if you
don't have the performance metrics to
guide the trainee through
the performance,
to give them explicit constructive formative
feedback on their performance,
literally they are only playing
on the machine.
So they need guidance.
So the simulation is just a tool,
whereas in medicine and in
surgery in particular, it's seen
some sort of magic bullet.
It's not. It's a tool. And it has got to
be used as a tool, as part
of a curriculum.
And the curriculum has to come
from the clinicians,
the surgeons, the cardiologists,
the radiologists,
who actually know how to
do the procedure.
Not individuals who have an opinion
on how to do the procedure,
but individuals who are experienced
in the procedure,
who know the best approach
for a trainee at the
start of their learning curve
to learn that procedure.
And by and large, that's not
really what's happened.
There are a few exceptions, like the Arthroscopic
Association in North America,
like the European Robotic
Urological Society.
So there are a few exceptions, but by and
large, simulations use it in
education experience.
And the bottom line is what we're left
with is trainees who are graduating.
As, you know, fit to be independent
consultants.
And when you objectively assess
the performance,
even on a relatively straightforward
procedure,
we find that the vast majority can't
do the procedure safely.
So Tony, now that you've identified the
problem very well, you've also
presented a solution.
And your study with Neil Seymour published
in the Annals of Surgery in 2002
showed that simulator-based residents
reached proficiency faster
and made 60 percent fewer errors than their
non-virtual reality trained peers.
But that was over 20 years ago.
Why didn't that finding transform
surgical training overnight?
To some extent it did.
I mean, the paper that you
talk about we did in
2000 at Yale, the Department
of Surgery in Yale.
And it was the American Surgical
Association
education paper in their
April 2002 meeting.
It was published in Annals
in October of that
year and was a citation classic
by Christmas.
And the surgical community
were looking for some
sort of solution to the problem
of reduced workers
and reduced procedure volume
for the trainees.
Virtual reality simulation
seemed to be ideal.
But it had been around, Satava first
talked about it in the late
1980s, early 1990s,
so it had been around for over a decade
and nothing really had happened.
And what the surgical community wanted was
robust evidence demonstrating that
simulation produced a better
training outcome.
And we provided that.
The American College of Surgeons
took that on board.
Jerry Healy led a meeting in April
or March in Boston College.
And the question was, what position
should American
College of Surgeons take towards
simulation-based training?
And at the start of the meeting, they
basically said, well, we need
a clinical trial.
And I piped up and said, okay,
we're going to
present it at the American
Surgical Association.
This is what the results suggest.
Jerry Healy is an interesting man.
He worked at Boston College.
He was a paediatric surgeon.
And Jerry could be quite
a tough character.
And I'm not sure that he was taught by
everybody's popularity list, but
he was a good leader.
Jerry listened to what was said, and he
turned the meeting on its head.
And then rather than asking, oh, should
the American College of Surgeons, what
position should they take in terms about
simulation-based training,
he said, how should the American
College of
Surgeons adopt and implement
simulation-based training?
Which, I mean, I thought Jerry showed
true leadership that day.
The rest of the meeting rolled
in behind him.
Okay, there were awkward questions and
so on, but basically they got up.
And they knew that if we were presenting
at the American
Surgical Association, the study was
good and the data were signed.
So in 2006, the American
College of Surgeons
established the accredited educational
institutes.
And all of them were to establish some sort
of accredited simulation centres.
These were evaluated in terms
of the types of surgery
and surgeons who were going to teach
or being taught on it,
the instrumentation, the space
and all the rest of that.
And it's quite a rigorous process.
But the problem is they have
something like over
100 of these centres in the US
and around the world.
They're all accredited and they're
all doing different things.
And they're all using simulation as
an educational experience.
They're all using Likert
skills and so on.
And that really shouldn't have
been the function of...
But Tony, does that not show then
that you're finding
in that paper with all the research
that you've done,
both then and since, that it
hasn't transformed
surgical training to the overwhelming
extent that you want.
I appreciate it, the citation classic.
And I appreciate that with
leaders like Jerry,
as you mentioned, things can
be pushed forward.
But eventually there's a regression to the
mean, a regression to the status quo.
Yeah, I mean, it has...
I mean, for example, if the American
College of Surgeons
decided today to use an outcome based
approach to training,
they have everything in place.
What they have not done is insisted on a standard,
the establishment of performance
metrics, the establishment
of proficiency.
I mean, they have all the pieces
in place, but that last.
And so, I mean, a lot of it's really
like smoke and mirrors.
Yes, the trainees enjoy it. Yes,
the faculty enjoy it.
Yes, the facility looks fabulous
and all the rest of it.
But when you assess the trainees,
when they come out
at the end of the training, I
mean, which they don't.
But when we do, what we find is
they can't do very much.
And we know now for certain,
since 2015, having
simulation facilities, no matter
how good they are,
having those facilities available, have
trainees go through, it means
almost nothing.
You know, it's an interesting
educational experience.
And we know that from the Arthroscopic
Association of North America.
We completed a prospective randomised and
blinded study of something like
trainees from 20-odd,
ACGME approved orthopaedic residency
programs in the United States.
And they all come into Chicago for a
simulation-based training course.
One group got what they would
normally get today.
And I've got to say, ANA prides itself
on doing good training.
So the first group got by any measure what
ANA would consider good training.
The second group got the ANA training,
plus they had simulation.
And group three got what
ANA would normally
give, plus proficiency-based
progression simulation.
So all three of them shared the same
fundamental elements, which was
the ANA curriculum.
Group two got simulation added to that.
And group three had simulation,
but they had to
demonstrate proficiency benchmarks
the whole way through.
And what we found was that the simulation,
the ANA trained group did worse.
And I think ANA was kind
of shocked by that.
The group that had simulation added to their
training, the assumption was it was
going to have some sort of marked impact.
It didn't, like five or six percent more
of the trainees demonstrated
the benchmark.
Whereas the proficiency-based progression
train group, 75 percent demonstrated
the benchmark.
And I mean, the results were quite clear.
Having a good curriculum, having fabulous
facilities and adding simulation to it
means almost nothing in terms of
the training outcomes.
It's the metrics used to deliver
a curriculum on
the simulation that actually
makes a difference.
That's the well-articulated exposition of
the problems with current training.
But then we allied that to the
issue we're going to
uncover hopefully with the next
question, which is,
there's a cultural assumption that
experience equals competence.
That when you have a seniority, that grey
hair means experience effectively.
So what does the evidence that you've
gathered actually say about that?
Yeah, I know I started out with that
assumption 30 years ago.
And I would, when I was going to assess a
surgeon or video record the performance,
I would ask him, okay, do you know
how to do this task?
Yes. Okay. Have you done it before? Yeah.
And how many times have you
done it before?
Because they assumed that
the last two almost
certainly were going to impact
on their performance.
I now ask another question. Show me.
Because what I find is the correlation
between the first
three questions and the last question
means absolutely nothing.
Somebody can have a large amount of
procedure volume experience.
And you would think that they're learning
during that. A lot of them don't.
The mistakes that they were making or the errors
that they were making at the start
of their procedure experience, they're
still making it today.
That's not the case for all of the surgeons.
I mean, I think the majority, I
wouldn't say the vast majority,
I would say the
majority actually do learn from
their experience.
Experience is not a bad thing. It's correlated.
It absolutely is correlated in the
past with procedure outcomes, with skill.
But the problem is the correlation is so
noisy, you have no idea what
you're getting.
And it was used as a surrogate for the
performance or the operating
skill of the surgeon.
And what we found is there's
actually a better
way to establish the skill
level of the surgeon.
And that's objectively assessed, their
interoperative performance on a
straightforward case, not a complex case.
Because the complex case can throw
you so many curve balls.
A straightforward procedure that the surgeon
or the cardiologist or the radiologist
should know how to perform
has done frequently.
You record that, you objectively assess it
with their metrics. That'll tell you
more about their skill level than
their procedure volume.
And one of the shocking things
that we find is the
trainees, I don't mind, the trainees
are still fixable.
And as you would understand,
I mean, there's
different performance levels
the trainees are at.
That's fine. They're still
in the training. And
once you identify these issues,
you can fix that.
But if you have a consultant or an attending
out in independent clinical practice,
and you find that they're performing,
not only are they
performing significantly worse than
some of their colleagues,
they're consultant colleagues, they're performing
significantly worse than some of
the residents in training. That's
a significant issue.
The first time we found that we probably
didn't believe it.
For example, the study that we did at the American
College of Surgeons in 2001, and
that was published in the Journal of the
American College of Surgeons 2003,
we were shown that some of the consultants
were performing particularly badly.
To be honest, in 2001, we thought we did
something wrong. I mean, we set out to
establish what's the correlation between
doing a task in a virtual reality
simulator and doing the exact same
task in a box trainer.
That's what the college asked
us to address.
The most important finding to come
out of that study was
some of the consultants really weren't
performing very well.
And we have now replicated that across
procedures, across disciplines.
I mean, and what we're finding is some of
the consultants that are actually in
clinical practice, when we objectively
assess their
performance from their video recorded
clinical performance,
we find that some of them do very, very
poorly indeed, and indeed some
of them are unsafe.
Now that we've identified the problem, we'll
get to the solution and the solution
that I've mentioned in many talks as being
the single most impactful methodology
I've come across in my career as
an instructional designer.
So Tony, to proficiency-based
progression.
Walk us through the mechanics.
What does a
proficiency-based progression
curriculum look like?
What are the errors, metrics,
critical and
sentient errors? Can you explain
the methodology to us?
Proficiency-based progression. One of the reasons
I developed a scientific approach
to training was there was so much
in surgical training,
surgical appraisal, surgical assessment
that was subjective,
that I thought it was unfair on the
trainee or the consultant who
was being assessed.
So I approached it with a
more scientific eye.
And one of the big issues in the
late 1990s, early 2000s
was the how do you establish a benchmark
for a procedure?
And to me it seemed obvious. You establish
the benchmark on the surgeons who are
actually good at the procedure, and
that's what we did at Yale.
But to do that, you've first of all
got to develop the metrics.
And one of the things I learned fairly early
on is that you have individuals who
have lots of opinions about
how to do a procedure.
And when you check with colleagues that you
know that are pretty good at what they
do, they don't necessarily agree
with that approach.
So one of the first things about
proficiency-based progression is
establishing the metrics.
What I do is I get three to five clinicians
who are good at the procedure, not
necessarily world leaders, but
individuals who you'd be
happy operating on you, your child,
your partner or your family.
I get those three individuals or five individuals
together and we work through a
straightforward or reference approach to
the performance of the procedure.
This is a procedure that the
trainee should be
able to do at the start of
their learning curve.
And we assume there's going to
be no complications.
We assume it's being done on a
straightforward patient.
And we work out what are the phases of the
procedure, the chunks of the procedure.
And within each phase, what are
the procedure steps and
instruments that the trainee must
use to achieve that phase.
And then most importantly, what
was new to training
was we established deviations from
optimal performance.
Okay, we're looking at a trainee or somebody
doing the procedure and they do
something you probably shouldn't
have done that.
Now, we call that an error and it's a
deviation from optimal performance.
Medicine and surgery have difficulty
conceptualising that because they
consider errors, critical error.
And that is something that you
see a consultant or a
trainee doing and you go on, what
the hell are you doing?
It's something that's going to compromise
patient safety or the safe completion
of the procedure.
So those are your building blocks for the
procedure or for proficiency
based progression.
You have the steps of the procedure,
you have the phases,
you've got the procedure, you break
it up into chunks or phases.
You then have the steps, the
errors, which are
deviations from optimal performance
and the critical errors,
which actually compromise the procedure
or patient safety.
Then you've got to validate because
I mean those just might
be the opinion of three surgeons
who were there on the day.
And so one, you've got to
then get consensus
from their peers, not agreement,
consensus.
Okay, that may not be the way I would do
the procedure, but it's not wrong.
Then you would take them to construct
validity, do the metrics.
One, can you observe them? Two, can
you score them reliably?
And then three, do the metrics distinguish
between the objectively assessed
performance of experienced
and novice operators?
And by and large, I mean, I have yet to see
a study where they don't discriminate.
And once you have construct validity, then
you've got your proficiency benchmark
because the proficiency benchmark
is based on
the main score of the experienced
practitioners.
How many errors did they make? Because
every procedure has errors.
I mean, even the best surgeon
in the world is going
to make some errors and possibly
some critical errors.
So how many steps must they complete under
how many errors and critical errors?
So there you've got your proficiency
benchmark.
The metrics also point you towards what
simulation you should use.
Unfortunately, that's not
the way it is today.
People build a simulator and then
they go look and then
say, okay, the simulator will fit
this procedure or that task.
That's not the way it should
be done at all.
You first of all need your metrics.
The construct validity will give you your
benchmark, but it will also tell you
which parts of the procedure the trainee
finds most challenging.
And okay, you want something that's going
to simulate the entire procedure,
but you also want something that's
going to allow the
trainee to hone their skills for that
particular task or phase.
And so part task trainers are
very useful here.
And if you develop your simulation from the
metrics, the validated metrics, you're
on a share winner because everyone thinks
that the simulator, it's got to be
pretty, it's got to be full physics,
virtual reality.
That's not the case.
So once you've got your metrics, you've either
built or decided what simulator that
you're going to use, you then
train your faculty
on the metrics because they
have to be able to,
one, they have to know the metrics, two, they've
got to be able to give the trainee
explicit constructive formative feedback
using the metrics on their performance.
And what we find is this accelerates learning
enormously because the trainee also
has to know the metrics before they
turn up to the course.
They preferably have to learn the
metrics usually from online.
We know that given the trainees, the information
and lectures really doesn't work.
There's a study published by
Stefano Puliatti in
the British Journal of Urology
fairly recently.
And what it shows is that trainees who
got the metric based information in a
lecture took 100% longer to reach the proficiency
benchmark than the proficiency
based progression train group who
got the information online.
So the trainees, they would turn up to
the course knowing the metrics.
They would have to pass at a proficiency benchmark
on the online part of it, again,
established on the mean of the
consultant's performance.
They would then progress in a skills lab.
They would be supervised and trained by a
faculty who could be a fellow who knows
the metrics and is able to give them explicit
constructive formative feedback.
And they continue training until they
demonstrate the benchmark.
It doesn't matter whether they take two
hours, two days, two weeks,
it doesn't matter.
They must demonstrate the proficiency
benchmarks before they progress
to clinical practice.
And what we see when we do
that is that trainees
perform on average about 60% better
in the operating room.
The systematic review and meta analysis published
by us in 2020 shows that trainees
who go through a proficiency based progression
training program make 60% on average
60% fewer objectively assessed inter-operative
errors than trainees trained on a
quality assured traditional
approach to training.
And the key to that approach,
Tony, and you've
mentioned it already is the idea
of deliberate practice.
So it speaks to the primacy
of feedback and the
training of the trainer and why that's
so important to PBP.
It's certainly something that stood out to
me when I've seen the in-person training
and the kind of feedback that the fellows
or the trainers are giving
to the trainees.
Can you just briefly expound on how important
that is to PBP and how deliberate
practice differs from repeated
practice in your mind?
Yeah, I mean, but if you go back, you've
got to go back about it.
One of the things I noticed when I was doing
trainings in the late 1990s, early
2000s, is if you went from training station
to training station, the feedback that
the trainee and the style
of the trainer was
very different from consultant
to consultant.
And it wasn't the skill level
of the trainee.
And Erickson developed the idea of deliberate
practice decades, a few decades ago.
And he talked about learning music
or high skilled performance.
And he talked about deliberate practice.
And he wasn't as explicit as they would like
them to be about deliberate practice.
But what he meant, what he actually said was
that trainees learn faster if you give
them constructive, explicit,
detailed performance on
what they were doing right and what
they were doing wrong.
If you transpose that onto medicine, we need
the metrics to be able to do that.
And you need a trainer that
knows the metrics,
but you also need a trainee that
knows the metrics.
When a music student is playing the wrong
notes, they can hear that so they can.
The trainee who's learning to perform a
procedure, a surgical procedure,
they may not know it.
Whereas if you ensure that they actually
know the information in an online
curriculum before they turn up to the skills
lab, they're going to recognise it.
And what it means is in the past, in a training
lab, the trainees did the task over
and over again, and hopefully they got
it. Frequently they didn't.
And on top of that, we never assessed
the performance.
Today, what happens is we give the trainee
explicit, constructive, formative
feedback in a timely fashion, proximate to
when they actually did something wrong.
Or even you might even say to them, are
you sure you want to do that?
For example, if they were playing staples
at an angle rather than perpendicular to
the longitudinal axis of the colon, you
would say, are you sure you
want to do that?
And they'd say, well, what do
you mean? And you would
point out, is that an error or a critical
error in the metrics?
And it's amazing how quickly
they actually get up.
I mean, what you have to remember is that
the trainees, these are bright
individuals that are committed and
they will learn very quickly.
One of the issues, Frank Lewis, who was
the chairman of the American Board of
Surgery when I worked in the US, we would
go for a beer after meetings.
Tony says, I like this idea
of proficiency based
progression. He says, but I have
two concerns about it.
Number one, if you're setting the benchmark
on the consultants, he said, are they
attending? Is this going
to be far too high?
He said, we set the benchmark today
on minimum competence.
And the other concern that he
had was how long is it
going to take to get somebody to
the proficiency benchmark?
And to be honest with you, those two questions
remained unanswered until a few
years ago and have only recently
been published.
Those questions were also very much part
of the program approval process that we
had to undertake in University
College Cork,
where there was always the question as to why
your proficiency benchmark or passing
scores were so significantly
higher than what would
have been the departmental norm
or the university norm,
which would be arbitrarily set
at 60 or 70 percent.
So you've spoken extensively and written extensively,
Tony, about simulation, and
it gets a lot of attention
in medical education,
the shiny new thing or sometimes a
solution looking for a problem.
But what's missing when you
put someone into a
simulator without the proficiency
framework around it?
The simulators are more
developed now as a
marketing tool rather than
a genuine training
tool.
And the reason is that, well
I think one of
the reasons is that they're
more for marketing
is that the manufacturers tend to be the
purchasers, the biggest purchasers
of virtual reality,
physics based virtual reality
simulation, number one.
And so they want something that's
a marketing tool.
Number two, it's expensive to develop the
metrics and it's even more
expensive to build
it into the virtual reality simulation.
A lot of medicine doesn't realise that.
They think it's just like
a video game and
you just tell it to measure this, measure
that.
And that's not the way it works.
If you're talking about highly
sophisticated physics based simulations
that are derived
from mathematics and engineering
solutions.
So this takes time.
But what we find time and
time again, if you
don't have the metrics in
a simulation, really
what you've got is an educational
experience.
Now, if you don't have the metrics in a
simulation and the simulation
is good enough, if you
have somebody standing beside
the trainee who
knows the metrics, they can
use that simulation
as a tool for the delivery of a
metric based curriculum.
And that's to a large extent, that's what
we're actually doing today.
We're supplementing what the simulation
should be doing.
And I think part of the problem is that I
said industry was the largest purchaser.
That shouldn't be the case.
It should be the health care service
organisations, which are the Department
of Health around
the world that are purchasing
simulations on
large scale for all of their
teaching hospitals.
They or the training body should then say
what the curriculum is, what
the metrics are,
what the benchmarks are.
And so that's not what we find.
I mean, frequently when I go to training
centres around Europe that we
see simulations and
all, some of them are in the box.
They haven't even been taken
out of the box.
And we're talking like a few years
after they're purchased.
And some of them that were
taken out of the
box and they were used for
a photoshoot or a
television appearance, they're
gathering dust.
You know, I mean, there are so
many reasons for it.
I mean, it's a change.
You've got to learn how to use it.
You've got to learn the metrics.
You've got to learn how to
use the metrics.
You've got to develop the metrics.
You've got to benchmark.
You've got to have validation.
And it's easier just to
use something like
Likert type scales, which
give generalised
qualitative feedback to the trainee.
And it's unfair on the trainee.
And I think that it slows down
their skill development.
Why?
No, it slows down their skill development
and the way that they go about learning
procedures.
That speaks to an earlier
question in terms
of why the methodology hasn't
been adopted
widely, is that it's pushing against that
status quo, which reverts to that more
experiential type of learning.
Yeah, that's right.
I was in Australia, I was in Melbourne at
the Society of Robotic Surgery.
I was given a keynote and somebody from the
audience asked me, OK, Tony, if you've
all this evidence, why hasn't
it been adopted by now?
And I hadn't thought about it until that moment
and the words were out before I'd
actually realised I was thinking it.
It's a failure of leadership.
The reason that the Arthroscopic Association
of North America, that European
Society, that the Orsi Academy, the reason
that they're using this methodology is
leadership.
From the top down, you know, whether it was
Rick Angel, Dr. Rick Angelo, Professor
Albert Obreda or Professor Alex Mattrie, they
said, this is what we're doing.
I want the metrics for this.
I want the benchmarks.
Tony, tell me what simulations we need.
What do you need?
The reason it's happening in these organisations
is it came from the top.
But would it be?
Would it be not the case, Tony, though?
That it can't it will never receive widespread
adoption if it's always reliant
on leaders like Alberto Breda, Alex
Mottrie and Rick Angelo.
It will need something more and rather than
just better leaders, it's going to
need its either medicine will eventually
have to adopt this because insurance
companies or others, other stakeholders in
the in health care will oblige them to
do so.
Yeah, but I think you've got to
start with leadership.
And I think what each one
of those leaders
are doing, they have juniors coming along
behind them that basically are 100% convinced
about this approach to training.
But you're quite right.
I mean, I was having a conversation with
one of my former fellows who
is a manager for
Metronic and training.
And he said, "Tony, this is
not about evidence."
He said, "There's sufficient evidence out
there that shows that by
any stretch of the
imagination, whether you're talking about
training effectiveness, whether
you're talking
about training outcomes, it
doesn't really matter.
The evidence is there to basically show
proficiency-based progression as a
superior approach to training.
And it's inevitable that medicine
will have to be adopted."
And each argument that has been raised to
say, "Well, we can't really
do proficiency-based
progression because we're not going to
get everybody to proficiency.
It's going to be too hard to develop
the metrics and so on."
These have been shown to be inaccurate.
The benchmarks are too high.
And one of the last and I think
significant, most significant
pieces of evidence comes
from Stefano Puliatti.
And how his study was looking at, "Okay,
can we train everybody to proficiency?"
But he also looked at, "What
does it cost?"
Because one of the bigger arguments was
that the metrics were too expensive
to develop.
And what Stefano's data shows,
if you're only
going to train 12 trainees,
the proficiency-based
progression approach to training is 100%
more expensive than the conventional
approach
to training.
Okay, the conventional approach
to training,
you only get 58% to proficiency,
whereas the
PBP approach, you get 100%.
Well, let's ignore that.
What he also looked at, what
happens if you
roll it out to say the size
of the US or maybe
metronically training?
And if you're doing 500 trainees a year, what
he found was that it costs about 1.7
million to train 500 trainees to proficiency
using the PBP approach.
It costs 3.5 million to train
the same number
of trainees using the conventional
approach.
One of the things that I really didn't
expect from that, so I thought
that we would have
to train like 100 or 150 before we would have
cost equivalents between the metric
development, the cost of the metric
development and the training.
That's not what he found.
The cost equivalence point
was 25 trainees.
That sort of data is very hard to ignore.
And the medical device manufacturers
are looking
at this quite closely because
they have devices,
new devices coming out all the time, and
there's a significant proportion
of their devices
fail in the marketplace.
And they get the device back to the
manufacturing plant and they dismantle
it and they take
a look at what there's something
wrong with the device.
There wasn't anything wrong
with the device.
This is particularly the case for the
CRT or the pacing devices.
And they've always suspected,
but they couldn't
really say publicly, well, we
think it's probably
the clinician is the reason
the device failed.
The data is now there.
The assessment methodology
is now there to
say conclusively, yes, this
was a device issue
or no, this was a clinician issue.
And all of that means that
it's uncomfortable
for medicine because we can
now, rather than
just subjectively say that somebody's not particularly
good at what they do, we now
know that if somebody's not
particularly good
at what they do, it almost
certainly is going
to impact on patient outcomes.
Because one of the big issues
that I've run
into over the years is individuals
in leadership
in medicine and surgery say, okay, Tony,
you've got this fabulous methodology,
but it makes
no impact.
It makes no difference really
on patient outcomes.
The data now is 100% clear.
The study published in 2013 in the New
England Journal of Medicine
by John Berkmar, what
he looked at was attending level laparoscopic
gastric bypass procedures.
That is, none of these procedures
were done by residents.
All of them were highly experienced
laparoscopic bariatric surgeons.
He very recorded their performance.
He had them blindly assessed as to who
was who and what level they were at.
And they banded the surgeons
into four levels,
those that were not particularly
good to middle
quartiles and a group that were
performing particularly well.
And what he found was he
followed up all of
the patients those surgeons
operated on over
a period of six years.
And what he looked at was,
okay, what are the
outcomes for the patients
of those surgeons?
Not surprisingly, what he found was the
surgeons who were assessed at
the outset of the study
as performing best had significantly
better patient outcomes.
To be quite frankly, that wasn't
really the surprise.
The surprise was the magnitude
of the difference.
The smallest difference between the
surgeons who were performing
well and the surgeons
who were performing badly
was 50 odd percent
patients who were operated
on by the surgeon
who was pretty good had a 50% plus
lower re-operation rate.
The highest difference was 81%
in terms of mortality.
Mortality rate was low in
the study, but the
difference was still 81% and
it was statistically
significant.
To be frank with you, that
was a shock to me.
I was working with George
Shorten, who was
the Dean of Medicine and
Cork at the time,
and we were having a discussion in the
boardroom one day and George said,
"Yeah, yeah, yeah,
I know, Tony, proficiency-based
progression
produces 60%, 70%, 80% better
skills levels
and trainees in comparison to...
But a lot of that's going to wash
out in clinical practice."
And to be quite frank with
you, I believed him.
I felt that too.
The data now shows that that's
not the case.
And we also have evidence that
proficiency-based progression, simulation
training, the impacts
on patient clinical outcomes to the
magnitude of about 60%.
In one of the studies where trainees were
trained on a proficiency-based
progression
training program to perform
an epidural for
women that were having a baby,
we saw a 50-plus
percent difference in epidural
failure rate
for trainees who didn't have
the PBP training.
So we know that the training, it's not
going to be 5 or 10% difference.
We know that the outcomes are going to
be of the magnitude of about 60%.
These sorts of findings is
going to make it
very difficult for medicine
and surgery to
retain the line that they're currently
retaining, which is non-adoption.
And it moves past a surgeon
too, doesn't it?
I mean, PBP has now been implemented for
communication skills in Cork
University Hospital
and they've recently been used to train
entire teams in a surgical
context or in, say,
wound care or patient management
post-operation as well.
I mean, I don't know that
I was surprised at
the findings because basically
the underpinnings
of proficiency-based progression
are very simple.
You take somebody that's good
at the task, not
world-class, but somebody that's
good at the task
and they begin to work, you know,
8 till 6 every day.
They're good at what they do.
They're diligent.
They attend to detail.
They make mistakes.
They recognise and they fix them.
You take that individual and you work
out what is it they do?
What is it they're good at?
You know, what sort of performance level?
And you work out, okay, what
are the units?
What are the metrics?
What are the steps?
What are the errors?
What are the critical errors?
And you can apply that to anything.
We applied it with Dorothy Breen at Cork
University Hospital to a communication
skill.
And lo and behold, guess what we found?
The trainees who were trained using a
proficiency-based progression training
program performed about
60% better than trainees who had gone through
the national training program.
In Ireland, the health service
executive have
a national training program
for the handover
of a deteriorated patient.
And before you go clinically,
it doesn't matter
who you are, you've got to pass
this training course.
So all the trainees who turned up to our
train--all of them had passed
that already.
And when we assessed the
trainees who turned
up, who had already passed the
national competency
communication for the handover
of a deteriorated
patient--when we assessed them,
none--none of them
demonstrated the proficiency benchmark.
So you start with something
relatively simple.
You start with somebody that's
good at it, work out what
they do, what they don't do, benchmark
on those individuals,
build a curriculum around those metrics.
And the outcome is pretty certain.
Each time, you're going to see at least
60% improvement in performance.
And it's moved out of the medical
and surgical space as well.
And you've read that case study that we collaborated
on with Reach Active as well,
in terms of the procedures and processes
that they characterised as well,
and the results were similarly striking
their word in that.
Yeah, my brother's own a fairly
successful utilities company.
And he had read, I don't know why he had
read my book, but he'd read my book.
And he said, I think you have a solution
to a problem that we have.
And so we talked.
He's in the country, he was trained
as an accountant.
And during the summers when he
wasn't at university,
he used to work on the roads putting
utilities into the ground.
And on one occasion, he put a jackhammer
that he was working on straight through
an electric cable, a high voltage electric
cable, and it blew him across the road.
And Seamus is quite a bright guy.
And as a company CEO, he knew that this
was working in the utility sector.
This was a risk for his workers.
And in Ireland and in the UK, what we see
is that there are a number of deaths
and very serious injuries from striking,
usually an electric cable,
but could be a water or a gas cable.
And so he asked me to develop a proficiency
based progression training program
for the identification of the
utilities in the ground
and then being able to safely excavate
those utilities.
So we went through the same program.
We took some of their workers who were
experienced and good at the job.
We developed the metrics, we validated
them, we set a benchmark.
They built a simulation yard.
They took a part of their space, they
put utilities into the ground,
and they could switch those
utilities on and off.
And the trainees had to go through an
online didactic course, which
I think you built,
and they had to pass that before they
got to the simulation space.
Now, remember, what we're talking about
here is non-university people.
These are youngish fellows, labourers who
are basically digging out
the roads and so on.
These are not university graduates.
They're not medical trainees.
And we found the exact same thing in the
utility sector as we found in medicine.
The trainees who went through a proficiency
based progression training program,
at the end of one year, Reach Active were
saving over a million euro a year
in utility strikes that they avoided.
So it works across the board, but you
always start with the same process.
You'd start with an individual or an individual
who are actually good at the task.
And that's a very important
message, and it's
one I think that anyone who's
listening to this
and who runs the training program
should pay heed to.
And speaking to that particular
audience, and
if people are running their
own training program,
everything from vocational training to
something like teacher education,
and they want to move towards proficiency
based progression, Tony,
what's the first question you think they should
ask about their current curriculum?
Who's good at the task?
Because you need to identify
those individuals.
And it can't be just because they're your
buddy, or it can't be just because
you like them,
or it can't just be because
they're senior.
You need to identify individuals
who are good
at doing the task that you
want to develop a
proficiency based progression
training program.
Then you're going to have to
learn how to do that.
You're going to have to either go back to
you either going to have to talk to me,
or somebody like me, you're going to have
to go back to my papers or the book,
and you're going to have to work
out the methodology.
I mean, I mean, one of the things that
we're seeing is, or I'm seeing,
is individuals who think they know
how to use the methodology.
And then they ask you, well, what
do you think of that?
And you're going, might have been better
if you'd done this, that and the other.
I was at Adelphi, I was there
as an observer
at a Adelphi meeting in Miami
a few years back.
They developed the metrics for
a colorectal procedure.
And they hadn't developed the metrics
for a colorectal procedure.
They were still developing them
during the Delphi.
And they asked me at the end of the meeting,
why do you think that went to me?
And I says, what do you have?
And they had nothing.
I mean, they had added to
the metrics, but
I mean, they had no idea
whether the metrics
were actually going to stand up
to more robust assessment.
Needless to say, the metrics, they
weren't, they didn't go anywhere.
I've never seen them published.
I've never heard of them being used.
That meeting cost over a hundred
thousand dollars.
I mean, I'm from Northern Ireland and we
would consider that a bit
of a waste of money.
I mean, I like to see what's coming
out at the end of it.
It's like if somebody's going to do a procedure
on me or my family or a friend,
I want to know, okay, are they any
good at what level of it?
And I don't need them to be expert, but I
do need them to do the procedure.
And I do need them to be able to
do the procedure safely.
Sure.
So you need to find in your particular
industry for the task that you're
looking to train,
you need to find people who
are objectively
good at doing that task and
start with them.
Yes.
And have a reasonable attitude.
You don't want somebody that
you need to do
therapy on whilst you're trying
to develop the metrics.
I mean, it's individuals.
Someone who's collegiate
and collaborative.
Yes, yes, yes, yes.
But I might put it more robustly.
After all, I am a psychologist.
I mean, it's just too much
like hard work.
It's like pulling teeth.
I suppose it's an issue across medicine.
It's an issue across every industry.
There's a tension between
time and training
and time where they need to
perform their task
or to deliver their service, be that in
medicine or be that in any industry.
So how do you make the case for PBP to a
department head or to an
employer who says,
I don't have the time or the
faculty for this?
That's the wrong attitude.
They have to if they're
in the business of
training, they're financed
and they're funded
for training and the trainees can't just
be used for the delivery of service.
If they don't want to do training,
get out of that business.
But if they're if they're in the
business of doing training,
they need to go and look at Stefano
Puglatti's data.
Basically what it says, number
one, you can
train somebody faster and more
cost effectively using PBP
than you can with a traditional approach.
But what they also must look at
is the Berkmar data.
It's unequivocal.
Individuals who are not trained properly,
who are not benchmarked properly,
they are going to have more bad outcomes,
patient outcomes,
than individuals who are
trained properly,
who are skilled and knowledgeable
about the task
that they're supposed to do.
And I mean, the hospitals in the United States,
I think that's starting to change.
I mean, somebody said to me when I worked
there that a lot of the hospitals
and health care organisations made their
profit on the complications
that the clinicians, you know, were, you
know, that caused or involved with.
That's changing.
The Center for Medicare Services
is now saying,
okay, what's what's the complication
rate, the
average complication rate
for that procedure?
Okay, 75%, 7%, whatever.
We'll allow that and a bit
of wiggle room.
You see anything more than that?
The hospital has to eat the cost.
Until that sort of approach is
introduced more en masse,
I'm not too sure there's
an incentive there.
For example, the manufacturers, as I said
earlier about a lot of the devices fail,
and it's nothing to do with the device.
If the device is ex-planted and
another one is implanted,
the manufacturer gets paid
for that device.
So there's no incentive there.
There has to be some sort of incentive.
But I would hope that the organisations
that are involved with training
are of sufficient integrity
and professionalism
that they take their commitment
seriously.
Trainees, yes, they deliver service,
but the onus is on the organisation
to train them.
And when we go around the world,
what we see is that onus
or that training delivery varies
very considerably.
I have a lot of Italian PhD fellows,
and some of the stories that I
hear are quite horrendous
in terms of the training that
those individuals get.
There was a guy who applied.
I was the head of the Laparoscopic
Center in Emory in Atlanta.
I was the director of research
and training.
And we had one Italian guy that applied.
The chairman of surgery sent
over his application.
He said, "This guy's finished
his residency,
and he's applied for residency here, and
we think he's made a mistake."
He was obviously applying for one of
the Laparoscopic Fellowship.
So anyway, we got the guy in
and we interviewed him.
He said, "No, no, no."
He said, "I was applying for residency.
I've completed my residency in Italy."
He said, "But I can't operate."
You know, I mean, he couldn't
do the residency.
That's why he applied for the residency
rather than the fellowship.
That brings us right back to the
very first question.
How do you know that someone is qualified
to do the task to do an operation?
Yeah, but a trainee is a trainee, and
I don't have an issue with that.
You mean it's only once a day that
they're independently licensed.
Did they have an issue?
I think that there should be no tricks.
There should be no hidden agenda in it.
The idea is to have people
good at what they do,
knowing what to do and how to do it.
We need some sort of a verification,
because at the minute somebody qualifies,
and the assessments that they get
post-qualification are,
shall we say, less than robust.
They're mostly didactic.
There's maybe a portfolio of procedures
that have already happened,
already been done.
What we need is some sort of an assessment
of the intraoperative performance
of the clinician on a straightforward
patient.
Because clinicians, I mean,
cases can go...
I'm not saying that all clinicians are
really not skilled or not up to...
I'm not saying that at all.
And every clinician can have a case that
just goes particularly badly
through no fault of that surgeon.
But if you pick a straightforward case
that went straightforward
and that case is assessed,
I don't see what the surgical community
or the medical community
have to be concerned about.
We have no robust verification of the performance
levels of our clinicians.
This is not just...
The procedure-based medicine individuals
are going to be relatively
straightforward,
because we can see what they do and
we can ask them about it.
The internal medicine clinicians, I think
that's going to be more challenging.
But it's not an issue, behaviour
is behaviour.
If they do something or say something,
we can assess that.
Was that correct?
Was it inappropriate?
And we're not looking to
catch anybody out.
Catholic University of Leuven in
Belgium is introducing
a proficiency-based progression
training program
into the robotics training program
and across residency.
And I was given a lecture or
a half-day session
and I was given one of the
keynote lectures.
And a lady who was head of training,
she stands up and she says,
"Hold me, hold the second
Professor Geller.
Are you telling me that you tell the
residents what to do online
and you assess them on that and they can
do it as many times as they
want to pass it?
Are you saying you tell them what to do?
You make sure that they know it before
they turn up for training?"
Then in training, you allow
them to do the
task or you supervise them
to do the task.
You assess them on the same metrics.
You give them feedback on what they did
and they can do that as many times
but they still have to demonstrate
the benchmark.
She said, "You're basically telling
them what the assessment is
and then you're telling them what they
did wrong so they can fix it."
And I said, "Yes."
She said, "That's not the way we do it."
She couldn't get up.
She couldn't get the tricks in us here.
The idea is to help the trainee to
get to the performance level.
Anything that we have to do
to get the simulations
and the curriculum are really
just there to help them.
We've invested...
Well, the kids have invested
so much of their
time and their effort getting
into medicine.
The onus is on us to make sure that
they can be a good clinician.
They can be a good doctor.
And there's no tricks in that.
If they know how to do a straightforward
procedure
and they're interested enough
and talented enough,
they'll learn to do more complex stuff.
They don't know how to do a straightforward
procedure or performance safely.
They're a risk to patient.
So a proficiency-based profession, we're
respecting the trainee's time,
we're respecting the trainer's time
and we're also respecting their
future patients.
Yes, that's right.
And training them with the most robust
methodology we can right now.
Yeah.
I mean, it's a scientific approach
to training.
It's objective, it's transparent,
it's fair.
I mean, the benchmark...
Repeated, but as key.
Yeah, yeah.
I mean, there's no tricks on this idea,
where an examiner gives you
these random questions
of this curveball case that
they saw 30 years.
No, that's not acceptable.
All that tells you is about the examiner.
I mean, I want to know, do
they know the basics?
Can they do the basics?
And I certainly don't want them to be
doing it on a competency level.
I mean, I thought I understood, you
know, when we say competency,
we mean minimum competency.
And I would hear that now and again, and
nobody says minimum competency.
And when I started reading about competency
and I'm going minimum,
and this is what we're training
to, and I'm going,
oh no, that's not acceptable.
Not procedure-based medicine,
too high risk.
And proficiency, I mean, the
question, Frank Lewis,
can we train everybody to proficiency?
Yes.
In every procedure, probably not.
You don't probably don't
want them in every,
but we can train the absolute vast
majority to proficiency.
And I think the outcome of that
approach to training,
it's certainly better for the trainee,
because I have a fellow,
he's now a consultant in independent
practice.
He said that when he learned proficiency-based
progression,
the metrics, I was training him to
do the assessments initially.
He said it changed his approach to
learning a new procedure
and it changed his approach to
how he trains others,
because here you have a structure,
objective, explicit, transparent,
and fair approach to training.
I mean, there's no curve ball, there's
no hidden agenda here.
And even if a consultant doesn't
like a trainee,
they can only assess what they objectively
see, their performance.
They can't assess, I don't
like this individual.
So Tony, to wrap up, so
30 years from now,
when someone is writing the history of how
professional training finally caught up
with what we know about
human performance,
what do you want the chapter to
say about this period
and about the work that you've done?
Well, I hope it clearly shows that,
number one, that it works.
I mean, I'm not too sure, I mean, I've been
working at this now for like 30 years.
And somebody said to me once,
"Tony, why are you still doing this?"
And I said, "If I'm stubborn."
I mean, if something's right and
you know that it's right
and you've got the evidence
that it's right
and it's so hugely impactful, why
would you not continue?
I'm not sure that'll be talked about.
And I'd be happy enough if, number one,
the approach had been adopted.
And number two, it had permeated into
other aspects of training
or performance even.
And medicine is going to become
more robotic.
That's 100% certain.
Artificial intelligence, as long as the
bubble doesn't burst too bad,
I think it will eventually become
an integral part
of simulation-based training and proficiency-based
progression training.
But artificial intelligence, everybody
tells me about,
"Okay, we're going to be able
to assess performance."
And we'll be able to...
No, they won't.
Because artificial intelligence,
first of all,
needs to be assessed, able
to assess the metric.
How far an instrument travels, how efficiently
the instrument travels,
to be quite frank with you, it's
a measure of process.
Like, if I'm going from here
to Dublin, I can go...
If all you're assessing is, "Okay, Tony drove
to Dublin from Belfast to Dublin."
It doesn't tell you how many
car accidents I had,
times I broke the speed limit,
the number of
pedestrians that I either
hit or nearly hit.
Measures of process are insufficient
for effective training.
And that's what AI does at the moment.
It needs to be able to assess the granular
performance of the surgeon.
Okay, was the linear stapler, was it
perpendicular to the longitudinal
axis of the colon
before it was closed?
How many staples did they have?
The suturing.
Where was the...
How was the suture driven
through the tissue?
Did it tear tissue on the way through
where the suture is equally...
It's not able to do that at the moment.
I absolutely think it will be,
but not at the moment.
And we're going to need huge
training data sets.
Properly performed surgeries to the
benchmark to train this.
Yeah.
And I think that the AI will probably be
introduced first into endovascular spheres,
because the endovascular virtual reality
simulators, the physics based virtual,
they're the best on the market.
They're the most expensive, but they're
the best on the market.
The surgical simulators have a long,
long way to come in.
But I mean, I'm a still huge fan of
virtual reality simulation.
I think that's part of the future.
But medicine must see simulation
as a tool.
Artificial intelligence needs
to be seen as a tool.
The clinician at the end of the day
has to make the judgment.
The metrics of optimal and suboptimal performance
has to come from the clinician.
And I think what we're going to do is
proficiency based progression
and the methodology,
the scientific methodology that underpins it
and the validation evidence means that
the direction of travel is inevitable.
And I think it's going to take somebody in
the leadership to say this is happening,
whether it's in government, whether it's
in the scientific societies.
I don't know.
Well, we watched a space with
interest, Tony,
and continue to impact it with more
papers and publications.
Thanks for your time, sir.
Thanks, Patrick.