Subscribe
Share
Share
Embed
Most training is sold on confidence. Show Me The Evidence is built on data.
In every episode we take a single study, clinical trial, or systematic review and work through what it found, how it was designed, and what it means for the way we teach and assess skill. We focus on metrics-based training and proficiency-based progression, the approach that asks learners to demonstrate measurable competence before moving on, and we trace its results across surgical, medical, and professional education.
This is a podcast for learning professionals and medical educators who want more than opinion. Expect plain-language breakdowns of the research, honest discussion of what the evidence does and does not support, and conversations with the people behind the studies.
If you make decisions about how people are trained, we think you deserve to see the evidence first.
Today I'm joined by Dr. Jorio
Mascheroni
I've known Jorio
for about 20 years.
Jorio is an engineer who
subsequently did
a Masters in Technology
Enhanced Learning
and a PhD in how optimally to
implant a CRT device.
Jorio has worked in industry
certainly since
I have known him and Jorio
is very welcome.
Hello Tony.
Jorio, can I start out with,
okay you've worked
in the medical device sector
for quite some time
certainly alongside known
you. So what
got you interested in educational
training?
Yes, thanks Tony. I got interested in
education from the early
days of my career
medical industry. I had the
privilege to to
witness the introduction
of one of the most
successful therapies of the
last 30 years in
cardiac pacing which is cardiac
resynchronisation
therapy. It's a therapy
for cardiac rhythm
abnormalities. Maybe we can
talk about that
more specifically later and
because it was a
totally new approach and a
new type of procedure
there were a lot of challenges
from the
pioneers that started with that
therapy and I had the
privilege to work with some
of them go through
the challenges and I realised
that the ability to
perform certain steps properly
and avoid
mistakes was a determinant for
success. So at the time I
also had the possibility to work on
developing some training programs
for these new
implanters new to therapy
at least and that's
how I started getting involved
with education.
The second aspect was in the
few years at the
beginning of this therapy
we had the chance to
start working with the beginning
of simulation
with virtual reality and we
had the chance to
get involved in the development
of simulator
for teaching cardiac resynchronization
therapy
procedure and so that was
also interesting for
me because it opened more
possibilities to move
training away from the patient and to
experiment how to do things outside
the operating room.
So all those years ago when
you started I mean
you talk about simulation you
talk about education
so how was education and
training approached
back then? Well I think there
was no single way
there were multiple degrees
of quality and
quantity as well and in general
if I compare
to what happens and I see
today what I do
training was certainly more knowledge
based at that time
and the lack of simulation
tools of course made
it more theoretical and at
the time the reality
was and still today that
most of those
physicians learning a new procedure
or a new skill
were and are learning on patients
and so I have
to say also that we were looking
more at technical
aspects at that time and
I found looking
backwards that there were
multiple ways
experts suggested to do
training and of
course that meant that there
were also different
training contents and different
teaching
ways even for the same for
the same procedure.
So there was a lot of variability
in terms of
recommendation of how to do
it there was a limited
chance to experiment outside
the training room
sorry the operating room and
certainly we were not
really measuring what at the
end of the training
people learned we had an idea
we had an impression
that people were getting better
but how far they
went it was unclear. So from
a personal perspective
you know somebody relatively
new to the
training education training was
the impression I'm not
about you know your views
today but looking
back at the time did you
think the approach
that was used for training
and education
do you think was it effective
and you know
why would you say that? I can say we
believed it was effective it was
based on impression more
than data and again
there were many different
ways of delivering
training we tried at least
in our group to do
the best with what was available
at that time.
My background as you mentioned at the
beginning was biomedical engineering
at that time so I
feel I was more focusing
on the technical
aspects of the devices of tools
that were used for the
procedure but there was
still a lot of
variability in the way each training
was conducted. I think
since then the journey has been
long and certainly
it's not over I personally learned
a lot from what
I did at the beginning I
believe that was
a successful approach I would
say today but
it would be difficult to
prove that was
effective or as effective as what
we are doing today.
So the impression was that
it was helpful and
people who knew very little at
least could practice
a little bit outside the
operating room and
that's certainly an advantage
at that time
thanks to the simulation. The use of
simulation was not probably the
best at different levels
from the very first simulator
created a lot
of interest many times they
were considered
the attraction of the training
event how well
they were used and I would
say it went through
spectrum from room decoration
to probably at
least the first attempts
of standardising a
procedural workflow so there
was no consistency
in the use of training and simulation
at that time
but I have to say also that
there were at
least on our side many efforts
to do better with
consistency so we started
at that time
especially because very few people
at the beginning knew
how to do things we tried
to find a bit of
consensus of how optimally
we should perform
procedure and there were some
people that the
very first pioneer which I
admire because they
figured out an optimal way
to perform the cases
and when I saw them getting together
and discussing
how to do things I realised
that probably the
power of having a group together
of people who
genuinely tried to help
the patients that
power was unbeatable and of
course was much more
effective than having a single
individual showing
up and telling his own way or
his own story so we
started getting experts
together and we
started having more consistency
in the way we trained
novices on a certain procedure. Again
looking back 25 years probably
we were ineffective
compared to where we should
be and we have
no data to say what people
learned at the end.
It was a it was already a paradigm
shift to be able to take people
outside the operating room
because again
going back to that those early
days of the therapy
people were taught how to perform
the procedure
in front of the patient and
at least the basics
with the first simulator simulators
available at
least the basics were done
outside the room and
discussions could happen outside
the operating
room that was a big change
although yes without
metrics without benchmarks without
standards at that time but it
was an important step
on the journey. And do you
think that the
industry and other training
centres did they
see simulation as some sort
of magic bullet? I
think there was a lot of attention
to simulation
because simulation could help
industry explaining
or I would say at that time
demonstrating the use
of the product at least the
workflow of the use
of the product in a safe protected
environment and
of course that is difficult
to do in the
operating room. At the same time
the simulation was
used in multiple ways and I
would say not always
in the most effective and many
times it depended
on the on the training team
on the experts
that were delivering the training
so there was
inconsistency in the way
it was used so
participants trainees were attracted
by the simulator
themselves because they believed
they gave them
the opportunity to do something
safer but also
was the novelty that attracted
them. Now we
know that simulators by themselves
don't teach much
if there is no solid curriculum
around so maybe
the technology was there maybe
the use was up to
the individual trainer to make it
effective and so you could see
multiple versions of
good or bad use of simulation. So the
shortcomings that you saw in
industry training,
I mean for me it always
struck me as an
educational experience but industry
was doing training on
simulation and as you know
I'm a huge fan of
virtual reality simulation. So
were the shortcomings
just in industry or were they in other
sectors as well such as the
scientific societies or universities?
What's your observation? I
think it's difficult
to generalise if I have to
give my perspective
I think there were many
common aspects
between what happened in industry,
what happened in
the medical field and one of them was
we sometimes underestimated the
how large the gaps were in
certain areas or
we gave for granted some skills
that were not
instead demonstrated. So
trying to build
new skills on a weak foundation
was not always
simple. So before simulators
and that's when
I started before simulator
of course training
happened mostly in the operating
room and that is
definitely not the safest and
most effective way
to do it. We also saw common
aspects with lack
of standardization so training
delivered in one
hospital was different from
training delivered in
the hospital maybe next door
although the therapy
was the same and sometimes
also this happened
depending on which experts
or consultants were
delivering training on a simulator
so this was a
common aspect and so there
was also tendency of
having more speaker-centred
training so the
learner was there to listen
to these pioneers
which of course had experience
that were admired
for what they did but listening
to an expert does
not mean that you become an
expert and so this
was probably a misunderstanding
that was happening
at the same at that time and
by the end of the
training people were satisfied
with the training
but not sure how much they
could do the same as
the speaker that was teaching.
To summarise
I mean I would say common
aspects I think the
most challenging and critical
was the fact that the
learning curve was happening
on patients in the
clinical environment so that
was something that
looking backwards would still
worry me. So I mean
you so you observed all this
for a number of years
at what point did you come
to the conclusion
that a different approach to
training was required
because you seemed to have
all the right
bits in place I mean you had
a structured course
you had a training on
simulation which
allowed training outside the
OR but you seemed to
move beyond that so you must
have come to the
conclusion at some point a
different approach
training was required. What
drove that? I think
in my experience for myself
I was no one single
turning point I think it
was an accumulation
of evidence and there are
some moments when
you wake up in the morning
and you put
together those pieces and bites
and say it's okay that
maybe there is something we can do
and I think it moved from
analyzing more critically
how different
techniques that were adopted in
the operating room and
given in training as well
we're generating
different outcomes so you
start realising that
actually it's not only the
way you apply the
technology it's also the way
the patient benefits
or benefit less from the technology
and you see
that the way you taught somebody
doing something
may determine the outcome
of the patient so
I think I learned that you
needed to pay more
attention to the gaps to be able
to analyse exactly
what the gaps were and try to
find solution to get
people at a certain level
I think if I have
to say and we have known each
other for so long
I think there were moments
like when I
first got across one of
your publication
in our group training group and when
I read that I was at that point
already a few years in my
training position
and I realised that everything
I read there
made sense to me although
was still mostly
applied to minimal invasive
surgery but the
approach made sense for other
types of procedure
as well so at that point I said
okay not only we are
realising that gaps exist
but also there may
be a potential solution that
can be applied to
fill those gaps the challenge
is the gaps were
very variable depending on where
you were looking
and so also the degree of
improvement required
was different at different levels
but that was one
of the moments where you
said okay I may be
novice in education and training
but I could learn from
experienced more experienced
practitioners
that you don't have to reinvent
the wheel
but you can apply something
that has been
validated and proven working
already and maybe
this could help in our sector
as well so that was
a moment I remember very well
and that's probably
the moment we started getting
in touch and I
thought okay there is a direction
the direction
is pretty clear and there is
something we can do
more I have to say I've learned
every year that
there is something else we
can do better and I
think we haven't got to the
point where I said
we have done everything there
is still a long
journey but I think I think
we or I personally
had to go through the different
steps at the
pace that was required to come
to the conclusion or
partial conclusion that we
needed to move one
step forward and I think we've
gone through a long
journey and there is still
a lot of room for
improvement so multiple moments
where accumulation
of evidence showed us that
we needed to move
one step forward and one step
after the other
I think we progressed a lot
it would be easy
to say now we could have skipped
that phase or
skipped the steps but probably
we would not be
here without all the stops we
made on the journey
yeah no I agree with you
I mean it's about
learning the science of this
approach because
it's an entirely different
approach to
conventional training so I think
if I may if I made only
I think I learned one thing
that probably drives
most of the innovation I think
if you genuinely
believe that you want to improve
training every
day and your purpose is to
make healthcare safer
and more effective I think you are motivated
to look for improvements
there was a time we discussed about
accumulation of marginal gains
I think life has been
this type of journey for me
and sometimes the gain
was big sometimes was small
but every small piece
made the construction I totally
agree with you
about that whole approach to
the aggregation of
marginal gains and to a large
extent that's
at the core of proficiency
based progression
so over the last number
of years you've
developed and validated a proficiency
based progression
approach to training of cardiac
pacing and cardiac
resynchronisation therapy can
you briefly explain
what pacing and CRT are?
Yes absolutely so
I've always been involved from
my early years in
medtech on the cardiology
side and pacing
cardiac pacing and cardiac resynchronisation
therapies
are treatments that are offered
to selected
patients who have some cardiac
rhythm abnormalities
and pacing has been cardiac
pacing permanent
cardiac pacing has been around
for about 70
years already cardiac resynchronisation
is more
recent or relatively very recent
because it's about
25 years old maybe almost
30 but all these
therapies require an implantation
of a cardiac
consists typically on a
generator which is
placed in the chest under the
skin and the generator
contains the battery and contains
the circuit
that decides what therapy
and when the therapy
should be delivered and they
are connected to
the heart rhythm where they can
listen to the heart
and deliver therapy to
the heart through
implanted leads inside the heart
and this is the most
traditional way of delivering
cardiac pacing
and recircurization in the recent
years there are
alternative ways that have
been suggested but in
general the therapies are delivered
for patients
who are suffering from any
cardiac abnormalities
and have certain criteria so
this is in summary
what we do and we train physicians
to make sure
they can use the technology
for the safest and
most effective approach to
have a better patient
outcome. So it's okay you're
working in the sector
and the education and the
training did you
think there were major issues
in the education
and training of these procedures
or were you
happy with how it was being
done? I think again
going back to what we discussed
before I think
the first challenge I observed
at the beginning was
that if training or observation
or coaching
happens in the OR, the laboratory
room or in the
cat lab, I think that's definitely
too late and
it's not certainly fair that
the learning curve
happens on a patient so that
that was for me
challenge number one. Challenge
number two was
there were a lot of different
interpretations
of what optimal therapy, optimal
implantation
of the cardiac device consisted
of and nobody
ever compared all those approaches
and of course
depending where you learn
the approach you
could be a very good or a
less good operator
and not only in different
countries there were
different approaches but even
in the same country
sometimes in different hospitals
they were
applying different techniques
with different
level of success and so I
would say the system
the medical education system
was not optimal because
at the end of their training
physicians had
a large variability of performances
despite
being licensed to perform
the procedures so
those were the aspects that
I considered the most
challenging and the most critical
at that time.
Were these simply catheter
wire performance or
was there other aspects of
the procedure that
you considered okay this is
not done optimally?
I think I've learned the clinical
side of the
therapy by working with very
smart clinicians
and I learned at the beginning
at my time I was
mostly focusing on the technology
and the aspects
that of the procedure that
required the
technology to be implemented but
if you look at the entire
procedure I realized over time
that were parts of
the procedure we gave for granted
or were assumed
to be conducted without the
problem instead
skills involves specific domains
were not certainly
guaranteed and that's when
we had to start
looking at gaps not only related
to the device but also
what two complications that
would ultimately lead
to a system a technology malfunction
but was not
were not strictly related to
the technology but
were related many times to
a suboptimal use of
a technology so that's when
we realised that
actually gaps may happen
at multiple levels
at all phases of the procedure.
I remember
Hartwick Redslaff took me to a
hospital in Dublin in 2006
I think it was. It showed
me the first pacing
procedure I ever saw and I
go this is a surgical
procedure what's the cardiologist
doing it and it
turns out that most or well
a significant number
of the complications to do
with cardiovascular
devices are to do with the
surgical end of it
and on the courses that
I observed there
seemed to be very little attention
paid to those
I mean when I was at Emory in 2003 we
were working with the carotid
artery stenting
and one of the highest complications
was on
the femoral axis there was
infections there was
hepatomas and so on and
like two decades
later I was working with
the TAVR procedure
and turns out that the complications
still
existed, so you've developed
a proficiency-based
progression approach to
pacing and CRT
devices so what does that training
program involve?
Okay so I'll start by saying
that it's not
just an interesting educational
experience as you
normally call it. It involves
from the very
beginning a very structured
approach to how
the procedure should be performed
according
to certain standards what
metrics need to be
checked to ensure that trainees
progress and
demonstrate that they can
do the procedure at
a certain level proficiency
benchmark that need to
be defined based on experience
operator performing
the same approach and it requires
after a good
investment into the development
and validation
of everything you created it
requires an extreme
rigour in the implementation
of what this science
has generated. A rigour that starts
certainly with trainers. The
trainers need to be
proficient in applying this
rigorous methodology
and need to demonstrate that
they can consistently
transfer those knowledge
and skills to the
students and they also need
to make sure that
not only in the first second
or third training
they perform but over time
they don't decrease
the level of attention they
have to the details
of a procedure when teaching
and they make sure
that trainees only progress when they
demonstrate all those details. So
it's attention to details
it requires multiple components
in fact it's not
a training event where people
get together around
it starts with it's a
blended learning
curriculum it starts with a knowledge
component which is
usually delivered by a learning
participants have
to demonstrate the proficiency
benchmark on their
knowledge before being able
to advance to a
hands-on training. So faculty
training I think
is critical you need to have
consistency you
have to have very solid agreement
about how the
procedure is performed and
taught to the people
and you need to be very disciplined
in the way
you train and assess the performance.
If you stick
to the methodology the results
are generally very
predictable if you deviate
even lightly from
the methodology you may encounter
some surprises.
Can I just back up a bit why is
the validation important?
I mean surely you develop
a depth restriction
you'll use them why is the
validation important?
I think because you want to make this
approach bulletproof you
want to make it
you want really to make it
as objective as you
can and nobody can come back
and say that's your
opinion that's the opinion of
10 people you need to
make sure that what you have
chosen is what works.
Yeah yeah no I totally
agree with you I
think it has to be objective
transparent and fair
for both the trainers
and the trainees
because this is a different
approach medicine
particularly procedure-based
medicine is not
accustomed to their performance
being assessed
to reach some quantitatively defined
performance benchmark and I think
the onus is on us
to demonstrate okay this
is objective it's
transparent and it's fair
and if somebody's
failed you know they don't
demonstrate
proficiency you can show them why
but the good thing is they
can do it again and they can
do it as many times
as they like but the the rule
is you don't progress
if I may add something to what you
said for the people listening
is sometimes
probably myself as well at
the beginning were
afraid of how participants
how trainees would
react to such a strict methodology
and actually
the the recipients have been
extremely positive
because as you said it is
very transparent
so they know what they are
expected to do
they are not afraid of
having different
individuals evaluating them depending
on the day of the week
or depending on the place
they come from they
know that if they do what they
are supposed to do
they will get where they are supposed
to get and proficiency-based
progression
I insist with that aspect
which is not always
very clear to people is not
meant to fail people
it is meant to take the right
amount of time
people need individually to
get to the benchmark
you expect so we are actually
helping people
finding their way with a very
structured approach
to the level of performance
they expect
themselves to reach and sometimes
they are afraid that they
can't get there and they
get conflicting
information and recommendation
about how to proceed
and here instead is very transparent
so it's been
well received we haven't had
any objection about
why do I have to do this who
says that this is
the best way they have been
looking at ways to get
better to demonstrate that
they could reach the
objective and the great majority
of them get to
where they need to get
yeah yeah, I mean I think
that's an important point because
the whole point
of proficiency-based progression
I mean
everyone concentrates on the metrics
and the assessment
and they think it's the assessment
is the most
important the assessment is
only important to give
the trainee objective transparent
feedback on
their performance and to coach
them how to pass
I remember giving a professor
a lecture in
Leuven and somebody stood up and
said hold on a second
you're educating them what
to do what the tasks
are before they get to the skills
lab and I'm going
yep and they can do it as
many times as they
like go yep and then they get
to the skills lab and
coaching them what to do and
they can fail that
and they get as many goes so
that's not the way
we do things today and I go
our job is to help
them get to the benchmark what's
non-negotiable
is to get to the benchmark
and I know I'm not
checking or we're not checking
how smart they
are I already know that
they've got into
medical school they've progressed
through the system
so the the idea is that
we help them the
metrics are there to help them
and that's one of the
I think the most powerful aspects of
proficiency-based progression it's
about the the objective
explicit formative feedback they get
about performance and my
experience is that
my experience is that they
received that
approach better than what I expected
at the beginning I
thought some people may question
or challenge
or be critical about the rigor
the methodology
requires but at the end
they are all very
compliant and and successful
in what they do
yeah but I think they pick
up fairly quickly
that we're on our side I mean
we'll help them
get to the benchmark I mean
there's no tricks
about that but what's non-negotiable
is to get
to the benchmark and Stefano
pulled out that
we get everybody to the benchmark
especially in a situation the
people I meet we train
international people from
mostly Europe they
come and their biggest challenge
today or one of
the biggest challenges is the
lack of systematicity
and they have to find their
way to navigate
through the system to get
the learning they
expect to get but is not provided
to them in a
structured way so they have
to to get it somehow
and here is pre-packed very
systematic and
they can go this is the comment
they often make
it's probably one of the
first time they can
go from A to Z through everything
they need
receiving feedback and being able to
observe and confirm their
own progression
and they always say it the
majority depending of
course again from where they
come from where they
learn but the majority of
them complain today
about the access to consistent
structure training
in the order they need in
the moment they
would need and with proficiency
based progression
training that is actually very
clear the pathway
is very clear so they appreciate
the transparency
and the objectivity of that
so over the last few
years you've been working on
a proficiency based
progression approach to device
implantation
what does the data show you
published this in
peer-reviewed high impact
peer-reviewed
journals so what's the data show
so the data our data
showed consistently that
people trained
with a proficiency based progression
approach
outperformed significantly
in an objective
standardised assessment
what people train
still train still with simulation
still with
same content but with the traditional
methodology
and our data showed that there
was a reduction
in procedural errors in lead
implantation of 77%
and there was a reduction in surgical
skills the surgical task related
to the device
implantation of 61% and and
we also showed more
recently that those skills
were transferred to
the real world because the
adherence to optimal
performance in in real procedures
was far superior
in people that were trained
with proficiency
based progression approach
compared to people
that were trained traditionally
so they they
reach the proficiency benchmark
not only they
perform better making fewer
errors they also
have higher chances to demonstrate
the proficiency
benchmark that is required in in a
reasonable amount of time compared
to their peers the
difference we had in the
two trials for lead
implantation we had 93 of
the students in the
PBP group demonstrating
the proficiency
benchmark and none of the other
group demonstrated the
benchmark and in the surgical
skills was 73
compared to 20% so not only
you make fewer errors
you are able also to progress to the
benchmark more efficiently
and more effectively
and the proficiency benchmark
was based
on the class average or
something else
no the proficiency benchmark is
defined up front and
the way we set the proficiency
benchmark is having
experienced practitioners going
through exactly
the same task the same procedure
that you train
participants at and you'd
measure the average
performance the average score
of the experienced
practitioner performing exactly
the same task we
know that's not perfection
we are not asking for
perfection but at least it's
a realistic real
world benchmark that these people
should at least
achieve in simulation before
moving on and
when i say moving on it doesn't
mean we make them
experts right away they
will still need to
probably practice under supervision
but at least
they will start from a demonstrated
fundamental
skill set that their supervisor
can trust because
it's been proven and the part
of the learning
curve that is typically happening
on patients
the early learning curve
which implies risks
higher risk than later is
moved away from the
patient and is now happening
on a simulator
with a systematic approach
so does the PBP approach
take longer than what you
would do normally
I think
Sorry I'm not
talking about the
development of the metrics I'm
just talking about
the the training program because
you normally train
what one two three days so
does the PBP approach
does it take longer?
So at the end it doesn't take
longer because if we compared
on the same
amount of time and we know that
proficiency based
progression should be not
should not be tied
to a fixed amount of time
because individual
different individuals learn
at different speed so
everyone should have the right
amount of time to
get to the benchmark that's
the ultimate
goal but if you have a determined
fixed time you
goal but if you have a determined
fixed time you
you compare the the progression
of the students
in the two groups as we did in
the randomised trial
You realise that actually
not only they
get more likely to the benchmark
but they also do it
faster i say or the majority
of them over the
two days of the training could
reach the end goal
and the others who are still
struggling with
getting there so you could
say that not only
is more effective but is also
more efficient so
that leads into my next question
so you publish
the results in high impact
peer review journals
so how has the data been received
by the clinicians
not just the people that
do the training
but also by your colleagues
in industry
I think it's a it's a big change of
mindset i mean some people
and luckily I had the privilege
to meet
quite some very open-minded
realise that
based on data there is a better way to
do things so you consider changing
you cannot say everyone
agrees with that
some people are surprised
by the magnitude
of the difference question
the simulation
question the methodology
have doubts that
the current system does
not work maybe
maybe will require several years
before a critical
mass of people accept that
it's time for change
but I think and this is true
inside outside the
industry there are early adopters
and people who
are thinking about and people
that will probably
join later however today i
don't know a better
alternative there may be one
alternative one day
if based on data we see that
there is an even
better way I would certainly
support that today
what we see is that a different
approach to
training as the one we propose
with proficiency
based progression leads to
better outcomes and
this is data driven and it's
not just opinion
so some people have understood
very quickly
some people are thinking about
some people at the
moment are still in denial
but the evidence
accumulated will probably
lead everyone to be
inevitable which is change
is needed to get
better patient outcome through
better performances of
operators
So is PBP more expensive?
So the data we
have and this is not data we
could collect on our
research but the data is
probably cross
therapies you require of course
with proficiency based
progression and a higher
investment at the
beginning of course you have
to invest more
than putting together
a few powerpoint
slides obviously however
because of the
more effective and more
efficient approach
once you implement you will
get quickly to the
break even so the research
you did showed that
you don't need so many trainees
to process to get to
that level that pays off
so the initial
investment was paid off after 25
students were trained and
and that means yes upfront
it looks like oh okay
I have to invest more here
but if your training
is not a one event only
but it's a program
that you want to consistently
deliver over time
the return will be most important
of course
better performances of the
people but also
the cost of the investment
will not be as
big as you believe at the beginning
Yeah, I mean
Stefano Puliatta's data honestly
surprised me
they gave me a huge fan of simulation
and obviously
I'm quite fond of proficiency
based progression
but I thought it would take
us we were talking
about a robotic surgical
skills task I
thought it would take us hundreds
till we got to the
break even point but as you
quite rightly pointed
out took 25 I think
shows that it's a much
more efficient and effective
approach to
training and I would have thought
that it was something
that the industry would they
would embrace that idea
I think it will come it's
it's it's something
despite you've been working
on this for probably
30 years I think we we
probably have to
create more awareness
of what you have
achieved over time and the
power of proficiency
based progression I think
the actual cost or
number of trainees will depend
on the therapy
on the resources that are
invested may vary
with the therapy but the result
will not change
so most likely the number
of trainees needed
to reach the break even will
be lower than what
you expect and your investment
in training
will pay off soon I think there
is an important
assumption we are making here
and that's something
that sometimes I find myself
to discuss with people
after a few minutes and
not at the very
beginning I think our assumption
is that we all agree
that we want people we want
trainees to reach a
predefined performance benchmark
so if you agree
whatever the training approaches
so if you agree
with that assumption then you
can start comparing
but sometimes I found myself
discussing with
people about the performance
benchmark and the
assumption was not correct
so I assume that
we all want the people to
reach a quantitative
performance benchmark but in
others people mind
this quantitative performance
benchmark is not
very clearly defined and so
then you can probably
get to some kind of learning
with the power point
although if you haven't defined
where you want
to end up of course you will
be successful because
there is no finish line but
you need to agree
first on what your goal is and
how you measure it
and then you can start comparing
the different
training approaches but this
assumption I learned
needs to be clarified at the
very beginning so
do we all agree that we want
people to reach a
predefined performance benchmark
and then we
can compare the approaches otherwise
no comparison
will survive
I mean one of the problems I find
I've worked with medicine
now for 40 odd years
and one of the problems I mean these
individuals are smart I mean they've
demonstrated that by
getting into college but
the assumption that
everybody else and they make
is because you know
how to do something or you
know about something
you can do it one of the things
i've learned over
the decades is that you can
get somebody really
smart and they really know
how to do something
but they're the correlation
between them being
able to do it well is not
a strong correlation
and I mean like 20 years
ago I would or 30
years ago I would ask okay do
you know about this
procedure yes do you know how
to do it yes have
you done it many times yes
I would have stopped
there today i say okay show
me and frequently what
I find is that they're not really
able to do it to
a performance level so you've
developed and
validated this approach to
proficiency based
progression for the the crt
procedure is what
you're offering just a better
education experience
or are you offering something
more?a
I think
offering a a good educational
experience
I don't say it's easy it's all and I
don't say it's easy to offer
but it's probably
easier than offering a structured
training
with a guaranteed outcome so
what we offer here is
if you first of all it's
evidence-based so
you know what you get and
how far you can go
second you know that this
is not only the
opinion of somebody that is there
today to teach you
but this actually comes from a very
structured consensus and validation
so again as we said
before I don't say bullet
proof but it's
quite rigorous and then once
you enter open-minded
in this training mode then
you see that you
indeed progress and you
indeed receive the
receive the feedback immediate
feedback that you
need in the moment you need
to make sure that you
correct yourself and proceed
to the next so what
we are offering is actually
a very clear pathway
with very transparent mechanisms
and a very clear
performance goal so you know
what you get from the
very beginning you just it's
not that easy but
you just have to follow that
path to get to where
you expect to get so you
will be happy with
probably much less but you will
not be performing
as well if you get something
less and I
I have to say
over the years another
thing I learned I
had the privilege to work
with very talented
open-minded clinicians and
also colleagues that
believe that there is always
a better way to do
things so we have improved
because we never
stopped with what we had
it was a journey
clinicians believe despite
they've been doing
the same job for many many years
with new therapies
of course but these people I
had the privilege to
work with have realised quickly
that there is room
for improvement and we're
talking about big
improvement and they have
dedicated their time
and efforts to make sure that
their fellows
their students and clinicians
in general of the
next generation can have
better access to
structured training and
leading to better
performances and ultimately
of course we expect
better patient outcomes
Okay, can I be bold and
ascii does the training you
offered today do you
think it's better than what
we've offered in the past
Yes I would not be I would
not sleep well
if I would offer today training
that is worse of
what we offered in the past
I think it's better
I think definitely better
I think I can say it
because we now have data and
we didn't have data
before I think having data
makes a difference in
terms of what you compare
with and based on
your own data you can always
do better but you have
to start from somewhere which
is not an opinion
which is not an impression
so we are here today
because we went through the
journey we have
accumulated evidence that
shows that the the
path we took is the right
path
Yeah, I mean I
totally agree with you data shortens
the discussion
because uh before you have
data it's all
qualitative it's your opinion
versus my opinion
but once you have scientific data published
in peer-reviewed journals
that changes the conversation
significantly.
So where does education and
training for the use
of new medical devices stand?
And where does it stand now
and where do you think we
need to go next?
I think I, I mean, the place I work,
I think there is a commitment
to patients
to make sure that every aspect from
the design of a device
to the training, to the collaboration
with clinician
leads to better outcomes.
So I think we have made over
the years effort
to provide training that generally
produces better outcomes.
I think we have improved and showed
that it is possible to do moving
from a good training
where you have an impression
that is a good training
where some experts say is
a good training.
Now we have moved to proving
with data
that the training is effective.
I think it is important
in all domains
that the quality of the training
should match
the quality of the technology
we provide
and the quality of the technology
investment
in the med tech is high.
And I believe that training should
match that quality
to be able to provide healthcare
with a better package
to optimize what the patient
could get.
And yes, we are not done yet.
I think we are on the
right trajectory
and I see things moving.
Maybe not as fast as people
would expect,
but actually faster than what
the general public today
is going to consider good enough.
Yeah, I mean, the number of
devices I've seen fail
in the marketplace and the devices,
I mean, from what I could see
were perfectly good,
but the application of the devices,
the use, the clinicians using them,
some of them were suboptimal
and some devices failed in
the marketplace,
not because of the device,
but because of the clinicians
actually using it.
And that's not something that's
been recognised,
well, publicly recognised
within medicine,
but the data basically now shows
across all specialties
that we have a group of clinicians
that perform sub-optimally in comparison
to their colleagues.
And I think if I may add
something, Tony,
I think the great majority
of trainees
I have a chance to meet are
really looking forward
to learning in a more effective,
more efficient way.
So they are genuinely open to ways
that would lead them to
more confidence
and better performance.
So I think they feel the need.
I think the system, it's the system,
that education system that
needs to improve
to make sure that those motivated
individuals
reach the level they are
set to reach,
and they're probably capable to reach
within a reasonable timeframe
and with very clear pathway
to a precise goal.
So the individuals are motivated,
the system needs to make
an extra effort
to provide those motivated
individuals
with the tools that are required
to make them successful.
Yeah, no, I totally agree with it.
I mean, our goal should be to
help those trainees
to get to that performance level,
but the performance level has to
be quantitatively defined.
It shouldn't be an attritional
system.
I mean, we basically, the whole point
of deliberate practice
is you help them get to
the benchmark,
but reaching the benchmark's
non-negotiable,
because what we believe is, and what
the data, I think, shows
is that your performance level
will impact on patient outcomes.
So you've been working on this now
more or less for 20 years.
So what would you say are the
major take-home messages
from your work, your experience,
your research,
your publications over the
last 20 years?
Yes, to summarise, so I think
you are an example
of the first recommendation
I want to give,
so never give up.
There is a path to make
training better,
make healthcare more successful,
and it's not only related to the
successful treatments
that are introduced or new therapies,
but it's also related to how
we train people.
So attention to the quality of
training is essential.
Not every training is
a good training,
and you should not give for granted
that after going through some
years of training,
you reach a certain level if you
haven't proven that.
The summary is also showing
me that, yes,
if you are motivated and interested
in doing a better job,
there is certainly a way
that is proposed
that could get you better results.
And I would invite people that are
still skeptical to try,
and once they will see
their own data,
they will probably find that
there is no other way
from the traditional approach
that can match,
at least from the data we have
shown over the years,
there is no traditional approach
that matches the outcomes
that we reach
with a very rigorous systematic
metric based training.
I think we need to have
people that are,
genuinely interested in moving
the learning curve
away from patient.
That's what is my number
one concern today.
And I think the operating room
at a certain point
will certainly become, or becomes,
an important environment where
you can observe
and you can be supervised
and perform,
but this should not be the
first thing you do.
I believe that there is a lot
that can be moved
into a safer environment for learning
and reducing the number of
errors you can make
by trying and retrying and
receiving feedback
until you demonstrate that
you have reached
a safer level with your skills.
And at that point, of course,
exposure to patient is absolutely
necessary.
But I think my recommendation
be if I could change
one thing today is that everybody
starts first
in a protected environment and
then moves to the OR,
performing cases in vivo
under supervision
and not the other way around.
That's a very good summary, Jorio.
You've been doing this
for a long time
and I've been with you for
a good part of it.
And thank you for sharing
your wisdom today.
It's always a pleasure
to talk to you.
Thank you.
It's been a pleasure working with you
and showing that there are better
ways of doing training
and of course, the journey, as
I said at the beginning,
doesn't stop here.
We're trying to make it always
better and better
and having more people involved
in this type of approach.
Thanks, Tony, for the discussion.
Thank you.