Why, Why, Why?

Today, we discuss aspects of root cause analysis and developing a mindset which responds positively to engineering failure.

Creators and Guests

Producer

Michael Crocco

Engineer and Educator. AI in Engineering innovator.

What is Ctrl+Alt+Engineer?

A podcast for mechanical engineers beginning their studies in Design.

(Music)

Yeah, so I'm a lecturer at Monash
University here in Australia.

And what we're gonna talk to you about
today is root cause analysis

or root cause determination
and where that sort

of fits within the engineering
design process.

I would say it's integral to the

engineering design process, wouldn't you?

100%.

I think a lot of people ignore it.

I think a lot of people start to think,
well, is that really part

of it or is that the sort
of the next phase?

But I disagree.

I think that this is a really
integral part of design.

Yeah, I mean, I think when you say
everybody, you're referring to

probably students, I guess, generally.

Is that right?

Students and young engineers.

Yeah, fair enough.

I mean, I guess in my experience,

you are entirely expecting
things to fail.

And big part of the engineering, at
least design process, was

finding those failures, learning from

them and adapting the design
as part of it.

Yeah, failure happens.

This is part of how we grow and how
we learn and where we work out

whether things are right and
fit for purpose or not.

Sometimes they're just not the right
solution and so therefore

they're scrapped, but other times

they're close and they just don't work.

So you have to work out why
that isn't working.

So how would we do that in industry?

Okay, let's talk about an
industry example.

I'm gonna talk about a process failure
initially as opposed to an

actual, let's call it an engineering
design failure, right?

It's actually where there was an
engineering design in place and

the process didn't quite follow, but

it was hard to determine that initially.

Yeah.

We had the wheels come off of a car once.

The literal wheels?

Like, excuse the pun, it's not.

A wheel came off a car.

That's obviously reported as a big deal,
something, this is a test

vehicle that was out running durability
and the wheels came off.

So of course, I had to go out and look
at the, it was on a gravel

road, found a bunch of metal filings
where parts of the bolt had

come apart and parts of the wheel
and that kind of thing.

And we found where the vehicle was and
then we followed back up and

found all the little bits and

collected them up and that kind of thing.

It was a long process of investigation

that involved going to the
individual body shop

that had done a bit of work on
the car at one point.

And we specifically looked at the
hoist that that car was on

and found that the torque wrench, the
pneumatic torque wrench that

was on the left-hand side there for

easier access was at a calibration.

And what had happened is it had
overtightened the bolt,

caused premature failure essentially
or overstressed the bolt.

And then that's what led to failure.

So there was a process thing that
was really hard to pick up.

But I guess I mean to bring this up
because this is an important

part of engineering is the build, right?

It's not just what you set down on paper.

Absolutely.

Is what you set down on paper
even possible?

Is one question to ask, is it feasible?

Is it realistic?

And then is it actually executed?

So I guess what am I talking about?

Talking about it's not just
how you design thing.

Sorry, I should rephrase that.

Design is not just what you
imagine should be

done but it involves what is
actually executed.

Yeah, design isn't to sort
of paraphrase you.

It isn't what you just put down on paper.

It's the build, it's the maintenance,
it's the work.

It's the process of how do you,
the manuals that come with

something, it's all of those
things encompassed.

All of those things need to be

thought about even down to end of life.

This is an entire process.

And unfortunately, especially at
universities, we can only teach

you a very, very small part of that

given the amount of time that we have.

So again, this is another reason why
we've done this is just to

sort of impart knowledge and start to
give people an idea of what

it's like moving forwards
from that because we

do have students for such a
short snippet of time.

So what did actually happen?

So the torque wrench was over
talking the bolt.

Yeah, it was up to like, you know,
spec was 120 newton meters.

It was probably hitting around 380 or
400 or something like that.

So I mean, obviously that's well over,
proof stress for the bolt.

And then that, we looked
at the rear bolts.

So this is the front left, we looked
at the rear left and clearly

there was like visible stress on the

bolts whereas the right hand
side was fine.

So we were reasonably confident
at that point.

But that's what

that's that that torque wrench being out

of spec was in fact the
root cause, right?

So see that, you know, we're going
through, we're asking, okay, we

determined the type of failure,
we looked at it.

What could have caused this?

You know, we looked at where could
it have been happened?

We looked at what tools were

implemented and you keep on
asking questions.

And how they were implemented, yeah.

You keep on asking questions until
you get to some level of

confidence and then you
confirm it, right?

So we said, well, if this was actually
an issue on the left hand

side of the vehicle, let's have a

look at that other one that didn't fail.

Oh, and by the way, in fact, that's okay.

What happened to the left
side of the vehicle?

Why might that be different?

So I remember when I was all the way
back in primary school, we

were told that when we were doing our
English work to think about

what, who, where, when, how, why.

And these are all things we still
implement within engineering.

We have to be curious.

And that's part of the culture of
engineers is that innate

curiosity to say, why did that happen?

How did that happen?

You know, it's not just about
a wheel fell off.

It's like go all the way back.

Cause that would have been
a five stud pattern.

That was six stud.

Six stud pattern.

So, you know, so a bunch of
over tighten bolts.

How many had actually failed?

Well, all of them ultimately.

Ultimately, but had it been
one at a time?

Probably initiated at one,
the failure pattern

on the other five were was different
from the one.

So one gave away and then the next one

gives away in a slightly different way.

And then you get to the point where
you've got four left and then

basically happens very like
precipitously, right?

Absolutely.

So it's almost all four at one

stage and they're in the end part of it.

And that can be told by the kind

of failure on the bolts themselves.

So the bits that you could
find of those bolts

before you then start to go
back through that.

So one of the other things that we
do as engineers is that when

something like that happens, we've gone

through our what, when, how, why, where,

those sorts of things.

But then we're really honing on the why.

There's a process called five whys.

Or,

why why why?

Why why why.

Which is my first introduction when
I graduated was I worked at a

company that they want to say why why
why instead of five whys.

There was a guideline though that in that

company you should ask why
at least six times.

So I don't know why that we
just said why why why.

Anyway, it was just easier to do.

Cause that's only three whys,
stop five whys.

You can ask a couple of whys.

But the point is, like you
say, the why is the

important part and you have
to keep asking, right?

And it's a really interesting way
to break down your problem

to understand what to investigate, right?

So, okay, the bolts fell off, why?

Well, they broke.

Okay, why did they break, right?

Yeah.

Well, because they were not strong
enough for the application.

Well, that doesn't get you
very far because we

know that they're stronger
on the other side.

So no, why did they fail on this side?

And then you keep asking
questions until you

get to and you might have some
forking too, right?

And that why tree, you might, it

might be, well, this and this happened.

And then you have to hunt down a
couple of different things.

Well, one of those whys that you said
is, so why did the bolt fail?

Part of that isn't just why did the

bolt fail but how did that bolt fail?

Because this thing gives
you information as

to where the why might be
coming from too.

One of the things that we did when we're
in aerospace was because

we were doing a lot of vibration and
vibration could be a little

bit tricky, a little bit hard
to sort of deal

with, was we had to look
at a YYY situation.

Still struggle with that one.

We had a component for a receiver

that was a counter lever situation.

And we thought that was going to be

our weakest, our biggest vibration hitch.

The problem was that when it did

fail, everything in the inside failed.

So trying to find out the five whys
and the why of where this

might've started was really,
really difficult.

So instead of just doing more physical
testing, we went back to,

even back in the day, ANSYS

and tried to simulate that to see
where we could actually,

pinpoint what that failure might've been.

So there's a whole range of things
that you can do.

It's not just sitting there thinking.

Sometimes it's giving little tests.

Sometimes it's doing smaller parts of
those bigger systems so you

can ascertain what might or might
not have gone wrong there.

Yeah, right.

I mean, the point being, there's
a lot of questions.

Not all of them are answerable.

You're trying to answer them
as best as possible.

Absolutely.

In your experience of watching
young engineers,

what do you think the,

this is perhaps a loaded question,
but what do you think is a

propensity to avoid this approach?

I think most young engineers do avoid it.

I think that they get concerned that
they're going to do something

wrong or that they have done
something wrong.

So I see a lot of students
or young engineers

when they do have something
where it's not right.

It's, they don't think about, is this
a causation of, this is the

wrong thing for that particular
situation.

And so therefore that should
be scrapped as an idea.

It just doesn't work in that situation.

Or is it that it's a good idea that's

not quite going right at the moment?

So therefore I have to do a bit more
analysis and then really dig

into what the problem is and
where it's going wrong.

I find they either persist with something
that is going terribly

wrong and then go, it'll be fine on
the day and just hope and pray

and cross fingers and toes and
all that sort of stuff.

Or they scrap the whole lot and
start somewhere else.

Do you think that's out of
a fear of failure

or is it something else in
your experience?

I think it kind of fits a little bit
with fear but I also think

part of it is inexperience that they

don't understand how to sort of do this.

So one of the other things that
we do as engineers,

just in and around the people that I
know who are older engineers

is that there's this thing called
the 80-20 rule, Yuri.

It takes you 20% of the time to
get 80% of your solution.

It takes you 80% of your time to
get the last 20% done.

And it's much easier to get an 80%

solution than it is to get 100% solution.

So sometimes people stop there but
people also don't know how to

sort of dig into that because 20% of
the problem will give you, oh

sorry, 20% of a situation will give
you 80% of your problems too.

Yeah, look, I would say that I've seen
a lot of propensity and then

probably exhibit it myself to feel

like you've gotten to the final why

or to try to explain things
away, for example.

So two mechanisms there.

One is you get to the point you've asked

only four of your five whys or only
four of your six and you think

that you're at the end of it, that

you're not ready yet to take action.

And so you might prematurely stop your
investigation or you might

abandon the data at that point
or something like that.

And then you're actually shortchanged

in the attempt to reconfigure or redesign

the way sort of answer to one of those
questions where there can be

embarrassment associated with that or
there can be just a desire to

move on, a pressure to move on with
design, a pressure to get the

drawings out, to hit production,
all these kinds of things.

And those pressures are one
of the things that I

think engineers have to balance
the most, right?

Despite that pressure, you need
to get an outcome.

Despite all of that perceived or real
pressure to perform, you need

to go through this process, give it the
time it's due and come to a

real conclusion that you can use because
there's nothing worse than

launching an investigation, writing a

report and it be completely useless.

Yeah, for example, if you had have
looked at the, your wheel

falling off and determining, well, maybe
it was just a faulty bolt,

so maybe it's an outlier, then that
actually puts your staff at

risk when they're testing a vehicle,
because in actual fact, there

was a much bigger causation
to that failure.

Yeah, and think of, that was a
multi-week investigation.

We finally did get to the bottom of
it and we were able to correct

an issue that would have been causing

this same issue on other vehicles, right?

So it wasn't actually our design.

It was a production issue out there
that was likely causing those

problems with other vehicles and possibly

other wheels could have
fallen off, right?

And maybe they did and we just weren't
aware of it because of the

way that the contracts were there.

But yeah, you have to keep going and
it is extremely difficult to,

whether you're under assignment submission
pressure or production

pressure or just the perception of
failure in your business or

client pressures or whatever they are.

Yeah, we want an answer now.

Yeah, right.

And an answer now might actually
be the wrong answer,

which means that you then end up in
much, much greater problem.

False sense of security, right?

Yeah.

And these things can easily get washed
under the table and you

think, well, because the drawing was

issued then everything must be all right.

If you're skipping over things and
then you're essentially

violating the primary directive
of an engineer, right?

Yeah.

The other thing too is that your

engineering design doesn't
stop at the drawings.

It doesn't stop when it goes
into production.

It also encompasses all the things that
could go wrong with that.

So how do we apply this in the classroom?

So applying this in the classroom
can look, it can be tricky

because the time pressures and not just
for students, they're also

for us who are in academia as well,
because we have to give you the

best learning outcomes we possibly

can in a very short space of time.

What I would suggest is that stop and
breathe for half a second.

I see a lot of students when something
goes wrong, it's literally

like scrap it, move to the next thing.

Take a little bit of time, even if it's
only 10 minutes and start

to ask yourself at least two why's.

Four or six would be better.

Start with two.

I do understand the kinds of

pressures that you're under, I promise.

But yeah, start to think about why.

This is not always, and maybe it might
be the case that you have to

pivot, maybe you have to scrap that
idea, but maybe it's a pivot

rather than a scrap, but you don't
know unless you know.

And the other thing too is that you

have to learn from those failures.

That's where all of your really deep

understanding and learning comes from.

Right.

We've talked before about, I think
we've talked before about the

companies I worked in at least where
we had to prove every single

design that was done against all the
other historical failures.

Yes.

And this is actually an area where
this is gonna get easier, I

guess, in the modern era with AI
enablement of engineering.

I think that's actually one of the
areas where it's gonna be the

biggest impact on actually getting better
results out of engineers

is putting historical knowledge into
AI accessible databases and

then being able to inquire about
the applicability of those

failures to new designs and
things like that.

Previously, it was very difficult because
you had to be either of

deep knowledge on the subject by studying
it, or you had to weed

through a bunch of documentation to
try to find what might apply

and those databases are growing
all the time.

And so AI is gonna enable that, I think,
not to get off topic, but

I do think it's an important area to

identify in the future of engineering.

Well, just the fact that all of those
databases are searchable

together in one place, using an AI

means that things can happen faster.

You're less likely to overlook

something that you might've in the past.

So anyway, what we're getting at
though, we're talking about

historical knowledge being one of the

really important aspects of engineering.

I mean, this is a profession that
did start as apprenticeship.

And so the whole aim was to learn from

experience and then you could do things.

So work in the machine shop, work with
the other engineers, learn

from all the mistakes that get made
or from the knowledge and

wisdom that they have and then adapt

your own wisdom, that was how it worked.

You still need to achieve that, right?

We still, in university, we teach you
the fundamentals, but you

don't have the knowledge
of the wisdom yet.

And so root cause investigation is an

important part of forming that wisdom.

Yeah, and we understand that
you don't have that.

And we lean into that rather than
leaning out from that.

Like we want you to have that
sort of experience.

I had an argument with my husband one
night, this is pre-COVID,

about a particular solution that
a team of my students had.

And he was like, that is the worst

solution I've ever heard,
that was terrible.

That doesn't sound like him.

This isn't going to work though, I

wasn't going to mention his name.

Thank you so much for that, apologies.

But yeah, he's like, this is terrible,
how dare you do that?

And he's built factories
all over the world.

And I said to him, when you build
that last factory, what went

right in a state of frustration, and
he said, well, how the hell

we're going to know what went right?

I said, well, what went wrong?

And he went, oh, because these are
the things that you remember.

It's very difficult to remember what

goes right when you do these things.

It's very easy to sit there
and pick at this,

pick at a scab per se of the
things that went wrong.

And this is why I'd like you to, and
encourage you to lean into

that if you're a young engineer, because
not out of a state of fear

or guilt or anything else, but
just introspection.

So how might that have worked better and

start to think about those
sorts of things?

Because I think that's where
the real learning is.

Yeah, that shift that there's, if there's
a need for a shift to

learning from failure or learning

from mistakes and all that kind of thing.

I mean, we've touched on it briefly
in the past, but that was

something that personally I struggled
with initially.

I know I did, because again, transitioning
from education where it

wasn't necessarily a part of it when I

studied at least some number
of decades ago.

I think it's more now, which is good,

but adapting that mindset of embracing
those sorts of things,

failure and mistakes and learning from

mistakes, et cetera, that was really,
really important.

Big part of my growth.

Super smart people who come
into engineering.

A lot of really smart people

choose engineering as their profession.

And it's shown by the kinds
of grades they get

in high school and the sorts
of study they pursue.

And so it can be sometimes a little
bit more challenging to come

into this space where, yeah, you're

going to have things that you get wrong.

And it's okay, but there's
no definitive answer.

And that's the other thing I think
is it's not, it can be really

hard to sort of put your finger on
when you don't have that

experience, whether this is going to
be right or wrong, hence why

doing a root cause analysis and really

understanding where that's
going is important.

Is it something that you
assess currently or

would think about assessing
in the classroom?

Putting you on the spot
here a little bit.

Yeah, you are, very much so.

I think that's a little bit challenging
to sort of do, but it's

definitely something that we talk about.

My students have recently had to
do a scissor lift design,

which I knew was going to be a failure.

Scale or full size?

No, small scale.

Had to fit within a 50 B, 50 B 100 box.

And it had to expand to 400 mil.

Okay.

Knowing that, you know, when you,

this is something that our students had

seen before when they were in first year.

So coming into second year, this was
experience that they would

have, that's right, you designed the, you

designed what the precursor to this was.

Oh, I did.

You did.

You're not supposed to reveal that.

No, I know, right?

But no, so Michael and I've had a fair
bit of experience, but what

students do, would do in our first year
course is that they would

design a bridge that would
have to span a gap.

And quite often they come up with a
truss that's a movable truss.

If you turn that 90 degrees, then
you get a scissor lift.

Surprise.

So back to your, the scissor lift.

So they had to design a scissor lift.

I haven't seen this project, so
please don't mind me.

They not only had to design it,
they've had to build it.

Now, I don't think there was a huge
amount of, your traditional on

paper design, but it was a lot
of thought process.

How do you put your cross
members together?

How do you get them to be more rigid?

That sort of thing.

But of course, once that scissor
lift has been expanded,

then of course, because it's a column
and it's no longer a beam,

you end up with the top wonk
that would happen.

And it was great.

Some of them were so fantastic,
you'd get them

almost leaning over onto the
ground, onto itself.

And then if you're depending on the
way, the way the wind blew in

the room, they'd go the opposite
direction or they'd give a full

sine wave or, it was really fantastic.

And a lot of students were really
worried about that.

So we sort of stepped them through
that, we wanted them to think

about what they already had within
their bank of knowledge.

So this is why we started here was that
we started very much with

the knowledge that the students had.

So we could sort of expand on that
and get them to think about,

okay, this is somewhere you might jump

because it is in your field of knowledge.

Sounds like an exercise in fits
and tolerances to me.

That too.

Isn't it though?

Yeah.

Yeah, right.

Absolutely.

But there were very few that were

successful with their fits
and tolerances.

But of course, it's also understanding

that you're a fixed support
free ended column,

your effective length is, like it

was way beyond its effective length.

Even if you had good fit and tolerance.

But this is what we wanted
to sort of think

about was, this is in my sphere
of knowledge.

Now that I've sort of dealt with
that, is it best fit?

It might be, it might not be, but it

also allows you to look at other things.

And I think that's the other thing
that looking at a root cause

analysis does, is it gets you to think
about what other things

might be in your sphere and other

things that you might need
to think about.

So how much failure did you have in that
practice in the classroom?

I think I saw two that were successful.

Okay, that's great.

Yeah, that they could actually lift and

sort of hold themselves
relatively simply.

So despite having done a similar project
in the previous year, the

students were able to, this project

was able to extend them, forgive the pun.

Is that right?

I think so.

Despite having done a similar thing,

they were able to have some growth there.

Because we got them to build it, we
could show them that when it

was in a horizontal plane,
it was fairly stable.

But as soon as you stood it and it was
in the vertical plane, that

you got all of this uncertainty as
far as movement was concerned

because the play in the support joints,

all of those sorts of things, but

we could also have a conversation.

So I wasn't getting them to do the
calculations on a column, but I

was getting them to start to think
about, well, what's the

difference between a beam and a column?

So you're building some intuition and
things like that character.

Yeah, hopefully.

Yeah, interesting.

And this is kind of where we wanted
to stay was, if we could lean

into the bank of knowledge that we knew
students had, and this is

what we're trying to pick
up with the unit

that I'm teaching is what knowledge
do they have?

Then we can kind of get them to think
about where that goes.

That's great.

So what is next?

What is next?

So hold on.

Did we cover off this whole
five wise thing?

We explained it well enough.

We talked about five wise enough.

Yeah.

You mentioned Pareto analysis, the 80-20
sort of rule, and I think

that's a guideline for 20%
of your effort.

And it's funny, I've often heard Pareto
been rephrased now that the

accepted norm is actually 10-90 where 10%

of an effort will get you
90% of a solution.

And then it requires an additional

90% of effort to get the remaining 10%.

And I think that's actually been

statistically proven to some degree,
if I'm not wrong.

The world moves on at a matter of knots.

You and I have discussed fishbone
diagrams in the past.

Do you wanna bother with that?

We have, like, well, as far as I'm
concerned, a fishbone diagram is

just sort of a way you collect
your five wise.

Yeah, right.

And it allows you to sort of walk
through the process, per se.

I'm a fairly,

I like sitting down thinking about
walking through a process.

I think that's really important, and
especially when you're at the

beginning of your career, it's a really
good idea to go, okay, let

everybody just stop, think about,
okay, where are we at?

What was the process?

How did things happen?

And walk yourself through things.

And I think that's where a fishbone
diagram can help.

Well, it helps convey what you've
learned to other people.

It also helps you probably
assign action and

identify, we talked about
those forks before.

If you don't know what a fishbone diagram

obviously is, listeners, you
should look into it.

I'm not gonna replicate it.

It's really just a branching tree
of cause and effect,

essentially.

What else do we have to touch on in
terms of root cause analysis?

I think that was kind of the most things

that I really wanted to sort
of get across.

I think it's just do it.

It's not gonna hurt.

Yeah, make sure you do it.

Make sure you do it.

Maybe that's the thrust of it.

Maybe that's the thrust of it is,
do a root cause analysis.

Every chance you get.

Yeah, about everything.

It's be an over-analysts.

Stop and think about the things
that you're doing.

It's okay to sort of pick them apart
and have a look at them.

Even if they're really successful,
what worked?

Yeah, and break through that stigma now.

Like if you're a student or a young
engineer, maybe start now to

make that a regular part
of your practice.

Because it's going to become ever
increasingly important.

And again, we talked about the

pressures, but don't worry
about the pressures.

You can take notes afterwards.

So I still, every time I do
a project, and there

are people who think I'm crazy
and that's fine.

You can think what you like.

But I sit down and I make notes

about what worked, what didn't work.

And then I kind of pick out what
was the causation of that.

So even now in my classrooms,
so my academic

life, I'm still thinking
about those things.

So it's very engineering, apparently,
I get told this a lot.

But I sit down and I have a little
bit of analysis not to beat

myself up, but so that I can make

whatever I'm doing the next time better.

It's funny that you mentioned that.

I'm going to talk about AI again.

I've transitioned from doing it in

notebooks and things like that too.

I now do this in, believe it or not,
markdown format, which is

easily machine, like AI readable at
least, because I know that I'm

going to be working in that paradigm
in the future.

And so, yes, I do still write
all those things down.

I make notes as I go along and I'm now
using this different format,

which is somehow the bridge between
humans and AI for whatever

reason, at least at this point in time.

And this will probably date here in a
little while, but it seems to

be the norm at least for the
coming year or so.

So yeah, so I absolutely do.

And I guess, yeah, it becomes
part of learn practice.

I don't know who doesn't, but I might
be wrong about that, right?

Like that might be an assumption as an

engineer that this is what
everybody does.

And perhaps that's false.

I was gonna say that the engineers
that I know that are in my

circle, and there are quite a few
of them, mainly all do this.

Yeah, right.

I think it's a really good practice.

Like give it a go.

Like it doesn't have to be a book.

It doesn't have to be something that's
publishable for anyone else.

It's about notes for yourself.

I think we talked last time we got
together about notebooks and how

I went through some old notebooks.

Maybe that was a couple of episodes ago.

It is, you know, there was
knowledge that I'd

kind of not abandoned, but
kind of let slip.

And it was interesting to revisit that
and regain that knowledge.

So yeah, totally recommend that sort
of approach for everybody.

On that note, thanks so much for
listening, everyone.

You can do that thing where you
say where we are today.

Because we are in a different spot again.

Everybody will notice that there is
no YouTube video associated

with this episode for the first time.

We were just under a little
bit of time pressure.

And we're at...

We are currently in my review
room at my house.

We've kicked the family out
for the afternoon.

So thanks so much for joining
us in the Derrionrangers

(Music)