[MUSIC]
Hello and welcome to another episode of
Undercooled and
Materials Education Podcast.
Today we have a special guest.
We have my friend Jeff Fergus from Auburn
University who knows more about ABET than
anyone else in the
entire world that I know.
And so we're going to be
talking all about ABET today.
But first, Jeff, why don't you tell us a
little bit about yourself?
Sure.
So glad to be here.
So I'm originally from Illinois.
Went to University of Illinois and the
University of Pennsylvania, which is
where I first met Steve.
Did a post-doc at Notre Dame and had been
at Auburn ever since.
I came here in 92, so
I've been here for 31 years.
My research area is generally kind of
high temperature chemistry,
electrochemistry, batteries,
fuel cells, things like that.
But been in the dean's office about 10
years in two different
associate dean roles.
First was program
assessment and graduate studies.
And now I moved over to undergraduate
studies and program assessment.
So that's a little bit about me.
Tim?
Awesome.
Well, with that background, of course,
the theme of today's show is ABET.
So can you first actually just tell us a
little bit about what is ABET?
There may be people in
the audience who don't know.
And then how are you
personally involved in the program?
Sure.
So ABET's a confederation
of professional societies.
There's 35 professional societies that
have basically come together to establish
criteria to prepare programs,
to evaluate programs to prepare students
in areas, engineering, computing,
engineering technology,
applied and natural sciences.
And so the purpose is to set these
criteria that programs can develop
programs so that they can prepare their
students to be prepared, be ready to
mentor the profession.
I've been involved with ABET as a
volunteer for about 20 years or so.
As a program evaluator, I
served on the commission.
There are four different commissions in
ABET in the different technical areas.
So engineering is the largest of those.
Engineering technology is another,
computing and applied
and natural sciences.
So I served on that commission.
I moved into the executive committee,
also then into the officer change.
So I was the chair of
the commission in 2019-20.
And since I've also worked on training.
So I'm a facilitator for the training
programs used to
train program evaluators.
And I'm now a lead facilitator for that.
Been doing that for I
think since about 2011 or so.
And currently I'm the chair of the
training committee of
the Accreditation Council.
So the Accreditation Council is a
committee that represents the four
different commissions.
Some of the things that ABET does are
specific to the different commissions and
some are common across
the four commissions.
And when things are common, then this
body is the one that kind of coordinates
so that those can be done.
And things can be
accomplished for all four commissions.
And so that council
has a training committee.
And that training committee is the body
that prepares the training,
organizes the training for,
mostly for program evaluators, but also
some of the team chair training that's
common among the commissions.
That's great.
And so ABET's been around, I believe,
since 1930 something.
Is that right?
And under different names, reinvented
themselves multiple times.
Two times since I've been around since
around 1998 when they started doing the
outcomes assessment based approach.
And then I believe it was another, I
don't remember how many years when they
redid the student outcomes pretty
dramatically going from A
through K to one through seven.
So ABET is always asking programs to do
continuous improvement.
And I think ABET's been practicing
continuous
improvement for quite some time.
That said, why should, you know, this is
a podcast about materials education.
So our prime audience are the materials
programs in the United
States and around the globe.
I believe there's something like 125
materials programs, something plus or
minus five programs, which is a lot.
Why should materials
programs care about ABET?
Why are we involved with ABET?
So there's a first kind of a practical
reason that many entities look for ABET
accreditation when
they're looking for engineers.
In particular, if licensing is involved,
most of the jurisdictions for licensing
of engineers require a degree from an
ABET accredited program.
Now, many materials engineers don't need
that licensing, but there are some areas
where it is important.
For example, if you're going to go into
consulting and say failure analysis, it's
very important to have that professional
engineering license.
So that's kind of a practical reason.
And many companies or other entities,
when they're hiring, they'll put that as
a requirement for the degree or
preference, if not a requirement.
So those are kind of practical reasons,
but it really the process involved,
although it sometimes may seem tedious
and parts of it may not seem
clear why they're important,
that process of evaluating your program,
looking at it to see what you can do
better, is going to help you improve that
program and better prepare your students.
So I think when you really understand
what's required from ABET, it's not as
hard as sometimes people might think when
you really understand.
And if you do it in the right way, it can
actually be a benefit and really help you
improve your program.
So I think it's really about, and that's
the reason I'm involved, if it were just
about checking a box for a licensure, I'm
not sure I would have spent
all the hours I have on it.
But when it's really about trying to
improve the preparation we're providing
for our students, that's really what I
think is important about it.
I completely agree. It's so valuable to
be able to say not just, hey, let's keep
doing what we've always been doing, but
to say, what can we do better so that our
students are more successful and more
prepared to make a
difference in the world?
Absolutely.
So on the receiving side of things, as an
educator at an ABET accredited
institution, I'm often interacting with
these criteria, the
accreditation criteria.
And so this makes me curious about where
do the criteria come from, what are they
specifically and what are the criteria
that ABET is using to
accreditate programs?
So maybe I'll start
with where they come from.
As I mentioned, ABET is a confederation
of professional societies.
And so it really is us.
It's all the different professional
societies have
representatives on the commission.
It's the commission that
decides what those criteria are.
It's the commission that creates the
teams that go out and evaluate programs.
And it's a commission that decides
whether or not they're compliant.
So it is people from our different
professional societies that determine
what those criteria should be.
And that's important to keep in mind that
when we complain about ABET requires this
or that and may not agree
with it, ABET's not this.
There is an office in Baltimore that has
30 or 40 staff members, but they're
really there just to manage the process.
The decisions are made by representatives
from all our professional societies.
Now, those different professional
societies have different
interests and priorities.
And so there is some
negotiation that has to go into that.
It has to apply to all these different
types of institutions, types of programs.
And so sometimes it can be the language
can be a little bit vague because it's
got to fit many different instances,
which which takes some judgment and some
understanding of them.
So that's kind of where it comes from.
The criteria in terms of what they are,
there are eight what we
call general criteria.
And those are the criteria
that apply to all programs.
So the way I think of them at the top,
you have what are called program
educational objectives.
And these are what the program expects
its graduates to be attaining
a few years after graduation.
So it's kind of your North Star.
What are you trying to do?
And the program needs to determine those
with involving its constituencies,
whoever they determine those to be, which
is typically employers and
alumni, people like that.
And I think that's one of the real
benefits of the impacts of the change
that Steve mentioned
that occurred around 2000.
And that is that's forced programs to
really engage more with their alumni and
their employers to get feedback on what
they should be doing.
So then below that,
it's how do you get there?
And the first step is what
are called the student outcomes.
These are skills and knowledges that that
you expect your graduates to demonstrate
before they graduate so that they can
then attain those
objectives that you've identified.
Those, again, had to be determined and
agreed to by all
these different societies.
And one of the improvements that Steve
was referring to is actually has has
roots in material in
our materials community.
The the outcomes that were set back in
probably late 1990s had
been around for a long time.
And Beth Judson, who is from the American
Ceramic Society, was then on the
executive committee and she said, well,
we've been we ask our programs to always
speak improving, so we
should be doing that, too.
So let's re look at these outcomes and
that she kind of started that process
back in, I think, was about 2009, 2010.
Sometimes in that time frame.
And it finally took 10 years or so before
they they were changed.
And it took a long time to come to
agreement as to what they should be.
I think they were an improvement.
So there is that trying to make it
better, try to adapt to these changes.
So that's the outcomes.
And then to support all that are
basically the things that the institution
or the program needs to do.
Students, what are your processes for
advising and making sure that students
meet all the graduation requirements?
What is the curriculum?
What is the content of
the courses they take?
What's their experiences that they need
to go through before
they can for they graduate?
Are your faculty qualified?
Do you have enough faculty?
Are the facilities adequate to support
those student outcomes?
Do you have the the the institutional
support, financial and administrative to
support the program?
And then on top of all this, there's that
requirement for a continuous improvement.
So you have to have a process by which
you're determining are your students
meeting the outcomes that that have been
specified and looking at other other data
to see how how are you doing
and what can you do to improve?
And you need a document that you're
continuing that process.
Most of the most of what the evaluators
are looking for are related to processes.
And processes are important.
Sometimes it might say, well, it's kind
of a bureaucratic thing, but processes
are what keep things going.
If you don't have a process established,
then if people change, you might not keep
doing what you're doing.
But if there's a process that helps make
sure that people are doing things in a
certain way, it's being consistent and
not letting things
fall through the crack.
And that's really mostly what what the
team is looking for.
Do you have these processes? How are they
working? Are they documented?
Yeah, that's well said. I think that's
exactly captures what a bet tries to do.
And it's always distressing to me how so
many faculty and even many administrators
misinterpret what a bet trying to do.
I can't tell you how many meetings I've
been in where someone says, oh, we can't
teach that in our class
because a bet won't let us.
And it's like, no, a bet doesn't say
anything about what we actually teach.
There's no curricular mandates by a bet
other than it be a engineering course or
a math science course.
There's 30 credits of math and science,
45 credits of engineering. That's it.
And there are clear definitions in the
criteria that explain the difference
between those two words.
And that's it. They don't dictate the
content, the pedagogy, the kinds of
courses that's totally up to a program as
long as the student
outcomes are being assessed.
And they don't even have to be met.
Right. The criteria doesn't say the
students have to meet the outcomes.
It only says that you have to measure the
extent to which the graduates have
achieved those outcomes.
And everything, the processes, everything
is up to the programs. A bet does not
dictate any of that.
And I think it's all encapsulated really
nicely with something.
Was it Chet Van Tyne who started saying
the phrase, improve your program for
yourselves first and
worry about a bet later.
If you're a good program, meeting the
criteria for a bet should be trivial.
You just have to document things and have
a process where you actually look at it.
And the details are already being done
because you care about your students.
That's been my experience as well,
tangentially working with some of these
things is looking at the criteria,
looking at the
outcomes and really saying,
this is what I should be doing all along
anyway, right? You just want to make sure
that I'm actually doing a good job.
It doesn't really seem so onerous if, as
you said, Steve, if you're trying to make
your program better for
yourself and for your students,
the A-bet process will happen naturally
as a consequence of that.
Yeah, and there's an analogy for that.
You know, if you give a question on a
test that requires maybe a sentence or
two to answer and the students who know
the answer will answer
it in a sentence or two.
And those that don't will fill every
little spot on the paper with words
hoping that you will find the answer in
what they've written.
And it's kind of the same thing you'll
see with a program that doesn't really
know what's required.
They just do a lot of stuff
hoping it's the right thing.
And if you understand what is really
required, as Steve said, it's really not
that hard to meet the criteria.
You have to document some things, you
have to do some things.
But if you're just interested in getting
your making your program better, it's not
that much work if you
know what work is required.
Part of this, maybe you can you probably
know the history better than I do, but
EC2000, when A-bet started the outcomes
assessment based accreditation, which was
a big shift for them at that time.
I believe that was born out of ISO 9000.
Is that correct?
I it sounds it sounds
like it would be to me.
I don't I wasn't around at
that time, so I'm not sure.
But so I think that's likely you haven't
said this, but industry is pretty
involved with A-bet as well.
And I because I first got in when our
program was going to be evaluated in
1999, I think under the EC2000.
That was the Engineering Commission 2000
project has changed
the way A-bet did things.
We agreed to do it then.
And lucky me, I got sucked into being the
one who had to do the self study.
And so that was my initiation.
And, you know, it was explained to us
that in factories, people always make
measurements to make sure the product
that comes out at the end of
the line is a quality product.
And so they had all
sorts of metrics involved.
And this was pretty much done
universally in all industry.
So they were saying, why don't
you do the same for education?
And while I think that's a great idea and
I think it's it's come a long way, I
really do have to always say measuring a
part in a factory is a lot easier than
measuring student outcomes.
And I think that there has
to be some give by industry.
And I saw this on teams where the
industrial evaluators didn't quite get
that and they were always much pickier
than the academic evaluators.
Yeah, sometimes although they also they
get the continuous improvement part more
easily than some of the academics, I
mean, they get the idea
that, oh, yes, you do need to.
It's about the improvement as opposed to
sometimes, at least in the beginning
stages, I guess, I think
we've gotten better at it.
But in the in the beginnings, we as
professors didn't quite get it.
I think it took us a little while to
understand that we have we are assessing.
It's just where we give tests, we know we
give homeworks, we have
them do things to assess.
We don't think of it that way.
So I think it would be useful.
Maybe you can tell us go through the
whole A-bet process, because a lot of
programs out there, you know, it's one of
these things I've been doing it at my
school for a long time and helping the
dean much as like you've done.
And what's remarkable to me is that every
six years, the chairs are all completely
different and none of
them have experience.
And so it's reinventing the wheel every
single time you do it, which is why
process is so important.
So, in case there are programs out there
that are listening to this to try to
figure out, oh, my
God, we have to do a bit.
I have no idea how to do it.
Can you go through how the whole process
begins every six years?
And so people understand that.
Sure.
So A-bet comes at the
invitation of the institution.
No program has to be accredited by A-bet.
And so it's an invitation.
And that invitation is done through
what's called a request for evaluation.
That's done in January of the year
preceding when your
onsite evaluation will be.
So if you were to be evaluated, say it's
too late for fall 2024, so you could be
evaluated in fall 2025.
In that case, in January of 2025, you
would submit a request for evaluation,
which will just list these other programs
you want to have evaluated.
You provide one example transcript
because they need to see how the degree
is described on the transcript to make
sure the name is correct
and consistent and so forth.
So then you just wait.
And the next step is the team chair is
assigned to the evaluations.
That occurs typically around this time.
So they're in the process right now of
probably assigning team chairs.
The team chairs are commissioners, either
active commissioners or what
we call the team chair pool,
which are commissioners who have been off
the commission for a few years and are
still familiar enough that
they can still lead a team.
And so they're assigned typically in late
April, May, that time frame.
Once the team chair is assigned, then the
evaluation date is set.
And actually, for the last several years,
when you submit your request for
evaluation, you can at
that time request a date.
You can say this is the date when we want
to have our evaluation.
And in most cases,
that that is accomplished.
Some cases it may not work
out, but usually it does.
So once the team chair is established,
they confirm the date, the date set, then
the rest of the team is is assigned.
The program evaluators are assigned by
the professional societies.
So in the case of materials programs, the
program evaluators are assigned by TMS.
The only exception would be ceramics
programs are assigned by
the American Ceramics Society.
And all the different disciplines will
assign teams to all
the different programs.
So that's what's that's happening in
terms of preparing for the visit.
Now, in the meantime, the program needs
to provide a prepare a self-study report.
That's due in July, July 1st of the
summer before the visit.
Now, you would have already been should
have already has been working on that for
a while when we prepared for our so we
were visited about a year before last.
We started essentially really preparing
about a year before
the self-study is due.
Basically, the after that previous
academic year, you have that academic
year completed, your processes, most of
the self-study report isn't going to
change very much at that point.
So you can really write the
vast majority of the report.
I coordinated that so far.
The last two visits, I coordinated that
for our college and I asked for drafts by
actually January, even
though it's not due to July.
And my argument was, well, if you submit
it in January, you have a draft and we
find an issue, you got the spring to fix
it and turn it from a problem to an
example of a continuous improvement.
And some of the programs, well, nobody
disagreed with that.
Some of them complied, some of them did
not, but that's that's their choice.
So the preparation is take some time to
prepare that document on that document
basically describes how you comply with
all the different criteria.
There are some kind of bookkeeping type
things that collecting resumes and
collecting syllabi, which
could be a bit time consuming.
So starting on that early is a good way
to not have it all concentrated in June.
So now we have the team is set in July
and the self-study report submitted.
And then that team starts to work with
the institution to prepare for the visit.
The visits occur typically in the fall
from usually the earliest or maybe the
week after Labor Day.
And they go until most are the highest
concentrations October.
They go into November or
maybe a few in December.
And then the team can even before the
visit, the team is encouraged to interact
with the institution.
So the program evaluator will read the
self-study might have some questions.
If the PV doesn't have questions, it
probably means they're
not really doing their job.
There's always something that's not
clear, some additional information.
So if you get questions,
don't worry about that.
In fact, I would think that's a good
thing that you're getting.
You're finding out what issues that might
be there and you have a chance to address
it even before the visit in many cases.
Or if not, at least be prepared to
provide the evidence on site that you
might need to demonstrate compliance.
So that all occurs up until the visit.
The typical visits are the kind of
standard is starts on a Sunday.
Sunday morning, the team meets, comes on
campus in the afternoon, looks at
facilities, reviews, materials,
which are course materials and assessment
materials you've collected.
That's changing now.
That's a lot of that
might be available in advance.
So there's some discussion of how that
Monday might look a little bit different.
If you've already evaluated some of those
materials, that's the
traditional purpose of that Sunday.
And then most of the
meetings are on Monday.
So Monday is just a day full
of meetings from the program.
So each the team consists of a team chair
and then program evaluators where there's
one program evaluator for each program.
Exceptions might be if a program has a
name that has two that
requires two program criteria.
And one thing I forgot to mention when I
was talking about criteria in a division
in addition to those
eight general criteria,
there are what are called program
criteria and those are specific to the
different disciplines.
So materials engineering and material
science and engineering has one set of
program criteria which are different from
aerospace engineering or
mechanical engineering.
And those program criteria describe
either curricular topics or faculty
qualifications that are needed for that
discipline that are necessarily needed
for a general any engineer program.
So you're preparing so that each program
has one or maybe two if
they have a name like that.
Sometimes there are other members of the
team like observers who are part of the
training are just tagging along and
observing or from state boards or various
types of observers that can be part of
the team but aren't part of the
evaluation decision.
So that you'll meet with your program
evaluator will want to meet with the
chair will want to be
with faculty with students.
And so you'll work with the program
evaluators schedule
that that day of meetings.
There's typically a luncheon where the.
That you're allowed to host the team and
typically use that to have some alumni or
maybe some star
students to meet with this.
Be with the PV. You always want to get
your best students to have lunch with the
PV because that gives a very good
impression that you have these very
bright students and successful alumni.
And then the visit goes till Tuesday. So
on Tuesday the morning there's typically
not any meetings it's meant for
preparation of the report and for any
pickup meetings that might be needed.
So the team prepares a report a Sunday or
Tuesday afternoon
they'll read that report.
First they'll debrief the the program
chair just to talk through it and then
they have a formal reading
for with the president provost.
Really whoever the institution wants to
have their anything from just the
president provost and the team to
inviting all the faculty and that had
everything in between.
And so at the end of the visit the team
will read this report. This is what's
called the draft statement.
And you're not given a written draft
statement which you are given is a
written program audit form and that
program audit form will have a
description of any
shortcomings that were identified.
So you have a written version of those
shortcomings to see and work on if there
are issues to be addressed and you can
immediately start working on those. You
don't have to wait for
the rest of the process.
So the rest of the process is first you
have seven days to say if there's any
mistakes in that what was read.
But that's really just errors of fact.
It's not we disagree with this or we've
done this additional work.
It's just that was wrong.
You said we have 15 faculty. We really
have 16 or something like that. And then
it goes through an editing process. And
this editing is not just language.
It's it's really about content and
interpretation and judgment.
So the first level it goes to.
And there's two letters two levels of
editor editors editors one editors two
editors one are members of the executive
committee of the commission.
I think EAC now has about 20 of those 20
21 maybe something like that.
So all the AC will probably be evaluating
seven 800 different programs.
So it's a lot. So those are
divided among those 20 people.
So they'll read it and come back.
They may have some questions.
But why was this done this way.
What did you really see here.
You know this doesn't look
like what we typically do.
So what they're really looking for is
trying to make everything consistent
among all the different teams that are
going out because those couple hundred
teams are going to all different
institutions with different people.
Hundreds of different people all trying
to make a consistent decision.
And that's a that's a challenge.
We have training to try to do that.
But training is not
going to cover it all.
So you have this process to have checks
of people who have seen more have more
experience and can help them to make
things similar at different institutions.
Then the second level of
editing are the officers.
And now in the AC its
officers plus one or two more.
And they're looking at even more programs
and they're they have more experience and
they may go back and forth with some
discussion of what this should be.
And then finally it goes to what are
called the adjuncts and the adjuncts are
part time a bet staff.
AC has four of them.
All four of those have been chairs of the
commission and now they work part time
for a bet that they don't make the
decisions but they provide guidance.
And because they have even more
experience and we'll ask
questions and go back and forth.
So it goes through all of this process
before you finally have
what's called the draft statement.
This takes a couple of months typically.
That draft statement will may look a
little different than what was read.
Some of the shortcomings
might have changed in level.
Some might have gone away.
So that's based on the judgment and what
what what's typically done
at different institutions.
Then that draft statement
comes back to the institution.
And at that point the
program has 30 days to respond.
What's called 30 day
response at that point.
Then you can provide well we have changed
this in this way or provide a different
additional information.
Talk about what you're planning to do to
address a shortcoming because sometimes
maybe it's not ready yet.
But you still have to
respond within 30 days.
Now if there is information that is
needed afterwards you can request to
submit what's called post 30 days post
post 30 day information.
And that can typically be provided until
about May 15th or May 30th.
And it's at the discretion of the team
chair but any team chair is going to
accept anything that's
reasonable in a reasonable time.
So there'll be an agreement that we're
going to provide this information by May
15th or whatever the agreed upon day is.
That would be things like say you're
changing your assessment processes and
you want some assessment
data from the spring semester.
So obviously you may not have that when
you have to provide your response.
That's a perfectly legitimate
request or capstone projects.
We were missing some aspect of the
capstone experience.
So we need to provide
this is what we've done.
This is how we change the course.
And in May we will provide the actual
report so you can see that what we did
was actually accomplished.
Then once that's that that goes through
that whole same editing process again.
So the response is evaluated
just like the initial report.
And that's the report
that goes to the commission.
The commission meets in July typically
about second third week in July.
So they have this all these reports and
they're they're going to make the vote on
these whether they whatever that
recommendation is for accreditation.
The in addition to all these other checks
there's a what's called
a consistency committee.
So that's a committee that's assigned.
They look at all of the reports and try
to look for anything that
might not be consistent.
Like there's all these two situations
look kind of similar.
This one's a weakness.
This is a concern.
And they'll ask the
commission to take a look at it.
The commission evaluates these in panels
so that they're divided into smaller
groups so that each group has about
probably about 10 or 12.
Institutions to look at
which might be 40 or 50 programs.
And so looking at a smaller subset of
this these hundreds of programs and
they'll discuss and make sure that yes
this looks like it's
the right thing or not.
Then that goes to the commission.
The other thing is all of the
professional societies have
representatives there.
And those professional societies will
look at reports from their programs.
So the TMS and the
Trammell Society group.
We get together and we look at all the
materials programs and look at this
doesn't look right to us and we can go
and raise an issue or we can go talk to
the team chair and and address something
if we think something
doesn't look quite right.
And then the commission will finally vote
and vote on some accreditation action.
And so those actions the one you want is
what's called the next general review.
And that means you don't have to do
anything in terms of formal reporting
until your next review which is six years
later or really five years by then.
By the time you get to this this because
this is already the
summer following your visit.
Then there's also interim actions which
can either be an interim
report or interim visit.
Those are in two years.
And those those the depends on the
different shortcomings and whether it's a
report or visit kind of depends on the
nature of whether it's something that can
be reported a document
or needs to be observed.
Or the or there's another type of interim
is a show cause which is more serious
that it could lead to loss of
accreditation after that.
It's just when you have a deficiency as
opposed to weaknesses would
be for just to enter and visit.
And so that decision is made and then
it's communicated to the program or the
institution directly from a bet typically
in August August or maybe into September.
They after the meeting they have to make
sure everything's correct and takes a
little bit of time to do that.
So it's a long process.
It's going on two years.
In addition to that if it's the first
time there's a program from an
institution there's what's called a
readiness review which occurs before that
request for evaluation.
And the purpose of that is it's
submitting a kind of a poor part of a
self study and it's really just to make
sure that the program kind of understands
what they're getting into.
And then there's a recommendation from
that that said yes looks like you're
ready to go or no we don't think you are
but it's not binding the program can go
ahead and institution can go ahead and
even if it's recommended to
not try they can still go ahead.
It's just a recommendation that we think
you don't quite understand what you know
what you're up up against.
So maybe you should wait or the program's
not quite ready or you
don't really understand.
So that's the process.
It's about a two year almost a two year
process to get through.
Thank you.
And obviously the goal of a program is to
have no shortcomings at that final exit
meeting because that
means you have an NGR.
There's nothing to complain about.
I think it's really important for
programs to realize it's really important
that they think deeply about whether or
not they're going to be compliant the
summer before the visit.
And to be really honest with the
evaluator and accept what the evaluator
tells them way back in the summer before
the visit because good evaluators will
let programs know early if
they think there might be a
problem especially these days when the
resource room is all in Dropbox or
Box.com or something like that.
So there's you know except for meeting
with all the people and
seeing the facilities.
Ninety five percent of the visit should
be done before the
evaluator even gets there.
And that's important because if a program
gets early warning that they haven't been
doing continuous improvement.
They haven't been
documenting things all of that.
They can start in the fall term and make
sure that they've done everything that
they need to do to demonstrate that
they're now compliant.
So when the draft statement comes to them
they're in very very good shape to make
sure those shortcomings are removed.
And I can't emphasize that enough for any
program going through.
So if you just you know even if you're in
trouble you know you should do it Jeff
suggested have yourself study written in
January before you're going to be
evaluated the following fall because that
gives you the winter term
or spring term to get ready.
And even if that doesn't work even if you
get shortcomings you
have the whole fall term.
But if you just sit there and wait and
take it and your evaluation happens to be
in November boy you're in a tough spot.
And so don't let
yourself get in that spot.
That's that's my personal advice.
And I would say the most shortcomings are
resolved most can be and many are.
So if you do get a
shortcoming at the visit.
Obviously it'd be nice if you can avoid
that as Steve describes.
But even if you if there is still
something there you have time and most of
the shortcomings in my experience do get
resolved or at least reduced to the level
where they don't affect
accreditation the action.
So don't get discouraged
if you do get something.
There's time and that that's the goal is
that everyone is compliant.
It's not the goal to see how many
shortcomings we can ring up.
It's we want the we want to confirm that
the program is compliant.
And if not can we help them get there.
You know that's the perfect lead to the
next question that I wanted to ask which
is what advice do you have for programs
to help them be more successful.
What can they do to
learn about these processes.
Where can they go.
Is there training they can do.
And how can they make sure
they have a successful visit.
To me the best way to learn is to become
a program evaluator.
You learn so much.
I mean it's just like if
you're writing a proposal to NSF.
What do you do to prepare for that.
You serve on a panel.
So you serve on a panel and you say oh
now I see what kills a proposal and I
know now I see why that proposal it's
it's it's it's stood out.
And this is what made it stand out.
It's it's the same.
And as I mentioned before once you
understand what's required
it's really not that hard.
Now that does that is a commitment
because it I would estimate it probably
takes a maybe a week of your time.
If you do one visit per year which is
kind of typical of a P.E.V.
Maybe a week of your time over the travel
the preparation and it varies a lot.
It gets shorter as
you get more experience.
I'm accurate a self study much faster
than I could when I started.
But it is a commitment.
There's no cost.
It pays for all your costs.
Of course your time is cost
but no out of pocket costs.
That's the best way I think.
But there are other opportunities.
TMS training that we
have for program evaluators.
We allow other people
to come just to learn.
That's how I got involved.
I was I was going to be in charge of the
next review for my program.
And TMS had the P.E.V. training and they
said anyone can come.
So I went thought I might have to learn
about this and they said
well do you want to be a P.E.V.
I guess I guess I'll
learn more if I do that.
So I did and and I kept with it.
And I find it very rewarding to do.
But it is a bit of a commitment.
So that's not for everybody
but that would be the best.
But there are other ways to learn.
There is a bit symposium this way.
There are other ABET has other training
programs to help to train
people in understanding these
these criteria.
We have a workshop or
a symposium every year.
It's been at at the fall meeting until
but this next year is
not going to be this fall.
We moved it to the annual meeting.
So it'll be in the
2025 TMS annual meeting.
It's called the Judson symposium.
It's named after Beth Judson who I
mentioned earlier was the one
who started the reevaluation
of the student outcomes and she
unfortunately was killed in
a plane crash about the year
after she started that.
So we named the
symposium in honor of her.
And in fact getting I think one of you
mentioned earlier about
just doing the right thing and
a credential come along the way.
The original name of the symposium was
continuous improvement of academic
programs and then in
parentheses and
satisfying ABET along the way.
We wanted to put ABET in there to get
people's attention
because ABET does get attention.
But we didn't want it to
be just this is about ABET.
It is but it's really about how do you
improve your program.
Yes you do need to look and see well what
does ABET require and
just make sure you have
those things.
But I think you find if you if you're
just trying to improve
your program the things you
have to add are probably pretty minimal
and some of them might
even help you to do what
you were doing better particularly things
with documentation and things like that.
So that's in TMS we
have that opportunity.
I think other professional societies have
similar types of programs where they have
committees that are working on this and
trying to help their programs as well.
That's great and to reinforce that even
more the Judson
Symposium that used to really be
all about from the accreditation
committee has now joined
forces with the education committee
and it's a joint meeting.
And Tim and I are both on the education
committee now and that's
where this whole idea for the
podcast actually came from.
So it's very appropriate we're talking to
you since you've been
very involved in the Judson
committee symposium for a long time.
So you already mentioned that ABET is
continuously improving.
Where is it going now?
What's next for ABET?
Yeah so there I mean there are a number
of areas that are kind of hot topics.
Probably one of the
biggest one right now is DEI.
And I was happy when I was a chair we
actually approved some changes to the
criteria to incorporate
DEI.
And ABET had been
trying to do this a while.
They first worked on kind of the overall
statements of the
organization and those all
fine but really where the impact is going
to be is if it's in the
criteria where programs have to
do something.
That's where I think we're
going to really have impact.
So they asked all the commissions to try
to put DEI related
topics into the criteria.
And so we this was so this is 2019-20 and
they had asked that and so
I was a chair at the time
and so I kept on our criterion committee
to make sure they kept working on this.
And they did and they came up with
proposals in four
different criteria to have changes.
And happy that we did this
even though this was COVID.
So we were also dealing with how to
transition all of the visits to virtual.
So it was a it was a hectic
year but we did pass those.
And so I was really pleased with that and
it's been a while the so
after the commission approves
that it has to be approved by another
body called the
engineering area delegation.
So ABET used to have a board of like 55
people and a board of 55
people cannot function.
No absolutely not.
And the reason it was
so big is because ABET is
has representatives all these
professional societies and you have to
give them representation
and you have to have a little bit
different representation
for the ones that have more
programs and so it was a big body.
And so they reorganized so they now have
a small board of I think 13 people
but they still have the area delegation
or the well the area
delegation are for the different
commissions and the board of delegates is
all of those together.
And that's where the
representation comes.
So TMS has a representative on the
engineering area delegation
and the board of delegates.
And that's the body
that has to then approve.
Well they didn't want to approve it they
wanted to they kind of kicked
it down the road went through
talking about it for a while and tried
again and finally a year
later it did approve did get
approved two of the four because of the
of the criteria there's
eight criteria of those five are
what's called harmonized which means
they're the same for
all four commissions.
So anything that changes and knows has to
be agreed on by all four commissions
which is can be a challenge.
The other three are commission specific
so we can EAC could pass those.
So two of them those two passed the other
two they passed the
commission but then they got
stopped by the accreditation council
because none of the
other commissions were ready.
So the other two got through and they
were in criterion five and criterion six.
So criterion five is curriculum and
criterion six is faculty.
And so it went through it took a couple
years to finally get them approved.
And now they're still not approved but
they are they have approved a pilot study
that pilot study we're going into the
second year of the pilot study.
And so the idea there is we'll go through
the pilot study kind of learn
how is this being valid how are people
responding how are they how are they
saying they're compliant how are we going
to judge that how are we
going to train people to
evaluate that so there's some issues that
have to be worked out and
that's kind of what we're
trying to do these these couple years. So
that's that's the in process
the essence of those additions
are essentially I mean you're designing
things well I should say it
mentions that you have to have
DEI in the professional context and to me
that's the important part
it's not just oh you need to go to a DEI
seminar and learn about diversity,
equity, and inclusion.
How is that important for engineering?
And obviously it is I
mean to me the two ways it
stands out is one is you have to work
with different people
you have to understand how
to do that and do that effectively. And
secondly probably with even
the bigger impact is you're
designing whether it's materials or
products or processes for a lot of
different people and if
you don't understand and don't have
perspectives from different people your
market share is going
to be people just like you right and if
you're going to really just to be
effective even getting
setting aside the social justice aspects
of it just talking
about the business case for
the diversity is you're going to have a
product that is is more
useful for more people.
And so that's kind of to me the essence
of what I think is
appropriate to put in a engineering
criteria because that is important to be
a successful engineer.
And of course for the
faculty it's about understanding those
concepts I would say so you
can support that education.
So that's going through this pilot study
I'm hopeful that it will
you know it may be tweaked
a little bit because as as we in the
first year of this self the
first year of the pilot study
what happened was a lot so that was
optional and any program could
decide to do it or not it had
absolutely no bearing on the evaluation
it was evaluated not
even by the team it was
evaluated by a separate group of people
that just evaluated
that. So there's no risk and
some programs did I think it ended up
being maybe 40 institutions said they
would and maybe 20 some
actually did something on that order.
What they found was they
the programs often just talked
about what they did in DI but didn't
really address the criteria
and how they were compliant
with those words. So in the second year
they changed the prompt so
there's a like an addendum
to the cell study if you're if you're
going to participate to really try to
make that more clear
that you really have to demonstrate how
are you doing what we've
described here. And so I'm going
to be working with a task group to look
at that and also start
thinking about what are we looking
for in terms of their criteria and how
are you going to train our
our teams to be able to look
at that and interpret that. So that to me
that's the biggest thing
that's on the horizon. There's
also there's DEI coming in different
areas too. There's a proposal coming
through in the institutional
support with some kind of general DEI
related language. The kind
of language that's in there
now is the kind of thing that all
institutions are doing just because of
federal law. So I'm not sure
that that's going to be as impactful but
I think in criterion five
where you're talking about
in that experience in that design
experience how are you
preparing your students
in the context considering equity and so
forth when you're
designing that's going to make
better engineers and if every program has
to every six years
explain how are they doing that
that's I think that's I think that'll
really move the needle more than a lot of
other things that we
we do need to do the other things but I
think this is really
impact. Totally agree with you.
Are you at all concerned about the new
politics of DEI and DEI anti-DEI
legislation and now I believe Congress is
proposing to write a law specifically
talking about accreditation bodies. So
how is ABET responding to this?
ABET's very cognizant of that and they're
looking at it and
they're looking at the laws and
they don't want to put in it they don't
want to they don't want
to not do this I think ABET
I mean I know I know enough of the people
in the leadership and
so forth I think they are
committed to doing this and they're going
to find find a way to do it
but they don't want to put a
program in in the position where if they
want to be accredited
they're going to lose state funding
I mean they don't want to do that so
they'll look at a way to do that. I'm
experienced that we had a
legislation passed a couple weeks ago
here and when you read it
first it looks like we're in
trouble but actually ours fortunately
related accreditation does
have a clause that says you
know if you have to do something because
of accreditation you're
excluded from this so we're
okay and at this point but it's going to
take I think it's going to
be a lot of um a lot of it
is the words and those words are
triggering and in our areas we're
stopping we're not using DEI as
much of course I'm in Alabama where it's
a little bit different
than Michigan but we we
so we try to describe the things without
using the triggering words
you can because what you do
it's not as people don't really object to
the things you might
do they object to those
words that they have some other
connotation of what
you're really talking about
so some of it's going to be trying to
describe things in a different way
that accomplish what we want them to
accomplish for example you talk about
equitable design well
you can talk about universal design in
kind of doing the same
things how do you make this design
usable by more people well you're doing
that for equity but if you
call it equity then that may
trigger some people so you might call it
universal design you're also
doing it for everybody so you're
doing it for people who are introverts
people who are extroverts people from
different socio-economic
groups and it's everybody it's not
absolutely particular group
that gets you in trouble with
politics absolutely yeah something else
i'm really happy to hear about that
approach is that it's
about programs taking action not just
saying i have this
principle that i believe in i mean
it's nice to believe in principles but if
that's all you're doing is
believing in a principle and
not doing anything about it it's kind of
like so what so the fact
that you're saying what are you
doing to make your students more
successful in you know being a part of
this world that we're in
together i'm really happy to hear that
yeah in fact our
legislation that was passed in alabama
they um it it talks about the what
they're really objecting to
is forcing someone to believe
something which we don't do that we teach
this is the you know this
is and this is how you design
things this is why it's good but we don't
say you have to believe in
this principle which is what
they're kind of objecting to and some of
the exclusions are the
things like well if you're
supporting students even if you're
targeting a particular group
that's okay if it's just about
supporting students as long as you're not
making limiting
participation based on being in a group
which we don't do you know so um yeah i
agree and so it's when you think about
well what do you need
to do to accomplish what you might
describe as your dei goals well what do
you have to do to get
there you describe that and that that
you're not going to get
objection to so i think that's
probably the way abet's going to have to
do it they may have to change the
language to not make
it offensive but still force programs to
think about those things
and it's already in there i
mean with the teams right we talked about
work on diverse teams has
been there for years and that's
important and it's already in there but
then the kind of the design
part maybe it is in there too
but and communication to a range of
audiences it's there too
absolutely well jeff this has been
fantastic thank you so much i want to
mention that we're hosting
the north american materials
all the programs.
outcomes assessment process, but some of
those programs just
thought it was done and they
forgot to also document the continuous
improvement of their
program, which is really the whole
point.
And it's okay if measuring
the assessing the outcomes.
It's a really important thing to do
because it gives you
great insight into whether or
not you have a problem
anywhere and need to address it.
And one thing I've learned is by using
all the data and having
a histogram, I can't help
but look at the little
tick marks in the tails.
Even though, yeah, we have, you know, 80%
of our students are beyond our threshold,
you see that tail and now I'm wondering,
well, what about them?
And because we actually have all the
data, the actual unique
names, it's all secret.
I can't see who they are, but we have it.
And now we have a new AI tool that knows
how to look into SQL databases.
Again, our AI tool is private.
It's completely FERPA compatible because
we don't give any
information to OpenAI or Microsoft
or Google.
We've licensed their tools, we pay for
it, so that we can work
on our curated dataset.
Why not work on that?
We have 6,000 students in our system now.
Maybe we can do small longitudinal
studies with, you know,
here are these people in the
tails.
How did they do in the other criteria?
How did they do the next
year they were assessed?
And so I think this ABET process,
although this is not
explicitly part of ABET, it's
allowing us to view our data in ways we
probably never
dreamed we'd be able to do.
And it might be one of the advantages of
artificial intelligence.
It's certainly these large language
models to figure out what's going on.
So we're kind of excited about this, but
you still have to improve your program.
And we do that with lots of other ways.
Tim is creating a new math
course for all of our sophomores.
We had instituted a fifth math course in
our curriculum,
thinking it would help them
learn the math they
needed for the courses.
And the students all waited until their
last year, senior year
to take it, which defeated
the purpose.
So we reevaluated.
Now we're going to offer it as a
sophomore level course
and cover, you know, complex
variables or orthogonal series,
eigenvalue problems, all this
stuff that they don't really
get in their calculus sequence.
So these are all the things that good
programs will always do
to improve their program.
And it never ends.
We're constantly having to deal with it.
So anyway, any final
thoughts before we take off?
The only thing I guess I would say is,
you know, I know
people sometimes have issues
with specific things that ABED does, but
I think ideally the
idea that someone's coming
to look at your program and give you
feedback should be a positive thing.
And when we find that our program
developers are not doing
that, we either stop sending
them or try to encourage
them to follow that philosophy.
But that really is what it
should be, is meant to be.
And I think I'm still hopeful that we're
moving closer to that.
And I hope we look at it that way and try
to work towards that would be what I like
to do.
All right.
Well, thank you very much.
And I think we'll sign off.
So see you later.
Excellent.
Thanks so much.
See you next time.
Okay.
Thank you.