1 01:00:00,033 --> 01:00:10,466 [MUSIC] 2 01:00:15,533 --> 01:00:17,966 Hello and welcome to another episode of 3 01:00:17,966 --> 01:00:19,066 Undercooled and 4 01:00:19,066 --> 01:00:21,166 Materials Education Podcast. 5 01:00:22,233 --> 01:00:23,699 Today we have a special guest. 6 01:00:23,733 --> 01:00:26,333 We have my friend Jeff Fergus from Auburn 7 01:00:26,333 --> 01:00:30,033 University who knows more about ABET than 8 01:00:30,033 --> 01:00:31,000 anyone else in the 9 01:00:31,000 --> 01:00:32,266 entire world that I know. 10 01:00:33,099 --> 01:00:33,966 And so we're going to be 11 01:00:33,966 --> 01:00:36,233 talking all about ABET today. 12 01:00:36,633 --> 01:00:39,233 But first, Jeff, why don't you tell us a 13 01:00:39,233 --> 01:00:40,366 little bit about yourself? 14 01:00:44,266 --> 01:00:44,533 Sure. 15 01:00:45,033 --> 01:00:45,966 So glad to be here. 16 01:00:46,933 --> 01:00:48,466 So I'm originally from Illinois. 17 01:00:50,033 --> 01:00:52,099 Went to University of Illinois and the 18 01:00:52,099 --> 01:00:53,900 University of Pennsylvania, which is 19 01:00:53,900 --> 01:00:55,599 where I first met Steve. 20 01:00:56,566 --> 01:00:58,733 Did a post-doc at Notre Dame and had been 21 01:00:58,733 --> 01:01:00,633 at Auburn ever since. 22 01:01:01,133 --> 01:01:02,533 I came here in 92, so 23 01:01:02,533 --> 01:01:03,966 I've been here for 31 years. 24 01:01:05,333 --> 01:01:06,966 My research area is generally kind of 25 01:01:06,966 --> 01:01:08,066 high temperature chemistry, 26 01:01:08,333 --> 01:01:10,666 electrochemistry, batteries, 27 01:01:10,866 --> 01:01:12,133 fuel cells, things like that. 28 01:01:13,066 --> 01:01:15,166 But been in the dean's office about 10 29 01:01:15,166 --> 01:01:16,233 years in two different 30 01:01:16,233 --> 01:01:17,166 associate dean roles. 31 01:01:17,666 --> 01:01:18,833 First was program 32 01:01:18,833 --> 01:01:20,333 assessment and graduate studies. 33 01:01:21,066 --> 01:01:22,333 And now I moved over to undergraduate 34 01:01:22,333 --> 01:01:24,300 studies and program assessment. 35 01:01:25,400 --> 01:01:26,366 So that's a little bit about me. 36 01:01:27,466 --> 01:01:27,633 Tim? 37 01:01:29,833 --> 01:01:30,233 Awesome. 38 01:01:31,333 --> 01:01:35,099 Well, with that background, of course, 39 01:01:35,099 --> 01:01:36,666 the theme of today's show is ABET. 40 01:01:37,066 --> 01:01:38,966 So can you first actually just tell us a 41 01:01:38,966 --> 01:01:40,833 little bit about what is ABET? 42 01:01:41,033 --> 01:01:41,800 There may be people in 43 01:01:41,800 --> 01:01:42,866 the audience who don't know. 44 01:01:43,533 --> 01:01:44,266 And then how are you 45 01:01:44,266 --> 01:01:46,066 personally involved in the program? 46 01:01:46,866 --> 01:01:47,066 Sure. 47 01:01:47,433 --> 01:01:50,033 So ABET's a confederation 48 01:01:50,033 --> 01:01:51,433 of professional societies. 49 01:01:51,733 --> 01:01:55,266 There's 35 professional societies that 50 01:01:55,266 --> 01:01:58,400 have basically come together to establish 51 01:01:58,400 --> 01:02:00,733 criteria to prepare programs, 52 01:02:02,133 --> 01:02:04,466 to evaluate programs to prepare students 53 01:02:04,466 --> 01:02:07,566 in areas, engineering, computing, 54 01:02:08,300 --> 01:02:09,333 engineering technology, 55 01:02:09,333 --> 01:02:10,566 applied and natural sciences. 56 01:02:11,500 --> 01:02:13,400 And so the purpose is to set these 57 01:02:13,400 --> 01:02:16,933 criteria that programs can develop 58 01:02:16,933 --> 01:02:19,300 programs so that they can prepare their 59 01:02:19,300 --> 01:02:21,800 students to be prepared, be ready to 60 01:02:21,800 --> 01:02:22,666 mentor the profession. 61 01:02:24,433 --> 01:02:26,099 I've been involved with ABET as a 62 01:02:26,099 --> 01:02:28,733 volunteer for about 20 years or so. 63 01:02:29,766 --> 01:02:31,733 As a program evaluator, I 64 01:02:31,733 --> 01:02:33,166 served on the commission. 65 01:02:33,833 --> 01:02:35,599 There are four different commissions in 66 01:02:35,599 --> 01:02:38,699 ABET in the different technical areas. 67 01:02:38,733 --> 01:02:40,833 So engineering is the largest of those. 68 01:02:41,733 --> 01:02:43,066 Engineering technology is another, 69 01:02:43,599 --> 01:02:44,733 computing and applied 70 01:02:44,733 --> 01:02:45,666 and natural sciences. 71 01:02:46,400 --> 01:02:47,733 So I served on that commission. 72 01:02:48,333 --> 01:02:50,300 I moved into the executive committee, 73 01:02:51,033 --> 01:02:52,933 also then into the officer change. 74 01:02:52,933 --> 01:02:54,233 So I was the chair of 75 01:02:54,233 --> 01:02:57,433 the commission in 2019-20. 76 01:02:58,833 --> 01:03:01,033 And since I've also worked on training. 77 01:03:01,333 --> 01:03:04,066 So I'm a facilitator for the training 78 01:03:04,066 --> 01:03:05,233 programs used to 79 01:03:05,233 --> 01:03:06,500 train program evaluators. 80 01:03:07,333 --> 01:03:10,066 And I'm now a lead facilitator for that. 81 01:03:10,066 --> 01:03:11,366 Been doing that for I 82 01:03:11,366 --> 01:03:13,433 think since about 2011 or so. 83 01:03:14,666 --> 01:03:16,633 And currently I'm the chair of the 84 01:03:16,633 --> 01:03:18,033 training committee of 85 01:03:18,033 --> 01:03:19,433 the Accreditation Council. 86 01:03:19,933 --> 01:03:22,466 So the Accreditation Council is a 87 01:03:22,466 --> 01:03:24,833 committee that represents the four 88 01:03:24,833 --> 01:03:25,500 different commissions. 89 01:03:26,233 --> 01:03:28,033 Some of the things that ABET does are 90 01:03:28,033 --> 01:03:30,400 specific to the different commissions and 91 01:03:30,400 --> 01:03:32,766 some are common across 92 01:03:32,766 --> 01:03:33,566 the four commissions. 93 01:03:34,400 --> 01:03:36,000 And when things are common, then this 94 01:03:36,000 --> 01:03:37,966 body is the one that kind of coordinates 95 01:03:37,966 --> 01:03:40,466 so that those can be done. 96 01:03:41,166 --> 01:03:41,933 And things can be 97 01:03:41,933 --> 01:03:43,266 accomplished for all four commissions. 98 01:03:44,066 --> 01:03:45,666 And so that council 99 01:03:45,666 --> 01:03:47,266 has a training committee. 100 01:03:47,800 --> 01:03:49,933 And that training committee is the body 101 01:03:49,933 --> 01:03:51,266 that prepares the training, 102 01:03:51,466 --> 01:03:54,133 organizes the training for, 103 01:03:54,666 --> 01:03:56,666 mostly for program evaluators, but also 104 01:03:56,666 --> 01:03:59,633 some of the team chair training that's 105 01:03:59,633 --> 01:04:00,733 common among the commissions. 106 01:04:00,800 --> 01:04:01,300 That's great. 107 01:04:02,000 --> 01:04:03,833 And so ABET's been around, I believe, 108 01:04:03,833 --> 01:04:05,233 since 1930 something. 109 01:04:05,533 --> 01:04:06,033 Is that right? 110 01:04:07,266 --> 01:04:11,266 And under different names, reinvented 111 01:04:11,266 --> 01:04:13,733 themselves multiple times. 112 01:04:14,099 --> 01:04:16,400 Two times since I've been around since 113 01:04:16,400 --> 01:04:21,366 around 1998 when they started doing the 114 01:04:21,366 --> 01:04:24,733 outcomes assessment based approach. 115 01:04:25,266 --> 01:04:27,733 And then I believe it was another, I 116 01:04:27,733 --> 01:04:29,733 don't remember how many years when they 117 01:04:29,733 --> 01:04:33,466 redid the student outcomes pretty 118 01:04:33,466 --> 01:04:34,866 dramatically going from A 119 01:04:34,866 --> 01:04:36,599 through K to one through seven. 120 01:04:37,666 --> 01:04:43,466 So ABET is always asking programs to do 121 01:04:43,466 --> 01:04:44,699 continuous improvement. 122 01:04:44,966 --> 01:04:47,133 And I think ABET's been practicing 123 01:04:47,133 --> 01:04:47,900 continuous 124 01:04:47,900 --> 01:04:49,633 improvement for quite some time. 125 01:04:50,533 --> 01:04:52,966 That said, why should, you know, this is 126 01:04:52,966 --> 01:04:55,733 a podcast about materials education. 127 01:04:56,533 --> 01:05:01,566 So our prime audience are the materials 128 01:05:01,566 --> 01:05:02,833 programs in the United 129 01:05:02,833 --> 01:05:04,666 States and around the globe. 130 01:05:05,800 --> 01:05:07,366 I believe there's something like 125 131 01:05:07,366 --> 01:05:09,933 materials programs, something plus or 132 01:05:09,933 --> 01:05:12,833 minus five programs, which is a lot. 133 01:05:14,633 --> 01:05:15,833 Why should materials 134 01:05:15,833 --> 01:05:17,566 programs care about ABET? 135 01:05:17,800 --> 01:05:19,333 Why are we involved with ABET? 136 01:05:23,166 --> 01:05:25,666 So there's a first kind of a practical 137 01:05:25,666 --> 01:05:30,233 reason that many entities look for ABET 138 01:05:30,233 --> 01:05:31,066 accreditation when 139 01:05:31,066 --> 01:05:32,566 they're looking for engineers. 140 01:05:33,733 --> 01:05:36,266 In particular, if licensing is involved, 141 01:05:36,533 --> 01:05:39,566 most of the jurisdictions for licensing 142 01:05:39,566 --> 01:05:42,333 of engineers require a degree from an 143 01:05:42,333 --> 01:05:43,733 ABET accredited program. 144 01:05:45,066 --> 01:05:47,800 Now, many materials engineers don't need 145 01:05:47,800 --> 01:05:50,566 that licensing, but there are some areas 146 01:05:50,566 --> 01:05:51,466 where it is important. 147 01:05:52,066 --> 01:05:54,366 For example, if you're going to go into 148 01:05:54,366 --> 01:05:57,566 consulting and say failure analysis, it's 149 01:05:57,566 --> 01:05:59,000 very important to have that professional 150 01:05:59,000 --> 01:05:59,766 engineering license. 151 01:06:00,666 --> 01:06:03,133 So that's kind of a practical reason. 152 01:06:03,133 --> 01:06:06,166 And many companies or other entities, 153 01:06:06,166 --> 01:06:08,900 when they're hiring, they'll put that as 154 01:06:08,900 --> 01:06:10,666 a requirement for the degree or 155 01:06:10,666 --> 01:06:12,033 preference, if not a requirement. 156 01:06:12,833 --> 01:06:14,833 So those are kind of practical reasons, 157 01:06:15,800 --> 01:06:18,566 but it really the process involved, 158 01:06:18,833 --> 01:06:21,566 although it sometimes may seem tedious 159 01:06:21,566 --> 01:06:23,233 and parts of it may not seem 160 01:06:23,233 --> 01:06:24,466 clear why they're important, 161 01:06:25,233 --> 01:06:28,300 that process of evaluating your program, 162 01:06:28,933 --> 01:06:31,033 looking at it to see what you can do 163 01:06:31,033 --> 01:06:33,599 better, is going to help you improve that 164 01:06:33,599 --> 01:06:35,666 program and better prepare your students. 165 01:06:36,699 --> 01:06:40,033 So I think when you really understand 166 01:06:40,033 --> 01:06:42,599 what's required from ABET, it's not as 167 01:06:42,599 --> 01:06:45,233 hard as sometimes people might think when 168 01:06:45,233 --> 01:06:46,199 you really understand. 169 01:06:48,033 --> 01:06:49,833 And if you do it in the right way, it can 170 01:06:49,833 --> 01:06:52,066 actually be a benefit and really help you 171 01:06:52,066 --> 01:06:52,866 improve your program. 172 01:06:53,066 --> 01:06:54,966 So I think it's really about, and that's 173 01:06:54,966 --> 01:06:56,366 the reason I'm involved, if it were just 174 01:06:56,366 --> 01:06:59,000 about checking a box for a licensure, I'm 175 01:06:59,000 --> 01:07:00,066 not sure I would have spent 176 01:07:00,066 --> 01:07:02,033 all the hours I have on it. 177 01:07:02,633 --> 01:07:04,033 But when it's really about trying to 178 01:07:04,033 --> 01:07:07,400 improve the preparation we're providing 179 01:07:07,400 --> 01:07:10,599 for our students, that's really what I 180 01:07:10,599 --> 01:07:11,599 think is important about it. 181 01:07:12,800 --> 01:07:15,666 I completely agree. It's so valuable to 182 01:07:15,666 --> 01:07:17,533 be able to say not just, hey, let's keep 183 01:07:17,533 --> 01:07:18,933 doing what we've always been doing, but 184 01:07:18,933 --> 01:07:20,433 to say, what can we do better so that our 185 01:07:20,433 --> 01:07:23,633 students are more successful and more 186 01:07:23,633 --> 01:07:24,966 prepared to make a 187 01:07:24,966 --> 01:07:25,833 difference in the world? 188 01:07:26,266 --> 01:07:26,566 Absolutely. 189 01:07:26,633 --> 01:07:30,766 So on the receiving side of things, as an 190 01:07:30,766 --> 01:07:32,733 educator at an ABET accredited 191 01:07:32,733 --> 01:07:34,666 institution, I'm often interacting with 192 01:07:34,666 --> 01:07:36,866 these criteria, the 193 01:07:36,866 --> 01:07:38,066 accreditation criteria. 194 01:07:38,966 --> 01:07:43,933 And so this makes me curious about where 195 01:07:43,933 --> 01:07:45,500 do the criteria come from, what are they 196 01:07:45,500 --> 01:07:48,633 specifically and what are the criteria 197 01:07:48,633 --> 01:07:50,699 that ABET is using to 198 01:07:50,699 --> 01:07:52,766 accreditate programs? 199 01:07:54,800 --> 01:07:55,733 So maybe I'll start 200 01:07:55,733 --> 01:07:57,433 with where they come from. 201 01:07:58,566 --> 01:08:00,466 As I mentioned, ABET is a confederation 202 01:08:00,466 --> 01:08:01,933 of professional societies. 203 01:08:02,933 --> 01:08:05,466 And so it really is us. 204 01:08:06,099 --> 01:08:07,233 It's all the different professional 205 01:08:07,233 --> 01:08:08,199 societies have 206 01:08:08,199 --> 01:08:09,833 representatives on the commission. 207 01:08:10,566 --> 01:08:11,733 It's the commission that 208 01:08:11,733 --> 01:08:14,000 decides what those criteria are. 209 01:08:14,566 --> 01:08:17,366 It's the commission that creates the 210 01:08:17,366 --> 01:08:19,366 teams that go out and evaluate programs. 211 01:08:19,366 --> 01:08:20,966 And it's a commission that decides 212 01:08:20,966 --> 01:08:22,233 whether or not they're compliant. 213 01:08:23,433 --> 01:08:25,866 So it is people from our different 214 01:08:25,866 --> 01:08:27,966 professional societies that determine 215 01:08:27,966 --> 01:08:29,666 what those criteria should be. 216 01:08:30,833 --> 01:08:33,466 And that's important to keep in mind that 217 01:08:33,466 --> 01:08:36,600 when we complain about ABET requires this 218 01:08:36,600 --> 01:08:38,033 or that and may not agree 219 01:08:38,033 --> 01:08:39,899 with it, ABET's not this. 220 01:08:40,399 --> 01:08:42,800 There is an office in Baltimore that has 221 01:08:42,800 --> 01:08:45,766 30 or 40 staff members, but they're 222 01:08:45,766 --> 01:08:47,966 really there just to manage the process. 223 01:08:48,633 --> 01:08:50,699 The decisions are made by representatives 224 01:08:50,699 --> 01:08:52,633 from all our professional societies. 225 01:08:53,399 --> 01:08:54,366 Now, those different professional 226 01:08:54,366 --> 01:08:55,666 societies have different 227 01:08:55,666 --> 01:08:57,566 interests and priorities. 228 01:08:57,566 --> 01:08:58,533 And so there is some 229 01:08:58,533 --> 01:09:01,399 negotiation that has to go into that. 230 01:09:01,399 --> 01:09:03,433 It has to apply to all these different 231 01:09:03,433 --> 01:09:05,633 types of institutions, types of programs. 232 01:09:06,333 --> 01:09:08,533 And so sometimes it can be the language 233 01:09:08,533 --> 01:09:10,966 can be a little bit vague because it's 234 01:09:10,966 --> 01:09:13,433 got to fit many different instances, 235 01:09:14,133 --> 01:09:16,800 which which takes some judgment and some 236 01:09:16,800 --> 01:09:17,933 understanding of them. 237 01:09:18,933 --> 01:09:20,266 So that's kind of where it comes from. 238 01:09:20,833 --> 01:09:23,133 The criteria in terms of what they are, 239 01:09:23,399 --> 01:09:24,833 there are eight what we 240 01:09:24,833 --> 01:09:26,066 call general criteria. 241 01:09:26,766 --> 01:09:27,800 And those are the criteria 242 01:09:27,800 --> 01:09:29,500 that apply to all programs. 243 01:09:30,533 --> 01:09:32,433 So the way I think of them at the top, 244 01:09:32,433 --> 01:09:33,633 you have what are called program 245 01:09:33,633 --> 01:09:34,733 educational objectives. 246 01:09:35,666 --> 01:09:39,166 And these are what the program expects 247 01:09:39,166 --> 01:09:41,566 its graduates to be attaining 248 01:09:41,566 --> 01:09:43,433 a few years after graduation. 249 01:09:44,333 --> 01:09:45,866 So it's kind of your North Star. 250 01:09:45,866 --> 01:09:47,300 What are you trying to do? 251 01:09:48,266 --> 01:09:51,133 And the program needs to determine those 252 01:09:51,133 --> 01:09:54,266 with involving its constituencies, 253 01:09:54,500 --> 01:09:56,366 whoever they determine those to be, which 254 01:09:56,366 --> 01:09:57,566 is typically employers and 255 01:09:57,566 --> 01:09:59,266 alumni, people like that. 256 01:09:59,866 --> 01:10:00,833 And I think that's one of the real 257 01:10:00,833 --> 01:10:03,966 benefits of the impacts of the change 258 01:10:03,966 --> 01:10:04,899 that Steve mentioned 259 01:10:04,899 --> 01:10:06,500 that occurred around 2000. 260 01:10:07,333 --> 01:10:09,066 And that is that's forced programs to 261 01:10:09,066 --> 01:10:11,533 really engage more with their alumni and 262 01:10:11,533 --> 01:10:13,466 their employers to get feedback on what 263 01:10:13,466 --> 01:10:14,533 they should be doing. 264 01:10:15,566 --> 01:10:16,566 So then below that, 265 01:10:16,566 --> 01:10:17,466 it's how do you get there? 266 01:10:17,733 --> 01:10:18,933 And the first step is what 267 01:10:18,933 --> 01:10:20,300 are called the student outcomes. 268 01:10:20,966 --> 01:10:24,566 These are skills and knowledges that that 269 01:10:24,566 --> 01:10:26,566 you expect your graduates to demonstrate 270 01:10:26,566 --> 01:10:28,966 before they graduate so that they can 271 01:10:28,966 --> 01:10:30,699 then attain those 272 01:10:30,699 --> 01:10:32,966 objectives that you've identified. 273 01:10:34,733 --> 01:10:36,366 Those, again, had to be determined and 274 01:10:36,366 --> 01:10:37,833 agreed to by all 275 01:10:37,833 --> 01:10:39,033 these different societies. 276 01:10:39,833 --> 01:10:41,600 And one of the improvements that Steve 277 01:10:41,600 --> 01:10:44,000 was referring to is actually has has 278 01:10:44,000 --> 01:10:44,966 roots in material in 279 01:10:44,966 --> 01:10:45,733 our materials community. 280 01:10:46,766 --> 01:10:49,833 The the outcomes that were set back in 281 01:10:49,833 --> 01:10:52,333 probably late 1990s had 282 01:10:52,333 --> 01:10:54,166 been around for a long time. 283 01:10:54,933 --> 01:10:57,033 And Beth Judson, who is from the American 284 01:10:57,033 --> 01:10:59,633 Ceramic Society, was then on the 285 01:10:59,633 --> 01:11:01,933 executive committee and she said, well, 286 01:11:02,566 --> 01:11:05,033 we've been we ask our programs to always 287 01:11:05,033 --> 01:11:06,533 speak improving, so we 288 01:11:06,533 --> 01:11:07,666 should be doing that, too. 289 01:11:08,133 --> 01:11:11,233 So let's re look at these outcomes and 290 01:11:11,233 --> 01:11:12,833 that she kind of started that process 291 01:11:12,833 --> 01:11:15,966 back in, I think, was about 2009, 2010. 292 01:11:16,333 --> 01:11:17,433 Sometimes in that time frame. 293 01:11:18,533 --> 01:11:21,166 And it finally took 10 years or so before 294 01:11:21,166 --> 01:11:22,533 they they were changed. 295 01:11:23,600 --> 01:11:25,633 And it took a long time to come to 296 01:11:25,633 --> 01:11:27,166 agreement as to what they should be. 297 01:11:27,866 --> 01:11:28,966 I think they were an improvement. 298 01:11:30,866 --> 01:11:32,766 So there is that trying to make it 299 01:11:32,766 --> 01:11:34,899 better, try to adapt to these changes. 300 01:11:35,433 --> 01:11:36,833 So that's the outcomes. 301 01:11:36,833 --> 01:11:38,433 And then to support all that are 302 01:11:38,433 --> 01:11:40,833 basically the things that the institution 303 01:11:40,833 --> 01:11:42,199 or the program needs to do. 304 01:11:43,033 --> 01:11:45,133 Students, what are your processes for 305 01:11:45,133 --> 01:11:47,033 advising and making sure that students 306 01:11:47,033 --> 01:11:48,666 meet all the graduation requirements? 307 01:11:49,533 --> 01:11:50,233 What is the curriculum? 308 01:11:50,433 --> 01:11:51,500 What is the content of 309 01:11:51,500 --> 01:11:52,666 the courses they take? 310 01:11:52,666 --> 01:11:54,233 What's their experiences that they need 311 01:11:54,233 --> 01:11:55,366 to go through before 312 01:11:55,366 --> 01:11:56,933 they can for they graduate? 313 01:11:57,366 --> 01:11:59,233 Are your faculty qualified? 314 01:11:59,233 --> 01:12:00,333 Do you have enough faculty? 315 01:12:00,666 --> 01:12:02,833 Are the facilities adequate to support 316 01:12:02,833 --> 01:12:04,133 those student outcomes? 317 01:12:04,699 --> 01:12:07,933 Do you have the the the institutional 318 01:12:07,933 --> 01:12:10,733 support, financial and administrative to 319 01:12:10,733 --> 01:12:11,600 support the program? 320 01:12:12,066 --> 01:12:14,066 And then on top of all this, there's that 321 01:12:14,066 --> 01:12:16,366 requirement for a continuous improvement. 322 01:12:17,033 --> 01:12:19,899 So you have to have a process by which 323 01:12:19,899 --> 01:12:22,100 you're determining are your students 324 01:12:22,100 --> 01:12:23,800 meeting the outcomes that that have been 325 01:12:23,800 --> 01:12:27,000 specified and looking at other other data 326 01:12:27,000 --> 01:12:28,800 to see how how are you doing 327 01:12:28,800 --> 01:12:30,466 and what can you do to improve? 328 01:12:31,033 --> 01:12:32,566 And you need a document that you're 329 01:12:32,566 --> 01:12:34,566 continuing that process. 330 01:12:35,633 --> 01:12:38,100 Most of the most of what the evaluators 331 01:12:38,100 --> 01:12:40,333 are looking for are related to processes. 332 01:12:41,833 --> 01:12:43,333 And processes are important. 333 01:12:43,766 --> 01:12:45,100 Sometimes it might say, well, it's kind 334 01:12:45,100 --> 01:12:47,433 of a bureaucratic thing, but processes 335 01:12:47,433 --> 01:12:49,533 are what keep things going. 336 01:12:50,066 --> 01:12:51,899 If you don't have a process established, 337 01:12:52,733 --> 01:12:55,533 then if people change, you might not keep 338 01:12:55,533 --> 01:12:56,566 doing what you're doing. 339 01:12:57,033 --> 01:12:59,233 But if there's a process that helps make 340 01:12:59,233 --> 01:13:01,266 sure that people are doing things in a 341 01:13:01,266 --> 01:13:04,733 certain way, it's being consistent and 342 01:13:04,733 --> 01:13:05,466 not letting things 343 01:13:05,466 --> 01:13:06,366 fall through the crack. 344 01:13:06,833 --> 01:13:09,133 And that's really mostly what what the 345 01:13:09,133 --> 01:13:10,666 team is looking for. 346 01:13:10,733 --> 01:13:12,766 Do you have these processes? How are they 347 01:13:12,766 --> 01:13:14,899 working? Are they documented? 348 01:13:16,133 --> 01:13:20,199 Yeah, that's well said. I think that's 349 01:13:20,199 --> 01:13:25,300 exactly captures what a bet tries to do. 350 01:13:25,600 --> 01:13:30,566 And it's always distressing to me how so 351 01:13:30,566 --> 01:13:33,333 many faculty and even many administrators 352 01:13:33,333 --> 01:13:36,733 misinterpret what a bet trying to do. 353 01:13:37,366 --> 01:13:39,166 I can't tell you how many meetings I've 354 01:13:39,166 --> 01:13:41,766 been in where someone says, oh, we can't 355 01:13:41,766 --> 01:13:42,800 teach that in our class 356 01:13:42,800 --> 01:13:44,199 because a bet won't let us. 357 01:13:44,633 --> 01:13:46,533 And it's like, no, a bet doesn't say 358 01:13:46,533 --> 01:13:49,366 anything about what we actually teach. 359 01:13:50,566 --> 01:13:54,866 There's no curricular mandates by a bet 360 01:13:54,866 --> 01:13:58,866 other than it be a engineering course or 361 01:13:58,866 --> 01:14:00,100 a math science course. 362 01:14:01,166 --> 01:14:03,733 There's 30 credits of math and science, 363 01:14:04,466 --> 01:14:07,000 45 credits of engineering. That's it. 364 01:14:07,966 --> 01:14:10,333 And there are clear definitions in the 365 01:14:10,333 --> 01:14:13,066 criteria that explain the difference 366 01:14:13,066 --> 01:14:14,699 between those two words. 367 01:14:17,300 --> 01:14:19,466 And that's it. They don't dictate the 368 01:14:19,466 --> 01:14:23,699 content, the pedagogy, the kinds of 369 01:14:23,699 --> 01:14:27,233 courses that's totally up to a program as 370 01:14:27,233 --> 01:14:28,066 long as the student 371 01:14:28,066 --> 01:14:29,633 outcomes are being assessed. 372 01:14:29,966 --> 01:14:31,433 And they don't even have to be met. 373 01:14:32,133 --> 01:14:33,733 Right. The criteria doesn't say the 374 01:14:33,733 --> 01:14:35,433 students have to meet the outcomes. 375 01:14:36,066 --> 01:14:37,766 It only says that you have to measure the 376 01:14:37,766 --> 01:14:40,699 extent to which the graduates have 377 01:14:40,699 --> 01:14:42,366 achieved those outcomes. 378 01:14:43,433 --> 01:14:47,266 And everything, the processes, everything 379 01:14:47,266 --> 01:14:49,766 is up to the programs. A bet does not 380 01:14:49,766 --> 01:14:51,166 dictate any of that. 381 01:14:51,833 --> 01:14:54,166 And I think it's all encapsulated really 382 01:14:54,166 --> 01:14:55,500 nicely with something. 383 01:14:56,366 --> 01:14:58,966 Was it Chet Van Tyne who started saying 384 01:14:58,966 --> 01:15:02,333 the phrase, improve your program for 385 01:15:02,333 --> 01:15:04,500 yourselves first and 386 01:15:04,500 --> 01:15:06,000 worry about a bet later. 387 01:15:06,966 --> 01:15:10,933 If you're a good program, meeting the 388 01:15:10,933 --> 01:15:13,633 criteria for a bet should be trivial. 389 01:15:13,633 --> 01:15:15,833 You just have to document things and have 390 01:15:15,833 --> 01:15:18,233 a process where you actually look at it. 391 01:15:18,500 --> 01:15:21,166 And the details are already being done 392 01:15:21,166 --> 01:15:22,899 because you care about your students. 393 01:15:26,866 --> 01:15:29,133 That's been my experience as well, 394 01:15:29,733 --> 01:15:31,066 tangentially working with some of these 395 01:15:31,066 --> 01:15:32,933 things is looking at the criteria, 396 01:15:32,933 --> 01:15:33,366 looking at the 397 01:15:33,366 --> 01:15:34,399 outcomes and really saying, 398 01:15:34,800 --> 01:15:36,833 this is what I should be doing all along 399 01:15:36,833 --> 01:15:39,233 anyway, right? You just want to make sure 400 01:15:39,233 --> 01:15:40,633 that I'm actually doing a good job. 401 01:15:40,966 --> 01:15:45,300 It doesn't really seem so onerous if, as 402 01:15:45,300 --> 01:15:46,766 you said, Steve, if you're trying to make 403 01:15:46,766 --> 01:15:48,066 your program better for 404 01:15:48,066 --> 01:15:49,233 yourself and for your students, 405 01:15:50,266 --> 01:15:53,533 the A-bet process will happen naturally 406 01:15:53,533 --> 01:15:55,233 as a consequence of that. 407 01:15:55,899 --> 01:15:58,266 Yeah, and there's an analogy for that. 408 01:15:59,366 --> 01:16:01,100 You know, if you give a question on a 409 01:16:01,100 --> 01:16:03,800 test that requires maybe a sentence or 410 01:16:03,800 --> 01:16:06,266 two to answer and the students who know 411 01:16:06,266 --> 01:16:07,766 the answer will answer 412 01:16:07,766 --> 01:16:08,866 it in a sentence or two. 413 01:16:09,233 --> 01:16:11,066 And those that don't will fill every 414 01:16:11,066 --> 01:16:13,966 little spot on the paper with words 415 01:16:13,966 --> 01:16:16,466 hoping that you will find the answer in 416 01:16:16,466 --> 01:16:17,233 what they've written. 417 01:16:17,800 --> 01:16:19,033 And it's kind of the same thing you'll 418 01:16:19,033 --> 01:16:21,100 see with a program that doesn't really 419 01:16:21,100 --> 01:16:22,300 know what's required. 420 01:16:22,300 --> 01:16:23,699 They just do a lot of stuff 421 01:16:23,699 --> 01:16:26,366 hoping it's the right thing. 422 01:16:26,933 --> 01:16:28,666 And if you understand what is really 423 01:16:28,666 --> 01:16:30,933 required, as Steve said, it's really not 424 01:16:30,933 --> 01:16:33,533 that hard to meet the criteria. 425 01:16:34,566 --> 01:16:35,866 You have to document some things, you 426 01:16:35,866 --> 01:16:36,699 have to do some things. 427 01:16:37,500 --> 01:16:39,266 But if you're just interested in getting 428 01:16:39,266 --> 01:16:41,133 your making your program better, it's not 429 01:16:41,133 --> 01:16:43,533 that much work if you 430 01:16:43,533 --> 01:16:45,033 know what work is required. 431 01:16:45,633 --> 01:16:48,600 Part of this, maybe you can you probably 432 01:16:48,600 --> 01:16:50,666 know the history better than I do, but 433 01:16:50,666 --> 01:16:55,500 EC2000, when A-bet started the outcomes 434 01:16:55,500 --> 01:16:58,899 assessment based accreditation, which was 435 01:16:58,899 --> 01:17:00,566 a big shift for them at that time. 436 01:17:01,433 --> 01:17:04,566 I believe that was born out of ISO 9000. 437 01:17:05,333 --> 01:17:06,033 Is that correct? 438 01:17:10,066 --> 01:17:12,000 I it sounds it sounds 439 01:17:12,000 --> 01:17:12,933 like it would be to me. 440 01:17:12,933 --> 01:17:14,166 I don't I wasn't around at 441 01:17:14,166 --> 01:17:15,366 that time, so I'm not sure. 442 01:17:15,366 --> 01:17:17,666 But so I think that's likely you haven't 443 01:17:17,666 --> 01:17:19,600 said this, but industry is pretty 444 01:17:19,600 --> 01:17:21,966 involved with A-bet as well. 445 01:17:22,166 --> 01:17:25,366 And I because I first got in when our 446 01:17:25,366 --> 01:17:28,766 program was going to be evaluated in 447 01:17:28,766 --> 01:17:32,699 1999, I think under the EC2000. 448 01:17:34,433 --> 01:17:37,033 That was the Engineering Commission 2000 449 01:17:37,033 --> 01:17:38,966 project has changed 450 01:17:38,966 --> 01:17:40,833 the way A-bet did things. 451 01:17:41,133 --> 01:17:43,133 We agreed to do it then. 452 01:17:43,833 --> 01:17:46,133 And lucky me, I got sucked into being the 453 01:17:46,133 --> 01:17:47,933 one who had to do the self study. 454 01:17:49,033 --> 01:17:50,366 And so that was my initiation. 455 01:17:51,133 --> 01:17:53,533 And, you know, it was explained to us 456 01:17:53,533 --> 01:17:56,666 that in factories, people always make 457 01:17:56,666 --> 01:17:58,533 measurements to make sure the product 458 01:17:58,533 --> 01:17:59,733 that comes out at the end of 459 01:17:59,733 --> 01:18:01,199 the line is a quality product. 460 01:18:01,899 --> 01:18:03,399 And so they had all 461 01:18:03,399 --> 01:18:04,800 sorts of metrics involved. 462 01:18:05,433 --> 01:18:07,899 And this was pretty much done 463 01:18:07,899 --> 01:18:10,566 universally in all industry. 464 01:18:11,266 --> 01:18:12,600 So they were saying, why don't 465 01:18:12,600 --> 01:18:14,133 you do the same for education? 466 01:18:14,866 --> 01:18:16,666 And while I think that's a great idea and 467 01:18:16,666 --> 01:18:19,633 I think it's it's come a long way, I 468 01:18:19,633 --> 01:18:22,733 really do have to always say measuring a 469 01:18:22,733 --> 01:18:26,333 part in a factory is a lot easier than 470 01:18:26,333 --> 01:18:28,433 measuring student outcomes. 471 01:18:29,633 --> 01:18:31,100 And I think that there has 472 01:18:31,100 --> 01:18:34,833 to be some give by industry. 473 01:18:34,833 --> 01:18:36,699 And I saw this on teams where the 474 01:18:36,699 --> 01:18:40,066 industrial evaluators didn't quite get 475 01:18:40,066 --> 01:18:42,666 that and they were always much pickier 476 01:18:42,666 --> 01:18:44,866 than the academic evaluators. 477 01:18:48,633 --> 01:18:50,533 Yeah, sometimes although they also they 478 01:18:50,533 --> 01:18:53,433 get the continuous improvement part more 479 01:18:53,433 --> 01:18:55,866 easily than some of the academics, I 480 01:18:55,866 --> 01:18:56,933 mean, they get the idea 481 01:18:56,933 --> 01:18:58,333 that, oh, yes, you do need to. 482 01:18:58,866 --> 01:19:01,066 It's about the improvement as opposed to 483 01:19:01,066 --> 01:19:02,733 sometimes, at least in the beginning 484 01:19:02,733 --> 01:19:03,733 stages, I guess, I think 485 01:19:03,733 --> 01:19:04,766 we've gotten better at it. 486 01:19:05,100 --> 01:19:07,833 But in the in the beginnings, we as 487 01:19:07,833 --> 01:19:10,500 professors didn't quite get it. 488 01:19:11,066 --> 01:19:12,500 I think it took us a little while to 489 01:19:12,500 --> 01:19:14,533 understand that we have we are assessing. 490 01:19:14,533 --> 01:19:17,666 It's just where we give tests, we know we 491 01:19:17,666 --> 01:19:18,666 give homeworks, we have 492 01:19:18,666 --> 01:19:20,133 them do things to assess. 493 01:19:20,833 --> 01:19:22,233 We don't think of it that way. 494 01:19:22,233 --> 01:19:23,333 So I think it would be useful. 495 01:19:23,633 --> 01:19:26,366 Maybe you can tell us go through the 496 01:19:26,366 --> 01:19:29,433 whole A-bet process, because a lot of 497 01:19:29,433 --> 01:19:32,500 programs out there, you know, it's one of 498 01:19:32,500 --> 01:19:34,033 these things I've been doing it at my 499 01:19:34,033 --> 01:19:36,366 school for a long time and helping the 500 01:19:36,366 --> 01:19:38,333 dean much as like you've done. 501 01:19:39,166 --> 01:19:41,399 And what's remarkable to me is that every 502 01:19:41,399 --> 01:19:45,000 six years, the chairs are all completely 503 01:19:45,000 --> 01:19:46,733 different and none of 504 01:19:46,733 --> 01:19:47,466 them have experience. 505 01:19:48,166 --> 01:19:50,333 And so it's reinventing the wheel every 506 01:19:50,333 --> 01:19:52,399 single time you do it, which is why 507 01:19:52,399 --> 01:19:54,533 process is so important. 508 01:19:56,666 --> 01:19:59,633 So, in case there are programs out there 509 01:19:59,633 --> 01:20:01,366 that are listening to this to try to 510 01:20:01,366 --> 01:20:02,333 figure out, oh, my 511 01:20:02,333 --> 01:20:03,566 God, we have to do a bit. 512 01:20:03,766 --> 01:20:05,000 I have no idea how to do it. 513 01:20:05,800 --> 01:20:08,433 Can you go through how the whole process 514 01:20:08,433 --> 01:20:10,866 begins every six years? 515 01:20:12,033 --> 01:20:14,366 And so people understand that. 516 01:20:16,566 --> 01:20:16,733 Sure. 517 01:20:17,633 --> 01:20:19,300 So A-bet comes at the 518 01:20:19,300 --> 01:20:21,033 invitation of the institution. 519 01:20:21,766 --> 01:20:24,233 No program has to be accredited by A-bet. 520 01:20:24,800 --> 01:20:25,866 And so it's an invitation. 521 01:20:26,600 --> 01:20:29,000 And that invitation is done through 522 01:20:29,000 --> 01:20:30,866 what's called a request for evaluation. 523 01:20:31,733 --> 01:20:34,166 That's done in January of the year 524 01:20:34,166 --> 01:20:36,033 preceding when your 525 01:20:36,033 --> 01:20:38,733 onsite evaluation will be. 526 01:20:40,033 --> 01:20:43,033 So if you were to be evaluated, say it's 527 01:20:43,033 --> 01:20:46,266 too late for fall 2024, so you could be 528 01:20:46,266 --> 01:20:47,933 evaluated in fall 2025. 529 01:20:48,399 --> 01:20:50,566 In that case, in January of 2025, you 530 01:20:50,566 --> 01:20:52,333 would submit a request for evaluation, 531 01:20:52,333 --> 01:20:54,566 which will just list these other programs 532 01:20:54,566 --> 01:20:55,833 you want to have evaluated. 533 01:20:56,366 --> 01:20:58,500 You provide one example transcript 534 01:20:58,500 --> 01:21:00,933 because they need to see how the degree 535 01:21:00,933 --> 01:21:04,033 is described on the transcript to make 536 01:21:04,033 --> 01:21:05,566 sure the name is correct 537 01:21:05,566 --> 01:21:07,066 and consistent and so forth. 538 01:21:08,433 --> 01:21:10,533 So then you just wait. 539 01:21:10,866 --> 01:21:14,233 And the next step is the team chair is 540 01:21:14,233 --> 01:21:16,333 assigned to the evaluations. 541 01:21:17,366 --> 01:21:19,266 That occurs typically around this time. 542 01:21:20,033 --> 01:21:21,899 So they're in the process right now of 543 01:21:21,899 --> 01:21:23,366 probably assigning team chairs. 544 01:21:24,166 --> 01:21:27,366 The team chairs are commissioners, either 545 01:21:27,366 --> 01:21:30,300 active commissioners or what 546 01:21:30,300 --> 01:21:31,866 we call the team chair pool, 547 01:21:31,933 --> 01:21:33,966 which are commissioners who have been off 548 01:21:33,966 --> 01:21:36,366 the commission for a few years and are 549 01:21:36,366 --> 01:21:37,566 still familiar enough that 550 01:21:37,566 --> 01:21:39,166 they can still lead a team. 551 01:21:40,466 --> 01:21:43,199 And so they're assigned typically in late 552 01:21:43,199 --> 01:21:45,699 April, May, that time frame. 553 01:21:46,566 --> 01:21:48,566 Once the team chair is assigned, then the 554 01:21:48,566 --> 01:21:50,166 evaluation date is set. 555 01:21:50,866 --> 01:21:53,199 And actually, for the last several years, 556 01:21:53,199 --> 01:21:54,966 when you submit your request for 557 01:21:54,966 --> 01:21:56,233 evaluation, you can at 558 01:21:56,233 --> 01:21:57,766 that time request a date. 559 01:21:58,433 --> 01:22:00,166 You can say this is the date when we want 560 01:22:00,166 --> 01:22:01,233 to have our evaluation. 561 01:22:02,033 --> 01:22:02,933 And in most cases, 562 01:22:03,233 --> 01:22:05,333 that that is accomplished. 563 01:22:05,666 --> 01:22:06,933 Some cases it may not work 564 01:22:06,933 --> 01:22:08,033 out, but usually it does. 565 01:22:09,033 --> 01:22:11,066 So once the team chair is established, 566 01:22:11,266 --> 01:22:13,766 they confirm the date, the date set, then 567 01:22:13,766 --> 01:22:16,466 the rest of the team is is assigned. 568 01:22:17,633 --> 01:22:19,666 The program evaluators are assigned by 569 01:22:19,666 --> 01:22:20,966 the professional societies. 570 01:22:21,966 --> 01:22:24,699 So in the case of materials programs, the 571 01:22:24,699 --> 01:22:27,433 program evaluators are assigned by TMS. 572 01:22:29,133 --> 01:22:30,699 The only exception would be ceramics 573 01:22:30,699 --> 01:22:32,233 programs are assigned by 574 01:22:32,233 --> 01:22:33,766 the American Ceramics Society. 575 01:22:35,466 --> 01:22:37,333 And all the different disciplines will 576 01:22:37,333 --> 01:22:38,566 assign teams to all 577 01:22:38,566 --> 01:22:39,533 the different programs. 578 01:22:40,966 --> 01:22:43,633 So that's what's that's happening in 579 01:22:43,633 --> 01:22:44,766 terms of preparing for the visit. 580 01:22:45,500 --> 01:22:48,533 Now, in the meantime, the program needs 581 01:22:48,533 --> 01:22:51,466 to provide a prepare a self-study report. 582 01:22:52,333 --> 01:22:55,600 That's due in July, July 1st of the 583 01:22:55,600 --> 01:22:57,933 summer before the visit. 584 01:22:59,333 --> 01:23:00,966 Now, you would have already been should 585 01:23:00,966 --> 01:23:02,566 have already has been working on that for 586 01:23:02,566 --> 01:23:06,500 a while when we prepared for our so we 587 01:23:06,500 --> 01:23:08,966 were visited about a year before last. 588 01:23:10,833 --> 01:23:14,166 We started essentially really preparing 589 01:23:14,166 --> 01:23:16,500 about a year before 590 01:23:16,500 --> 01:23:17,966 the self-study is due. 591 01:23:18,766 --> 01:23:21,233 Basically, the after that previous 592 01:23:21,233 --> 01:23:23,399 academic year, you have that academic 593 01:23:23,399 --> 01:23:26,833 year completed, your processes, most of 594 01:23:26,833 --> 01:23:28,133 the self-study report isn't going to 595 01:23:28,133 --> 01:23:29,699 change very much at that point. 596 01:23:29,699 --> 01:23:31,266 So you can really write the 597 01:23:31,266 --> 01:23:32,766 vast majority of the report. 598 01:23:34,133 --> 01:23:35,666 I coordinated that so far. 599 01:23:35,733 --> 01:23:37,366 The last two visits, I coordinated that 600 01:23:37,366 --> 01:23:40,566 for our college and I asked for drafts by 601 01:23:40,566 --> 01:23:43,466 actually January, even 602 01:23:43,466 --> 01:23:45,033 though it's not due to July. 603 01:23:45,899 --> 01:23:47,866 And my argument was, well, if you submit 604 01:23:47,866 --> 01:23:50,000 it in January, you have a draft and we 605 01:23:50,000 --> 01:23:52,733 find an issue, you got the spring to fix 606 01:23:52,733 --> 01:23:55,233 it and turn it from a problem to an 607 01:23:55,233 --> 01:23:56,833 example of a continuous improvement. 608 01:23:58,233 --> 01:24:01,266 And some of the programs, well, nobody 609 01:24:01,266 --> 01:24:02,166 disagreed with that. 610 01:24:02,233 --> 01:24:03,833 Some of them complied, some of them did 611 01:24:03,833 --> 01:24:07,000 not, but that's that's their choice. 612 01:24:08,699 --> 01:24:11,266 So the preparation is take some time to 613 01:24:11,266 --> 01:24:13,566 prepare that document on that document 614 01:24:13,566 --> 01:24:15,966 basically describes how you comply with 615 01:24:15,966 --> 01:24:17,199 all the different criteria. 616 01:24:18,366 --> 01:24:21,133 There are some kind of bookkeeping type 617 01:24:21,133 --> 01:24:23,166 things that collecting resumes and 618 01:24:23,166 --> 01:24:25,166 collecting syllabi, which 619 01:24:25,166 --> 01:24:26,466 could be a bit time consuming. 620 01:24:26,500 --> 01:24:29,100 So starting on that early is a good way 621 01:24:29,100 --> 01:24:32,566 to not have it all concentrated in June. 622 01:24:33,866 --> 01:24:36,366 So now we have the team is set in July 623 01:24:36,366 --> 01:24:38,500 and the self-study report submitted. 624 01:24:39,733 --> 01:24:43,899 And then that team starts to work with 625 01:24:43,899 --> 01:24:45,633 the institution to prepare for the visit. 626 01:24:46,300 --> 01:24:48,533 The visits occur typically in the fall 627 01:24:48,533 --> 01:24:51,300 from usually the earliest or maybe the 628 01:24:51,300 --> 01:24:52,233 week after Labor Day. 629 01:24:52,899 --> 01:24:56,199 And they go until most are the highest 630 01:24:56,199 --> 01:24:57,633 concentrations October. 631 01:24:58,466 --> 01:24:59,633 They go into November or 632 01:24:59,633 --> 01:25:00,633 maybe a few in December. 633 01:25:02,733 --> 01:25:05,500 And then the team can even before the 634 01:25:05,500 --> 01:25:08,233 visit, the team is encouraged to interact 635 01:25:08,233 --> 01:25:09,000 with the institution. 636 01:25:10,399 --> 01:25:12,466 So the program evaluator will read the 637 01:25:12,466 --> 01:25:14,300 self-study might have some questions. 638 01:25:15,766 --> 01:25:17,866 If the PV doesn't have questions, it 639 01:25:17,866 --> 01:25:18,566 probably means they're 640 01:25:18,566 --> 01:25:19,633 not really doing their job. 641 01:25:20,600 --> 01:25:22,333 There's always something that's not 642 01:25:22,333 --> 01:25:23,966 clear, some additional information. 643 01:25:24,433 --> 01:25:25,666 So if you get questions, 644 01:25:25,666 --> 01:25:27,300 don't worry about that. 645 01:25:27,300 --> 01:25:28,533 In fact, I would think that's a good 646 01:25:28,533 --> 01:25:30,166 thing that you're getting. 647 01:25:30,566 --> 01:25:32,566 You're finding out what issues that might 648 01:25:32,566 --> 01:25:34,933 be there and you have a chance to address 649 01:25:34,933 --> 01:25:37,333 it even before the visit in many cases. 650 01:25:37,899 --> 01:25:39,766 Or if not, at least be prepared to 651 01:25:39,766 --> 01:25:42,366 provide the evidence on site that you 652 01:25:42,366 --> 01:25:44,566 might need to demonstrate compliance. 653 01:25:45,600 --> 01:25:47,533 So that all occurs up until the visit. 654 01:25:48,533 --> 01:25:51,533 The typical visits are the kind of 655 01:25:51,533 --> 01:25:54,466 standard is starts on a Sunday. 656 01:25:55,199 --> 01:25:57,600 Sunday morning, the team meets, comes on 657 01:25:57,600 --> 01:25:59,933 campus in the afternoon, looks at 658 01:25:59,933 --> 01:26:03,199 facilities, reviews, materials, 659 01:26:03,199 --> 01:26:06,433 which are course materials and assessment 660 01:26:06,433 --> 01:26:07,666 materials you've collected. 661 01:26:08,399 --> 01:26:09,666 That's changing now. 662 01:26:09,666 --> 01:26:10,633 That's a lot of that 663 01:26:10,633 --> 01:26:12,500 might be available in advance. 664 01:26:13,366 --> 01:26:15,433 So there's some discussion of how that 665 01:26:15,433 --> 01:26:16,899 Monday might look a little bit different. 666 01:26:17,566 --> 01:26:19,266 If you've already evaluated some of those 667 01:26:19,266 --> 01:26:20,899 materials, that's the 668 01:26:20,899 --> 01:26:22,733 traditional purpose of that Sunday. 669 01:26:24,533 --> 01:26:26,033 And then most of the 670 01:26:26,033 --> 01:26:27,466 meetings are on Monday. 671 01:26:27,800 --> 01:26:29,133 So Monday is just a day full 672 01:26:29,133 --> 01:26:31,033 of meetings from the program. 673 01:26:31,600 --> 01:26:34,399 So each the team consists of a team chair 674 01:26:34,399 --> 01:26:36,533 and then program evaluators where there's 675 01:26:36,533 --> 01:26:38,666 one program evaluator for each program. 676 01:26:39,966 --> 01:26:43,666 Exceptions might be if a program has a 677 01:26:43,666 --> 01:26:46,766 name that has two that 678 01:26:46,766 --> 01:26:49,266 requires two program criteria. 679 01:26:49,266 --> 01:26:50,533 And one thing I forgot to mention when I 680 01:26:50,533 --> 01:26:52,500 was talking about criteria in a division 681 01:26:52,500 --> 01:26:53,466 in addition to those 682 01:26:53,466 --> 01:26:54,633 eight general criteria, 683 01:26:55,366 --> 01:26:57,133 there are what are called program 684 01:26:57,133 --> 01:27:00,466 criteria and those are specific to the 685 01:27:00,466 --> 01:27:01,233 different disciplines. 686 01:27:01,899 --> 01:27:03,966 So materials engineering and material 687 01:27:03,966 --> 01:27:06,266 science and engineering has one set of 688 01:27:06,266 --> 01:27:09,066 program criteria which are different from 689 01:27:09,066 --> 01:27:10,733 aerospace engineering or 690 01:27:10,733 --> 01:27:11,633 mechanical engineering. 691 01:27:12,233 --> 01:27:13,899 And those program criteria describe 692 01:27:13,899 --> 01:27:17,366 either curricular topics or faculty 693 01:27:17,366 --> 01:27:19,833 qualifications that are needed for that 694 01:27:19,833 --> 01:27:22,066 discipline that are necessarily needed 695 01:27:22,066 --> 01:27:24,533 for a general any engineer program. 696 01:27:26,766 --> 01:27:30,433 So you're preparing so that each program 697 01:27:30,433 --> 01:27:32,199 has one or maybe two if 698 01:27:32,199 --> 01:27:33,366 they have a name like that. 699 01:27:33,800 --> 01:27:37,000 Sometimes there are other members of the 700 01:27:37,000 --> 01:27:39,600 team like observers who are part of the 701 01:27:39,600 --> 01:27:41,899 training are just tagging along and 702 01:27:41,899 --> 01:27:44,666 observing or from state boards or various 703 01:27:44,666 --> 01:27:46,633 types of observers that can be part of 704 01:27:46,633 --> 01:27:47,733 the team but aren't part of the 705 01:27:47,733 --> 01:27:48,666 evaluation decision. 706 01:27:50,199 --> 01:27:52,166 So that you'll meet with your program 707 01:27:52,166 --> 01:27:54,100 evaluator will want to meet with the 708 01:27:54,100 --> 01:27:54,933 chair will want to be 709 01:27:54,933 --> 01:27:57,066 with faculty with students. 710 01:27:58,233 --> 01:27:59,833 And so you'll work with the program 711 01:27:59,833 --> 01:28:00,833 evaluators schedule 712 01:28:00,833 --> 01:28:02,733 that that day of meetings. 713 01:28:03,666 --> 01:28:05,966 There's typically a luncheon where the. 714 01:28:08,100 --> 01:28:11,899 That you're allowed to host the team and 715 01:28:11,899 --> 01:28:14,233 typically use that to have some alumni or 716 01:28:14,233 --> 01:28:14,966 maybe some star 717 01:28:14,966 --> 01:28:16,699 students to meet with this. 718 01:28:16,733 --> 01:28:19,699 Be with the PV. You always want to get 719 01:28:19,699 --> 01:28:21,466 your best students to have lunch with the 720 01:28:21,466 --> 01:28:23,266 PV because that gives a very good 721 01:28:23,266 --> 01:28:24,533 impression that you have these very 722 01:28:24,533 --> 01:28:27,833 bright students and successful alumni. 723 01:28:29,266 --> 01:28:31,633 And then the visit goes till Tuesday. So 724 01:28:31,633 --> 01:28:33,600 on Tuesday the morning there's typically 725 01:28:33,600 --> 01:28:35,533 not any meetings it's meant for 726 01:28:35,533 --> 01:28:39,366 preparation of the report and for any 727 01:28:39,366 --> 01:28:41,066 pickup meetings that might be needed. 728 01:28:41,600 --> 01:28:44,333 So the team prepares a report a Sunday or 729 01:28:44,333 --> 01:28:45,133 Tuesday afternoon 730 01:28:45,133 --> 01:28:46,333 they'll read that report. 731 01:28:47,600 --> 01:28:50,166 First they'll debrief the the program 732 01:28:50,166 --> 01:28:52,633 chair just to talk through it and then 733 01:28:52,633 --> 01:28:53,733 they have a formal reading 734 01:28:53,733 --> 01:28:56,566 for with the president provost. 735 01:28:56,566 --> 01:28:58,533 Really whoever the institution wants to 736 01:28:58,533 --> 01:29:01,066 have their anything from just the 737 01:29:01,066 --> 01:29:03,466 president provost and the team to 738 01:29:03,466 --> 01:29:05,733 inviting all the faculty and that had 739 01:29:05,733 --> 01:29:06,666 everything in between. 740 01:29:08,666 --> 01:29:10,633 And so at the end of the visit the team 741 01:29:10,633 --> 01:29:12,733 will read this report. This is what's 742 01:29:12,733 --> 01:29:14,100 called the draft statement. 743 01:29:15,766 --> 01:29:18,699 And you're not given a written draft 744 01:29:18,699 --> 01:29:20,633 statement which you are given is a 745 01:29:20,633 --> 01:29:24,199 written program audit form and that 746 01:29:24,199 --> 01:29:26,433 program audit form will have a 747 01:29:26,433 --> 01:29:27,500 description of any 748 01:29:27,500 --> 01:29:29,833 shortcomings that were identified. 749 01:29:30,566 --> 01:29:32,066 So you have a written version of those 750 01:29:32,066 --> 01:29:37,433 shortcomings to see and work on if there 751 01:29:37,433 --> 01:29:39,399 are issues to be addressed and you can 752 01:29:39,399 --> 01:29:41,033 immediately start working on those. You 753 01:29:41,033 --> 01:29:41,666 don't have to wait for 754 01:29:41,666 --> 01:29:42,566 the rest of the process. 755 01:29:43,699 --> 01:29:46,133 So the rest of the process is first you 756 01:29:46,133 --> 01:29:47,733 have seven days to say if there's any 757 01:29:47,733 --> 01:29:50,033 mistakes in that what was read. 758 01:29:51,533 --> 01:29:53,733 But that's really just errors of fact. 759 01:29:53,733 --> 01:29:56,333 It's not we disagree with this or we've 760 01:29:56,333 --> 01:29:57,800 done this additional work. 761 01:29:57,800 --> 01:30:00,033 It's just that was wrong. 762 01:30:00,066 --> 01:30:01,966 You said we have 15 faculty. We really 763 01:30:01,966 --> 01:30:04,966 have 16 or something like that. And then 764 01:30:04,966 --> 01:30:07,033 it goes through an editing process. And 765 01:30:07,033 --> 01:30:09,166 this editing is not just language. 766 01:30:09,466 --> 01:30:12,133 It's it's really about content and 767 01:30:12,133 --> 01:30:14,300 interpretation and judgment. 768 01:30:15,433 --> 01:30:17,100 So the first level it goes to. 769 01:30:17,100 --> 01:30:18,833 And there's two letters two levels of 770 01:30:18,833 --> 01:30:21,766 editor editors editors one editors two 771 01:30:21,766 --> 01:30:25,033 editors one are members of the executive 772 01:30:25,033 --> 01:30:26,033 committee of the commission. 773 01:30:27,033 --> 01:30:31,033 I think EAC now has about 20 of those 20 774 01:30:31,033 --> 01:30:32,500 21 maybe something like that. 775 01:30:33,533 --> 01:30:36,866 So all the AC will probably be evaluating 776 01:30:36,866 --> 01:30:39,333 seven 800 different programs. 777 01:30:40,133 --> 01:30:41,966 So it's a lot. So those are 778 01:30:41,966 --> 01:30:44,066 divided among those 20 people. 779 01:30:45,833 --> 01:30:48,566 So they'll read it and come back. 780 01:30:48,566 --> 01:30:49,766 They may have some questions. 781 01:30:50,566 --> 01:30:52,033 But why was this done this way. 782 01:30:52,699 --> 01:30:53,933 What did you really see here. 783 01:30:54,366 --> 01:30:55,199 You know this doesn't look 784 01:30:55,199 --> 01:30:57,366 like what we typically do. 785 01:30:57,366 --> 01:30:58,500 So what they're really looking for is 786 01:30:58,500 --> 01:30:59,933 trying to make everything consistent 787 01:30:59,933 --> 01:31:02,466 among all the different teams that are 788 01:31:02,466 --> 01:31:04,966 going out because those couple hundred 789 01:31:04,966 --> 01:31:06,366 teams are going to all different 790 01:31:06,366 --> 01:31:08,033 institutions with different people. 791 01:31:08,466 --> 01:31:10,566 Hundreds of different people all trying 792 01:31:10,566 --> 01:31:12,133 to make a consistent decision. 793 01:31:12,133 --> 01:31:13,933 And that's a that's a challenge. 794 01:31:14,600 --> 01:31:16,233 We have training to try to do that. 795 01:31:16,233 --> 01:31:17,399 But training is not 796 01:31:17,399 --> 01:31:18,100 going to cover it all. 797 01:31:18,133 --> 01:31:20,500 So you have this process to have checks 798 01:31:20,500 --> 01:31:22,333 of people who have seen more have more 799 01:31:22,333 --> 01:31:24,733 experience and can help them to make 800 01:31:24,733 --> 01:31:27,233 things similar at different institutions. 801 01:31:27,833 --> 01:31:28,833 Then the second level of 802 01:31:28,833 --> 01:31:30,233 editing are the officers. 803 01:31:30,800 --> 01:31:31,966 And now in the AC its 804 01:31:31,966 --> 01:31:33,566 officers plus one or two more. 805 01:31:34,433 --> 01:31:36,500 And they're looking at even more programs 806 01:31:36,500 --> 01:31:39,133 and they're they have more experience and 807 01:31:39,133 --> 01:31:40,666 they may go back and forth with some 808 01:31:40,666 --> 01:31:42,166 discussion of what this should be. 809 01:31:42,866 --> 01:31:44,966 And then finally it goes to what are 810 01:31:44,966 --> 01:31:46,866 called the adjuncts and the adjuncts are 811 01:31:46,866 --> 01:31:48,233 part time a bet staff. 812 01:31:49,066 --> 01:31:50,533 AC has four of them. 813 01:31:51,100 --> 01:31:53,033 All four of those have been chairs of the 814 01:31:53,033 --> 01:31:55,399 commission and now they work part time 815 01:31:55,399 --> 01:31:57,966 for a bet that they don't make the 816 01:31:57,966 --> 01:31:59,866 decisions but they provide guidance. 817 01:32:00,233 --> 01:32:01,800 And because they have even more 818 01:32:01,800 --> 01:32:03,333 experience and we'll ask 819 01:32:03,333 --> 01:32:04,933 questions and go back and forth. 820 01:32:05,633 --> 01:32:07,266 So it goes through all of this process 821 01:32:07,266 --> 01:32:08,899 before you finally have 822 01:32:08,899 --> 01:32:10,233 what's called the draft statement. 823 01:32:11,633 --> 01:32:13,266 This takes a couple of months typically. 824 01:32:14,433 --> 01:32:16,733 That draft statement will may look a 825 01:32:16,733 --> 01:32:18,133 little different than what was read. 826 01:32:18,766 --> 01:32:19,433 Some of the shortcomings 827 01:32:19,433 --> 01:32:20,966 might have changed in level. 828 01:32:20,966 --> 01:32:22,066 Some might have gone away. 829 01:32:23,533 --> 01:32:26,433 So that's based on the judgment and what 830 01:32:26,433 --> 01:32:27,766 what what's typically done 831 01:32:27,766 --> 01:32:28,866 at different institutions. 832 01:32:29,899 --> 01:32:30,933 Then that draft statement 833 01:32:30,933 --> 01:32:32,333 comes back to the institution. 834 01:32:33,033 --> 01:32:33,966 And at that point the 835 01:32:33,966 --> 01:32:36,733 program has 30 days to respond. 836 01:32:37,166 --> 01:32:37,866 What's called 30 day 837 01:32:37,866 --> 01:32:39,633 response at that point. 838 01:32:39,699 --> 01:32:42,166 Then you can provide well we have changed 839 01:32:42,166 --> 01:32:45,433 this in this way or provide a different 840 01:32:45,433 --> 01:32:46,566 additional information. 841 01:32:47,600 --> 01:32:49,866 Talk about what you're planning to do to 842 01:32:49,866 --> 01:32:51,466 address a shortcoming because sometimes 843 01:32:51,466 --> 01:32:53,066 maybe it's not ready yet. 844 01:32:53,466 --> 01:32:54,233 But you still have to 845 01:32:54,233 --> 01:32:55,766 respond within 30 days. 846 01:32:56,733 --> 01:32:58,766 Now if there is information that is 847 01:32:58,766 --> 01:33:02,766 needed afterwards you can request to 848 01:33:02,766 --> 01:33:06,033 submit what's called post 30 days post 849 01:33:06,033 --> 01:33:07,166 post 30 day information. 850 01:33:08,666 --> 01:33:11,333 And that can typically be provided until 851 01:33:11,333 --> 01:33:13,866 about May 15th or May 30th. 852 01:33:14,133 --> 01:33:15,366 And it's at the discretion of the team 853 01:33:15,366 --> 01:33:17,333 chair but any team chair is going to 854 01:33:17,333 --> 01:33:18,266 accept anything that's 855 01:33:18,266 --> 01:33:20,266 reasonable in a reasonable time. 856 01:33:21,466 --> 01:33:22,699 So there'll be an agreement that we're 857 01:33:22,699 --> 01:33:24,933 going to provide this information by May 858 01:33:24,933 --> 01:33:27,433 15th or whatever the agreed upon day is. 859 01:33:27,733 --> 01:33:29,266 That would be things like say you're 860 01:33:29,266 --> 01:33:31,566 changing your assessment processes and 861 01:33:31,566 --> 01:33:32,566 you want some assessment 862 01:33:32,566 --> 01:33:34,566 data from the spring semester. 863 01:33:35,233 --> 01:33:37,066 So obviously you may not have that when 864 01:33:37,066 --> 01:33:38,566 you have to provide your response. 865 01:33:39,199 --> 01:33:40,633 That's a perfectly legitimate 866 01:33:40,633 --> 01:33:43,566 request or capstone projects. 867 01:33:43,933 --> 01:33:45,533 We were missing some aspect of the 868 01:33:45,533 --> 01:33:46,366 capstone experience. 869 01:33:46,633 --> 01:33:47,566 So we need to provide 870 01:33:47,566 --> 01:33:48,933 this is what we've done. 871 01:33:48,933 --> 01:33:50,066 This is how we change the course. 872 01:33:50,733 --> 01:33:53,733 And in May we will provide the actual 873 01:33:53,733 --> 01:33:55,566 report so you can see that what we did 874 01:33:55,566 --> 01:33:57,233 was actually accomplished. 875 01:33:58,266 --> 01:33:59,766 Then once that's that that goes through 876 01:33:59,766 --> 01:34:02,033 that whole same editing process again. 877 01:34:03,100 --> 01:34:04,600 So the response is evaluated 878 01:34:04,600 --> 01:34:06,266 just like the initial report. 879 01:34:07,699 --> 01:34:08,366 And that's the report 880 01:34:08,366 --> 01:34:09,533 that goes to the commission. 881 01:34:10,366 --> 01:34:12,633 The commission meets in July typically 882 01:34:12,633 --> 01:34:14,266 about second third week in July. 883 01:34:15,233 --> 01:34:19,166 So they have this all these reports and 884 01:34:19,166 --> 01:34:21,566 they're they're going to make the vote on 885 01:34:21,566 --> 01:34:23,933 these whether they whatever that 886 01:34:23,933 --> 01:34:25,633 recommendation is for accreditation. 887 01:34:27,333 --> 01:34:29,466 The in addition to all these other checks 888 01:34:29,466 --> 01:34:30,699 there's a what's called 889 01:34:30,699 --> 01:34:31,866 a consistency committee. 890 01:34:32,733 --> 01:34:34,466 So that's a committee that's assigned. 891 01:34:34,466 --> 01:34:36,866 They look at all of the reports and try 892 01:34:36,866 --> 01:34:37,966 to look for anything that 893 01:34:37,966 --> 01:34:39,233 might not be consistent. 894 01:34:39,966 --> 01:34:41,866 Like there's all these two situations 895 01:34:41,866 --> 01:34:42,833 look kind of similar. 896 01:34:42,833 --> 01:34:43,533 This one's a weakness. 897 01:34:43,766 --> 01:34:44,466 This is a concern. 898 01:34:45,199 --> 01:34:46,233 And they'll ask the 899 01:34:46,233 --> 01:34:47,833 commission to take a look at it. 900 01:34:48,333 --> 01:34:50,399 The commission evaluates these in panels 901 01:34:50,399 --> 01:34:52,433 so that they're divided into smaller 902 01:34:52,433 --> 01:34:54,466 groups so that each group has about 903 01:34:54,466 --> 01:34:57,333 probably about 10 or 12. 904 01:34:59,600 --> 01:35:00,733 Institutions to look at 905 01:35:00,733 --> 01:35:02,533 which might be 40 or 50 programs. 906 01:35:03,233 --> 01:35:05,000 And so looking at a smaller subset of 907 01:35:05,000 --> 01:35:08,233 this these hundreds of programs and 908 01:35:08,233 --> 01:35:10,433 they'll discuss and make sure that yes 909 01:35:10,433 --> 01:35:11,300 this looks like it's 910 01:35:11,300 --> 01:35:12,233 the right thing or not. 911 01:35:12,699 --> 01:35:14,199 Then that goes to the commission. 912 01:35:15,166 --> 01:35:16,533 The other thing is all of the 913 01:35:16,533 --> 01:35:18,166 professional societies have 914 01:35:18,166 --> 01:35:19,199 representatives there. 915 01:35:19,733 --> 01:35:21,466 And those professional societies will 916 01:35:21,466 --> 01:35:24,166 look at reports from their programs. 917 01:35:24,366 --> 01:35:25,666 So the TMS and the 918 01:35:25,666 --> 01:35:26,633 Trammell Society group. 919 01:35:27,133 --> 01:35:29,566 We get together and we look at all the 920 01:35:29,566 --> 01:35:31,733 materials programs and look at this 921 01:35:31,733 --> 01:35:33,300 doesn't look right to us and we can go 922 01:35:33,300 --> 01:35:35,333 and raise an issue or we can go talk to 923 01:35:35,333 --> 01:35:38,166 the team chair and and address something 924 01:35:38,166 --> 01:35:39,166 if we think something 925 01:35:39,166 --> 01:35:40,166 doesn't look quite right. 926 01:35:41,533 --> 01:35:43,600 And then the commission will finally vote 927 01:35:43,600 --> 01:35:46,166 and vote on some accreditation action. 928 01:35:46,966 --> 01:35:49,066 And so those actions the one you want is 929 01:35:49,066 --> 01:35:50,533 what's called the next general review. 930 01:35:51,333 --> 01:35:53,033 And that means you don't have to do 931 01:35:53,033 --> 01:35:55,566 anything in terms of formal reporting 932 01:35:55,566 --> 01:35:58,800 until your next review which is six years 933 01:35:58,800 --> 01:36:00,933 later or really five years by then. 934 01:36:02,000 --> 01:36:03,733 By the time you get to this this because 935 01:36:03,733 --> 01:36:04,500 this is already the 936 01:36:04,500 --> 01:36:05,966 summer following your visit. 937 01:36:07,866 --> 01:36:10,033 Then there's also interim actions which 938 01:36:10,033 --> 01:36:11,166 can either be an interim 939 01:36:11,166 --> 01:36:12,533 report or interim visit. 940 01:36:13,466 --> 01:36:14,699 Those are in two years. 941 01:36:15,633 --> 01:36:18,233 And those those the depends on the 942 01:36:18,233 --> 01:36:20,833 different shortcomings and whether it's a 943 01:36:20,833 --> 01:36:22,399 report or visit kind of depends on the 944 01:36:22,399 --> 01:36:24,966 nature of whether it's something that can 945 01:36:24,966 --> 01:36:26,066 be reported a document 946 01:36:26,066 --> 01:36:27,266 or needs to be observed. 947 01:36:29,066 --> 01:36:31,466 Or the or there's another type of interim 948 01:36:31,466 --> 01:36:34,433 is a show cause which is more serious 949 01:36:34,433 --> 01:36:36,633 that it could lead to loss of 950 01:36:36,633 --> 01:36:38,133 accreditation after that. 951 01:36:39,266 --> 01:36:41,066 It's just when you have a deficiency as 952 01:36:41,066 --> 01:36:42,433 opposed to weaknesses would 953 01:36:42,433 --> 01:36:43,633 be for just to enter and visit. 954 01:36:44,966 --> 01:36:47,933 And so that decision is made and then 955 01:36:47,933 --> 01:36:51,733 it's communicated to the program or the 956 01:36:51,733 --> 01:36:54,533 institution directly from a bet typically 957 01:36:54,533 --> 01:36:57,633 in August August or maybe into September. 958 01:36:58,333 --> 01:37:00,033 They after the meeting they have to make 959 01:37:00,033 --> 01:37:04,266 sure everything's correct and takes a 960 01:37:04,266 --> 01:37:05,399 little bit of time to do that. 961 01:37:06,300 --> 01:37:07,466 So it's a long process. 962 01:37:07,466 --> 01:37:08,600 It's going on two years. 963 01:37:09,800 --> 01:37:11,433 In addition to that if it's the first 964 01:37:11,433 --> 01:37:13,066 time there's a program from an 965 01:37:13,066 --> 01:37:15,133 institution there's what's called a 966 01:37:15,133 --> 01:37:17,866 readiness review which occurs before that 967 01:37:17,866 --> 01:37:19,566 request for evaluation. 968 01:37:20,633 --> 01:37:23,366 And the purpose of that is it's 969 01:37:23,366 --> 01:37:25,533 submitting a kind of a poor part of a 970 01:37:25,533 --> 01:37:27,600 self study and it's really just to make 971 01:37:27,600 --> 01:37:29,500 sure that the program kind of understands 972 01:37:29,500 --> 01:37:30,666 what they're getting into. 973 01:37:31,433 --> 01:37:33,566 And then there's a recommendation from 974 01:37:33,566 --> 01:37:34,899 that that said yes looks like you're 975 01:37:34,899 --> 01:37:37,066 ready to go or no we don't think you are 976 01:37:37,066 --> 01:37:40,100 but it's not binding the program can go 977 01:37:40,100 --> 01:37:42,033 ahead and institution can go ahead and 978 01:37:42,033 --> 01:37:43,866 even if it's recommended to 979 01:37:43,866 --> 01:37:46,366 not try they can still go ahead. 980 01:37:46,866 --> 01:37:48,333 It's just a recommendation that we think 981 01:37:48,333 --> 01:37:51,466 you don't quite understand what you know 982 01:37:51,466 --> 01:37:52,566 what you're up up against. 983 01:37:53,399 --> 01:37:54,933 So maybe you should wait or the program's 984 01:37:54,933 --> 01:37:56,566 not quite ready or you 985 01:37:56,566 --> 01:37:57,866 don't really understand. 986 01:37:59,033 --> 01:37:59,933 So that's the process. 987 01:38:00,199 --> 01:38:01,866 It's about a two year almost a two year 988 01:38:01,866 --> 01:38:04,000 process to get through. 989 01:38:04,866 --> 01:38:05,266 Thank you. 990 01:38:05,733 --> 01:38:10,966 And obviously the goal of a program is to 991 01:38:10,966 --> 01:38:15,000 have no shortcomings at that final exit 992 01:38:15,000 --> 01:38:15,833 meeting because that 993 01:38:15,833 --> 01:38:17,966 means you have an NGR. 994 01:38:18,433 --> 01:38:20,500 There's nothing to complain about. 995 01:38:21,233 --> 01:38:22,500 I think it's really important for 996 01:38:22,500 --> 01:38:25,833 programs to realize it's really important 997 01:38:25,833 --> 01:38:28,833 that they think deeply about whether or 998 01:38:28,833 --> 01:38:31,433 not they're going to be compliant the 999 01:38:31,433 --> 01:38:32,866 summer before the visit. 1000 01:38:33,699 --> 01:38:35,633 And to be really honest with the 1001 01:38:35,633 --> 01:38:39,566 evaluator and accept what the evaluator 1002 01:38:39,566 --> 01:38:42,766 tells them way back in the summer before 1003 01:38:42,766 --> 01:38:45,633 the visit because good evaluators will 1004 01:38:45,633 --> 01:38:49,466 let programs know early if 1005 01:38:49,466 --> 01:38:51,066 they think there might be a 1006 01:38:51,133 --> 01:38:53,366 problem especially these days when the 1007 01:38:53,366 --> 01:38:56,433 resource room is all in Dropbox or 1008 01:38:56,433 --> 01:38:58,366 Box.com or something like that. 1009 01:38:59,566 --> 01:39:01,933 So there's you know except for meeting 1010 01:39:01,933 --> 01:39:03,166 with all the people and 1011 01:39:03,166 --> 01:39:04,333 seeing the facilities. 1012 01:39:05,066 --> 01:39:07,800 Ninety five percent of the visit should 1013 01:39:07,800 --> 01:39:08,833 be done before the 1014 01:39:08,833 --> 01:39:10,433 evaluator even gets there. 1015 01:39:11,033 --> 01:39:13,166 And that's important because if a program 1016 01:39:13,166 --> 01:39:17,333 gets early warning that they haven't been 1017 01:39:17,333 --> 01:39:19,433 doing continuous improvement. 1018 01:39:19,733 --> 01:39:20,433 They haven't been 1019 01:39:20,433 --> 01:39:22,466 documenting things all of that. 1020 01:39:22,933 --> 01:39:26,066 They can start in the fall term and make 1021 01:39:26,066 --> 01:39:28,866 sure that they've done everything that 1022 01:39:28,866 --> 01:39:31,266 they need to do to demonstrate that 1023 01:39:31,266 --> 01:39:32,266 they're now compliant. 1024 01:39:32,966 --> 01:39:35,233 So when the draft statement comes to them 1025 01:39:35,233 --> 01:39:38,233 they're in very very good shape to make 1026 01:39:38,233 --> 01:39:40,066 sure those shortcomings are removed. 1027 01:39:40,933 --> 01:39:44,733 And I can't emphasize that enough for any 1028 01:39:44,733 --> 01:39:46,033 program going through. 1029 01:39:46,566 --> 01:39:48,866 So if you just you know even if you're in 1030 01:39:48,866 --> 01:39:50,233 trouble you know you should do it Jeff 1031 01:39:50,233 --> 01:39:54,066 suggested have yourself study written in 1032 01:39:54,066 --> 01:39:56,633 January before you're going to be 1033 01:39:56,633 --> 01:39:58,866 evaluated the following fall because that 1034 01:39:58,866 --> 01:40:00,333 gives you the winter term 1035 01:40:00,333 --> 01:40:02,033 or spring term to get ready. 1036 01:40:02,566 --> 01:40:05,166 And even if that doesn't work even if you 1037 01:40:05,166 --> 01:40:06,533 get shortcomings you 1038 01:40:06,533 --> 01:40:07,699 have the whole fall term. 1039 01:40:08,300 --> 01:40:10,300 But if you just sit there and wait and 1040 01:40:10,300 --> 01:40:13,566 take it and your evaluation happens to be 1041 01:40:13,566 --> 01:40:16,566 in November boy you're in a tough spot. 1042 01:40:17,066 --> 01:40:18,033 And so don't let 1043 01:40:18,033 --> 01:40:19,300 yourself get in that spot. 1044 01:40:19,566 --> 01:40:21,466 That's that's my personal advice. 1045 01:40:24,500 --> 01:40:27,433 And I would say the most shortcomings are 1046 01:40:27,433 --> 01:40:30,266 resolved most can be and many are. 1047 01:40:31,000 --> 01:40:32,399 So if you do get a 1048 01:40:32,399 --> 01:40:33,566 shortcoming at the visit. 1049 01:40:34,933 --> 01:40:36,300 Obviously it'd be nice if you can avoid 1050 01:40:36,300 --> 01:40:37,399 that as Steve describes. 1051 01:40:37,666 --> 01:40:39,300 But even if you if there is still 1052 01:40:39,300 --> 01:40:42,566 something there you have time and most of 1053 01:40:42,566 --> 01:40:44,966 the shortcomings in my experience do get 1054 01:40:44,966 --> 01:40:47,133 resolved or at least reduced to the level 1055 01:40:47,133 --> 01:40:48,033 where they don't affect 1056 01:40:48,033 --> 01:40:49,733 accreditation the action. 1057 01:40:50,233 --> 01:40:51,266 So don't get discouraged 1058 01:40:51,266 --> 01:40:53,133 if you do get something. 1059 01:40:53,800 --> 01:40:55,800 There's time and that that's the goal is 1060 01:40:55,800 --> 01:40:57,933 that everyone is compliant. 1061 01:40:58,866 --> 01:41:00,766 It's not the goal to see how many 1062 01:41:00,766 --> 01:41:02,266 shortcomings we can ring up. 1063 01:41:02,699 --> 01:41:05,733 It's we want the we want to confirm that 1064 01:41:05,733 --> 01:41:06,866 the program is compliant. 1065 01:41:06,866 --> 01:41:08,366 And if not can we help them get there. 1066 01:41:10,233 --> 01:41:12,333 You know that's the perfect lead to the 1067 01:41:12,333 --> 01:41:14,233 next question that I wanted to ask which 1068 01:41:14,233 --> 01:41:16,699 is what advice do you have for programs 1069 01:41:16,699 --> 01:41:18,866 to help them be more successful. 1070 01:41:19,100 --> 01:41:20,000 What can they do to 1071 01:41:20,000 --> 01:41:21,733 learn about these processes. 1072 01:41:22,100 --> 01:41:23,033 Where can they go. 1073 01:41:23,033 --> 01:41:24,266 Is there training they can do. 1074 01:41:24,699 --> 01:41:25,766 And how can they make sure 1075 01:41:25,766 --> 01:41:27,166 they have a successful visit. 1076 01:41:28,533 --> 01:41:30,800 To me the best way to learn is to become 1077 01:41:30,800 --> 01:41:31,899 a program evaluator. 1078 01:41:33,366 --> 01:41:34,899 You learn so much. 1079 01:41:35,166 --> 01:41:36,300 I mean it's just like if 1080 01:41:36,300 --> 01:41:38,166 you're writing a proposal to NSF. 1081 01:41:38,699 --> 01:41:40,699 What do you do to prepare for that. 1082 01:41:40,699 --> 01:41:41,600 You serve on a panel. 1083 01:41:42,433 --> 01:41:44,266 So you serve on a panel and you say oh 1084 01:41:44,266 --> 01:41:46,399 now I see what kills a proposal and I 1085 01:41:46,399 --> 01:41:49,533 know now I see why that proposal it's 1086 01:41:49,533 --> 01:41:51,166 it's it's it's stood out. 1087 01:41:51,399 --> 01:41:53,733 And this is what made it stand out. 1088 01:41:54,199 --> 01:41:55,133 It's it's the same. 1089 01:41:55,633 --> 01:41:57,333 And as I mentioned before once you 1090 01:41:57,333 --> 01:41:58,433 understand what's required 1091 01:41:58,433 --> 01:42:00,766 it's really not that hard. 1092 01:42:01,766 --> 01:42:03,033 Now that does that is a commitment 1093 01:42:03,033 --> 01:42:05,933 because it I would estimate it probably 1094 01:42:05,933 --> 01:42:07,733 takes a maybe a week of your time. 1095 01:42:08,666 --> 01:42:10,266 If you do one visit per year which is 1096 01:42:10,266 --> 01:42:11,666 kind of typical of a P.E.V. 1097 01:42:12,233 --> 01:42:14,833 Maybe a week of your time over the travel 1098 01:42:14,833 --> 01:42:17,500 the preparation and it varies a lot. 1099 01:42:17,966 --> 01:42:18,833 It gets shorter as 1100 01:42:18,833 --> 01:42:19,866 you get more experience. 1101 01:42:20,533 --> 01:42:22,100 I'm accurate a self study much faster 1102 01:42:22,100 --> 01:42:23,333 than I could when I started. 1103 01:42:25,100 --> 01:42:26,300 But it is a commitment. 1104 01:42:27,133 --> 01:42:28,066 There's no cost. 1105 01:42:28,300 --> 01:42:29,699 It pays for all your costs. 1106 01:42:30,033 --> 01:42:32,066 Of course your time is cost 1107 01:42:32,066 --> 01:42:34,166 but no out of pocket costs. 1108 01:42:34,866 --> 01:42:36,233 That's the best way I think. 1109 01:42:36,766 --> 01:42:38,166 But there are other opportunities. 1110 01:42:38,933 --> 01:42:40,366 TMS training that we 1111 01:42:40,366 --> 01:42:41,566 have for program evaluators. 1112 01:42:42,966 --> 01:42:43,899 We allow other people 1113 01:42:43,899 --> 01:42:44,933 to come just to learn. 1114 01:42:45,800 --> 01:42:46,966 That's how I got involved. 1115 01:42:47,266 --> 01:42:50,566 I was I was going to be in charge of the 1116 01:42:50,566 --> 01:42:52,633 next review for my program. 1117 01:42:53,399 --> 01:42:55,800 And TMS had the P.E.V. training and they 1118 01:42:55,800 --> 01:42:56,766 said anyone can come. 1119 01:42:57,000 --> 01:42:58,933 So I went thought I might have to learn 1120 01:42:58,933 --> 01:42:59,766 about this and they said 1121 01:42:59,766 --> 01:43:00,866 well do you want to be a P.E.V. 1122 01:43:00,866 --> 01:43:02,500 I guess I guess I'll 1123 01:43:02,500 --> 01:43:03,566 learn more if I do that. 1124 01:43:03,566 --> 01:43:07,166 So I did and and I kept with it. 1125 01:43:08,233 --> 01:43:11,633 And I find it very rewarding to do. 1126 01:43:12,833 --> 01:43:14,233 But it is a bit of a commitment. 1127 01:43:14,566 --> 01:43:16,033 So that's not for everybody 1128 01:43:16,033 --> 01:43:17,300 but that would be the best. 1129 01:43:18,100 --> 01:43:19,833 But there are other ways to learn. 1130 01:43:20,266 --> 01:43:22,100 There is a bit symposium this way. 1131 01:43:22,133 --> 01:43:24,633 There are other ABET has other training 1132 01:43:24,633 --> 01:43:28,466 programs to help to train 1133 01:43:28,466 --> 01:43:29,966 people in understanding these 1134 01:43:30,033 --> 01:43:30,833 these criteria. 1135 01:43:31,866 --> 01:43:34,833 We have a workshop or 1136 01:43:34,833 --> 01:43:36,733 a symposium every year. 1137 01:43:37,233 --> 01:43:40,033 It's been at at the fall meeting until 1138 01:43:40,033 --> 01:43:41,066 but this next year is 1139 01:43:41,066 --> 01:43:42,133 not going to be this fall. 1140 01:43:42,133 --> 01:43:43,600 We moved it to the annual meeting. 1141 01:43:43,866 --> 01:43:45,833 So it'll be in the 1142 01:43:45,833 --> 01:43:47,800 2025 TMS annual meeting. 1143 01:43:48,533 --> 01:43:50,399 It's called the Judson symposium. 1144 01:43:50,533 --> 01:43:53,266 It's named after Beth Judson who I 1145 01:43:53,266 --> 01:43:54,433 mentioned earlier was the one 1146 01:43:54,433 --> 01:43:57,366 who started the reevaluation 1147 01:43:57,466 --> 01:43:59,266 of the student outcomes and she 1148 01:43:59,266 --> 01:44:00,600 unfortunately was killed in 1149 01:44:00,600 --> 01:44:02,633 a plane crash about the year 1150 01:44:02,633 --> 01:44:03,766 after she started that. 1151 01:44:04,633 --> 01:44:05,433 So we named the 1152 01:44:05,433 --> 01:44:06,766 symposium in honor of her. 1153 01:44:07,666 --> 01:44:11,433 And in fact getting I think one of you 1154 01:44:11,433 --> 01:44:13,066 mentioned earlier about 1155 01:44:13,066 --> 01:44:14,100 just doing the right thing and 1156 01:44:14,133 --> 01:44:16,466 a credential come along the way. 1157 01:44:16,666 --> 01:44:19,899 The original name of the symposium was 1158 01:44:19,899 --> 01:44:22,366 continuous improvement of academic 1159 01:44:22,366 --> 01:44:23,533 programs and then in 1160 01:44:23,533 --> 01:44:24,633 parentheses and 1161 01:44:24,633 --> 01:44:26,266 satisfying ABET along the way. 1162 01:44:27,133 --> 01:44:29,133 We wanted to put ABET in there to get 1163 01:44:29,133 --> 01:44:30,066 people's attention 1164 01:44:30,066 --> 01:44:32,166 because ABET does get attention. 1165 01:44:33,066 --> 01:44:34,166 But we didn't want it to 1166 01:44:34,166 --> 01:44:37,833 be just this is about ABET. 1167 01:44:37,833 --> 01:44:40,833 It is but it's really about how do you 1168 01:44:40,833 --> 01:44:41,666 improve your program. 1169 01:44:42,066 --> 01:44:43,866 Yes you do need to look and see well what 1170 01:44:43,866 --> 01:44:44,899 does ABET require and 1171 01:44:44,899 --> 01:44:45,766 just make sure you have 1172 01:44:45,766 --> 01:44:46,466 those things. 1173 01:44:47,266 --> 01:44:50,033 But I think you find if you if you're 1174 01:44:50,033 --> 01:44:50,699 just trying to improve 1175 01:44:50,699 --> 01:44:51,733 your program the things you 1176 01:44:51,733 --> 01:44:53,666 have to add are probably pretty minimal 1177 01:44:53,666 --> 01:44:55,366 and some of them might 1178 01:44:55,366 --> 01:44:56,899 even help you to do what 1179 01:44:56,899 --> 01:44:59,366 you were doing better particularly things 1180 01:44:59,366 --> 01:45:01,300 with documentation and things like that. 1181 01:45:02,633 --> 01:45:05,333 So that's in TMS we 1182 01:45:05,333 --> 01:45:06,100 have that opportunity. 1183 01:45:06,366 --> 01:45:08,233 I think other professional societies have 1184 01:45:08,233 --> 01:45:10,000 similar types of programs where they have 1185 01:45:10,500 --> 01:45:13,233 committees that are working on this and 1186 01:45:13,233 --> 01:45:14,766 trying to help their programs as well. 1187 01:45:14,766 --> 01:45:17,766 That's great and to reinforce that even 1188 01:45:17,766 --> 01:45:19,733 more the Judson 1189 01:45:19,733 --> 01:45:21,566 Symposium that used to really be 1190 01:45:21,566 --> 01:45:23,066 all about from the accreditation 1191 01:45:23,066 --> 01:45:24,833 committee has now joined 1192 01:45:24,833 --> 01:45:26,766 forces with the education committee 1193 01:45:27,066 --> 01:45:28,333 and it's a joint meeting. 1194 01:45:29,266 --> 01:45:31,766 And Tim and I are both on the education 1195 01:45:31,766 --> 01:45:34,133 committee now and that's 1196 01:45:34,133 --> 01:45:35,366 where this whole idea for the 1197 01:45:35,566 --> 01:45:37,466 podcast actually came from. 1198 01:45:38,033 --> 01:45:40,466 So it's very appropriate we're talking to 1199 01:45:40,466 --> 01:45:41,399 you since you've been 1200 01:45:41,399 --> 01:45:43,266 very involved in the Judson 1201 01:45:43,266 --> 01:45:46,166 committee symposium for a long time. 1202 01:45:47,066 --> 01:45:48,800 So you already mentioned that ABET is 1203 01:45:48,800 --> 01:45:50,433 continuously improving. 1204 01:45:51,033 --> 01:45:52,266 Where is it going now? 1205 01:45:52,266 --> 01:45:53,466 What's next for ABET? 1206 01:45:58,133 --> 01:45:59,366 Yeah so there I mean there are a number 1207 01:45:59,366 --> 01:46:01,566 of areas that are kind of hot topics. 1208 01:46:01,566 --> 01:46:02,199 Probably one of the 1209 01:46:02,199 --> 01:46:04,266 biggest one right now is DEI. 1210 01:46:05,000 --> 01:46:09,433 And I was happy when I was a chair we 1211 01:46:09,433 --> 01:46:13,733 actually approved some changes to the 1212 01:46:13,733 --> 01:46:15,533 criteria to incorporate 1213 01:46:15,533 --> 01:46:16,133 DEI. 1214 01:46:16,866 --> 01:46:17,433 And ABET had been 1215 01:46:17,433 --> 01:46:18,433 trying to do this a while. 1216 01:46:18,833 --> 01:46:21,866 They first worked on kind of the overall 1217 01:46:21,866 --> 01:46:23,066 statements of the 1218 01:46:23,066 --> 01:46:24,699 organization and those all 1219 01:46:25,333 --> 01:46:27,666 fine but really where the impact is going 1220 01:46:27,666 --> 01:46:28,633 to be is if it's in the 1221 01:46:28,633 --> 01:46:31,066 criteria where programs have to 1222 01:46:31,566 --> 01:46:32,133 do something. 1223 01:46:32,933 --> 01:46:33,966 That's where I think we're 1224 01:46:33,966 --> 01:46:34,933 going to really have impact. 1225 01:46:35,733 --> 01:46:38,800 So they asked all the commissions to try 1226 01:46:38,800 --> 01:46:41,166 to put DEI related 1227 01:46:41,166 --> 01:46:43,733 topics into the criteria. 1228 01:46:45,066 --> 01:46:49,066 And so we this was so this is 2019-20 and 1229 01:46:49,066 --> 01:46:51,233 they had asked that and so 1230 01:46:51,233 --> 01:46:52,933 I was a chair at the time 1231 01:46:52,933 --> 01:46:56,066 and so I kept on our criterion committee 1232 01:46:56,066 --> 01:46:57,533 to make sure they kept working on this. 1233 01:46:58,199 --> 01:46:59,699 And they did and they came up with 1234 01:46:59,699 --> 01:47:01,266 proposals in four 1235 01:47:01,266 --> 01:47:03,666 different criteria to have changes. 1236 01:47:04,933 --> 01:47:06,699 And happy that we did this 1237 01:47:06,699 --> 01:47:08,100 even though this was COVID. 1238 01:47:08,100 --> 01:47:10,033 So we were also dealing with how to 1239 01:47:10,033 --> 01:47:12,833 transition all of the visits to virtual. 1240 01:47:14,199 --> 01:47:15,833 So it was a it was a hectic 1241 01:47:15,833 --> 01:47:18,500 year but we did pass those. 1242 01:47:20,033 --> 01:47:21,666 And so I was really pleased with that and 1243 01:47:21,666 --> 01:47:23,966 it's been a while the so 1244 01:47:23,966 --> 01:47:25,233 after the commission approves 1245 01:47:25,266 --> 01:47:26,866 that it has to be approved by another 1246 01:47:26,866 --> 01:47:27,966 body called the 1247 01:47:27,966 --> 01:47:29,766 engineering area delegation. 1248 01:47:30,966 --> 01:47:33,533 So ABET used to have a board of like 55 1249 01:47:33,533 --> 01:47:36,366 people and a board of 55 1250 01:47:36,366 --> 01:47:37,833 people cannot function. 1251 01:47:38,766 --> 01:47:39,933 No absolutely not. 1252 01:47:40,466 --> 01:47:41,166 And the reason it was 1253 01:47:41,166 --> 01:47:43,100 so big is because ABET is 1254 01:47:44,033 --> 01:47:45,166 has representatives all these 1255 01:47:45,166 --> 01:47:48,000 professional societies and you have to 1256 01:47:48,000 --> 01:47:49,033 give them representation 1257 01:47:49,166 --> 01:47:50,133 and you have to have a little bit 1258 01:47:50,133 --> 01:47:50,966 different representation 1259 01:47:50,966 --> 01:47:52,666 for the ones that have more 1260 01:47:52,666 --> 01:47:55,800 programs and so it was a big body. 1261 01:47:56,066 --> 01:47:57,533 And so they reorganized so they now have 1262 01:47:57,533 --> 01:48:02,466 a small board of I think 13 people 1263 01:48:02,833 --> 01:48:04,766 but they still have the area delegation 1264 01:48:04,766 --> 01:48:07,066 or the well the area 1265 01:48:07,066 --> 01:48:08,066 delegation are for the different 1266 01:48:08,066 --> 01:48:11,866 commissions and the board of delegates is 1267 01:48:11,866 --> 01:48:13,066 all of those together. 1268 01:48:13,666 --> 01:48:14,333 And that's where the 1269 01:48:14,333 --> 01:48:15,233 representation comes. 1270 01:48:15,466 --> 01:48:17,600 So TMS has a representative on the 1271 01:48:17,600 --> 01:48:18,666 engineering area delegation 1272 01:48:18,666 --> 01:48:21,633 and the board of delegates. 1273 01:48:22,699 --> 01:48:23,433 And that's the body 1274 01:48:23,433 --> 01:48:24,466 that has to then approve. 1275 01:48:24,933 --> 01:48:26,533 Well they didn't want to approve it they 1276 01:48:26,533 --> 01:48:28,533 wanted to they kind of kicked 1277 01:48:28,533 --> 01:48:29,800 it down the road went through 1278 01:48:30,766 --> 01:48:32,333 talking about it for a while and tried 1279 01:48:32,333 --> 01:48:33,433 again and finally a year 1280 01:48:33,433 --> 01:48:35,566 later it did approve did get 1281 01:48:35,566 --> 01:48:38,233 approved two of the four because of the 1282 01:48:38,233 --> 01:48:39,800 of the criteria there's 1283 01:48:39,800 --> 01:48:42,333 eight criteria of those five are 1284 01:48:42,333 --> 01:48:44,000 what's called harmonized which means 1285 01:48:44,000 --> 01:48:44,833 they're the same for 1286 01:48:44,833 --> 01:48:46,466 all four commissions. 1287 01:48:47,533 --> 01:48:49,600 So anything that changes and knows has to 1288 01:48:49,600 --> 01:48:51,166 be agreed on by all four commissions 1289 01:48:52,300 --> 01:48:54,233 which is can be a challenge. 1290 01:48:55,133 --> 01:48:58,133 The other three are commission specific 1291 01:48:58,133 --> 01:49:00,766 so we can EAC could pass those. 1292 01:49:01,266 --> 01:49:04,566 So two of them those two passed the other 1293 01:49:04,566 --> 01:49:06,399 two they passed the 1294 01:49:06,399 --> 01:49:07,533 commission but then they got 1295 01:49:07,533 --> 01:49:09,066 stopped by the accreditation council 1296 01:49:09,066 --> 01:49:10,266 because none of the 1297 01:49:10,266 --> 01:49:11,300 other commissions were ready. 1298 01:49:12,733 --> 01:49:14,600 So the other two got through and they 1299 01:49:14,600 --> 01:49:18,966 were in criterion five and criterion six. 1300 01:49:19,000 --> 01:49:21,233 So criterion five is curriculum and 1301 01:49:21,233 --> 01:49:23,066 criterion six is faculty. 1302 01:49:24,433 --> 01:49:26,266 And so it went through it took a couple 1303 01:49:26,266 --> 01:49:27,766 years to finally get them approved. 1304 01:49:28,633 --> 01:49:30,633 And now they're still not approved but 1305 01:49:30,633 --> 01:49:33,466 they are they have approved a pilot study 1306 01:49:33,800 --> 01:49:35,500 that pilot study we're going into the 1307 01:49:35,500 --> 01:49:36,866 second year of the pilot study. 1308 01:49:37,933 --> 01:49:40,433 And so the idea there is we'll go through 1309 01:49:40,433 --> 01:49:41,866 the pilot study kind of learn 1310 01:49:42,433 --> 01:49:44,966 how is this being valid how are people 1311 01:49:44,966 --> 01:49:46,899 responding how are they how are they 1312 01:49:46,933 --> 01:49:48,433 saying they're compliant how are we going 1313 01:49:48,433 --> 01:49:49,899 to judge that how are we 1314 01:49:49,899 --> 01:49:50,866 going to train people to 1315 01:49:50,866 --> 01:49:52,566 evaluate that so there's some issues that 1316 01:49:52,566 --> 01:49:53,433 have to be worked out and 1317 01:49:53,433 --> 01:49:54,100 that's kind of what we're 1318 01:49:54,100 --> 01:49:58,433 trying to do these these couple years. So 1319 01:49:58,433 --> 01:50:00,033 that's that's the in process 1320 01:50:00,033 --> 01:50:02,733 the essence of those additions 1321 01:50:03,333 --> 01:50:06,000 are essentially I mean you're designing 1322 01:50:06,000 --> 01:50:08,399 things well I should say it 1323 01:50:08,399 --> 01:50:10,100 mentions that you have to have 1324 01:50:11,466 --> 01:50:14,199 DEI in the professional context and to me 1325 01:50:14,199 --> 01:50:15,233 that's the important part 1326 01:50:15,699 --> 01:50:18,066 it's not just oh you need to go to a DEI 1327 01:50:18,066 --> 01:50:19,833 seminar and learn about diversity, 1328 01:50:19,833 --> 01:50:20,466 equity, and inclusion. 1329 01:50:21,199 --> 01:50:23,566 How is that important for engineering? 1330 01:50:24,699 --> 01:50:26,466 And obviously it is I 1331 01:50:26,466 --> 01:50:28,133 mean to me the two ways it 1332 01:50:28,133 --> 01:50:29,766 stands out is one is you have to work 1333 01:50:29,766 --> 01:50:31,266 with different people 1334 01:50:31,266 --> 01:50:33,133 you have to understand how 1335 01:50:33,133 --> 01:50:35,766 to do that and do that effectively. And 1336 01:50:35,766 --> 01:50:36,833 secondly probably with even 1337 01:50:36,833 --> 01:50:38,133 the bigger impact is you're 1338 01:50:38,133 --> 01:50:41,366 designing whether it's materials or 1339 01:50:41,366 --> 01:50:44,399 products or processes for a lot of 1340 01:50:44,399 --> 01:50:45,066 different people and if 1341 01:50:45,100 --> 01:50:47,133 you don't understand and don't have 1342 01:50:47,133 --> 01:50:49,466 perspectives from different people your 1343 01:50:49,466 --> 01:50:50,166 market share is going 1344 01:50:50,166 --> 01:50:53,199 to be people just like you right and if 1345 01:50:53,199 --> 01:50:54,666 you're going to really just to be 1346 01:50:54,666 --> 01:50:55,666 effective even getting 1347 01:50:55,733 --> 01:50:58,733 setting aside the social justice aspects 1348 01:50:58,733 --> 01:51:00,266 of it just talking 1349 01:51:00,266 --> 01:51:01,733 about the business case for 1350 01:51:02,033 --> 01:51:05,066 the diversity is you're going to have a 1351 01:51:05,066 --> 01:51:06,600 product that is is more 1352 01:51:06,600 --> 01:51:07,699 useful for more people. 1353 01:51:08,600 --> 01:51:10,633 And so that's kind of to me the essence 1354 01:51:10,633 --> 01:51:13,433 of what I think is 1355 01:51:13,433 --> 01:51:15,300 appropriate to put in a engineering 1356 01:51:15,333 --> 01:51:18,199 criteria because that is important to be 1357 01:51:18,199 --> 01:51:19,466 a successful engineer. 1358 01:51:20,466 --> 01:51:21,033 And of course for the 1359 01:51:21,033 --> 01:51:23,933 faculty it's about understanding those 1360 01:51:23,933 --> 01:51:25,866 concepts I would say so you 1361 01:51:25,866 --> 01:51:29,333 can support that education. 1362 01:51:30,166 --> 01:51:31,899 So that's going through this pilot study 1363 01:51:31,899 --> 01:51:34,199 I'm hopeful that it will 1364 01:51:34,199 --> 01:51:36,033 you know it may be tweaked 1365 01:51:36,133 --> 01:51:38,866 a little bit because as as we in the 1366 01:51:38,866 --> 01:51:40,733 first year of this self the 1367 01:51:40,733 --> 01:51:41,966 first year of the pilot study 1368 01:51:43,066 --> 01:51:46,133 what happened was a lot so that was 1369 01:51:46,133 --> 01:51:47,933 optional and any program could 1370 01:51:47,933 --> 01:51:50,333 decide to do it or not it had 1371 01:51:50,333 --> 01:51:52,166 absolutely no bearing on the evaluation 1372 01:51:52,166 --> 01:51:53,866 it was evaluated not 1373 01:51:53,866 --> 01:51:54,899 even by the team it was 1374 01:51:54,899 --> 01:51:56,600 evaluated by a separate group of people 1375 01:51:56,600 --> 01:51:57,733 that just evaluated 1376 01:51:57,733 --> 01:52:00,666 that. So there's no risk and 1377 01:52:01,399 --> 01:52:03,233 some programs did I think it ended up 1378 01:52:03,233 --> 01:52:06,000 being maybe 40 institutions said they 1379 01:52:06,000 --> 01:52:07,300 would and maybe 20 some 1380 01:52:07,333 --> 01:52:09,933 actually did something on that order. 1381 01:52:10,666 --> 01:52:12,566 What they found was they 1382 01:52:12,566 --> 01:52:14,199 the programs often just talked 1383 01:52:14,199 --> 01:52:16,000 about what they did in DI but didn't 1384 01:52:16,000 --> 01:52:18,266 really address the criteria 1385 01:52:18,266 --> 01:52:19,766 and how they were compliant 1386 01:52:19,766 --> 01:52:22,633 with those words. So in the second year 1387 01:52:22,633 --> 01:52:24,333 they changed the prompt so 1388 01:52:24,333 --> 01:52:25,633 there's a like an addendum 1389 01:52:25,633 --> 01:52:27,366 to the cell study if you're if you're 1390 01:52:27,366 --> 01:52:29,433 going to participate to really try to 1391 01:52:29,433 --> 01:52:30,166 make that more clear 1392 01:52:30,233 --> 01:52:31,933 that you really have to demonstrate how 1393 01:52:31,933 --> 01:52:33,666 are you doing what we've 1394 01:52:33,666 --> 01:52:36,633 described here. And so I'm going 1395 01:52:36,633 --> 01:52:38,933 to be working with a task group to look 1396 01:52:38,933 --> 01:52:39,966 at that and also start 1397 01:52:39,966 --> 01:52:42,366 thinking about what are we looking 1398 01:52:42,366 --> 01:52:44,033 for in terms of their criteria and how 1399 01:52:44,033 --> 01:52:46,100 are you going to train our 1400 01:52:46,100 --> 01:52:48,366 our teams to be able to look 1401 01:52:48,366 --> 01:52:51,033 at that and interpret that. So that to me 1402 01:52:51,033 --> 01:52:52,833 that's the biggest thing 1403 01:52:52,833 --> 01:52:55,466 that's on the horizon. There's 1404 01:52:55,500 --> 01:52:58,100 also there's DEI coming in different 1405 01:52:58,100 --> 01:53:00,966 areas too. There's a proposal coming 1406 01:53:00,966 --> 01:53:02,399 through in the institutional 1407 01:53:02,533 --> 01:53:07,333 support with some kind of general DEI 1408 01:53:07,333 --> 01:53:09,899 related language. The kind 1409 01:53:09,899 --> 01:53:10,833 of language that's in there 1410 01:53:10,833 --> 01:53:12,566 now is the kind of thing that all 1411 01:53:12,566 --> 01:53:14,066 institutions are doing just because of 1412 01:53:14,066 --> 01:53:16,266 federal law. So I'm not sure 1413 01:53:16,266 --> 01:53:18,866 that that's going to be as impactful but 1414 01:53:18,866 --> 01:53:20,433 I think in criterion five 1415 01:53:20,433 --> 01:53:21,366 where you're talking about 1416 01:53:22,266 --> 01:53:24,466 in that experience in that design 1417 01:53:24,466 --> 01:53:25,566 experience how are you 1418 01:53:25,566 --> 01:53:26,633 preparing your students 1419 01:53:27,166 --> 01:53:29,966 in the context considering equity and so 1420 01:53:29,966 --> 01:53:30,566 forth when you're 1421 01:53:30,566 --> 01:53:32,399 designing that's going to make 1422 01:53:32,933 --> 01:53:35,333 better engineers and if every program has 1423 01:53:35,333 --> 01:53:36,533 to every six years 1424 01:53:36,533 --> 01:53:38,600 explain how are they doing that 1425 01:53:39,466 --> 01:53:41,266 that's I think that's I think that'll 1426 01:53:41,266 --> 01:53:43,066 really move the needle more than a lot of 1427 01:53:43,066 --> 01:53:43,966 other things that we 1428 01:53:44,533 --> 01:53:46,166 we do need to do the other things but I 1429 01:53:46,166 --> 01:53:47,133 think this is really 1430 01:53:47,133 --> 01:53:49,133 impact. Totally agree with you. 1431 01:53:49,800 --> 01:53:53,600 Are you at all concerned about the new 1432 01:53:53,600 --> 01:53:58,433 politics of DEI and DEI anti-DEI 1433 01:53:58,466 --> 01:54:01,333 legislation and now I believe Congress is 1434 01:54:01,333 --> 01:54:04,566 proposing to write a law specifically 1435 01:54:05,800 --> 01:54:09,466 talking about accreditation bodies. So 1436 01:54:09,466 --> 01:54:11,433 how is ABET responding to this? 1437 01:54:14,466 --> 01:54:16,300 ABET's very cognizant of that and they're 1438 01:54:16,300 --> 01:54:17,033 looking at it and 1439 01:54:17,033 --> 01:54:18,366 they're looking at the laws and 1440 01:54:19,566 --> 01:54:20,966 they don't want to put in it they don't 1441 01:54:20,966 --> 01:54:21,899 want to they don't want 1442 01:54:21,899 --> 01:54:23,633 to not do this I think ABET 1443 01:54:23,933 --> 01:54:25,533 I mean I know I know enough of the people 1444 01:54:25,533 --> 01:54:26,233 in the leadership and 1445 01:54:26,233 --> 01:54:27,199 so forth I think they are 1446 01:54:27,199 --> 01:54:29,566 committed to doing this and they're going 1447 01:54:29,566 --> 01:54:31,133 to find find a way to do it 1448 01:54:31,133 --> 01:54:31,966 but they don't want to put a 1449 01:54:31,966 --> 01:54:34,866 program in in the position where if they 1450 01:54:34,866 --> 01:54:35,933 want to be accredited 1451 01:54:35,933 --> 01:54:37,166 they're going to lose state funding 1452 01:54:37,166 --> 01:54:39,233 I mean they don't want to do that so 1453 01:54:39,233 --> 01:54:42,066 they'll look at a way to do that. I'm 1454 01:54:42,066 --> 01:54:43,466 experienced that we had a 1455 01:54:43,633 --> 01:54:45,466 legislation passed a couple weeks ago 1456 01:54:45,466 --> 01:54:47,199 here and when you read it 1457 01:54:47,199 --> 01:54:48,333 first it looks like we're in 1458 01:54:48,366 --> 01:54:50,199 trouble but actually ours fortunately 1459 01:54:50,199 --> 01:54:51,833 related accreditation does 1460 01:54:51,833 --> 01:54:53,899 have a clause that says you 1461 01:54:53,899 --> 01:54:55,233 know if you have to do something because 1462 01:54:55,233 --> 01:54:56,866 of accreditation you're 1463 01:54:56,866 --> 01:54:57,966 excluded from this so we're 1464 01:54:57,966 --> 01:55:01,266 okay and at this point but it's going to 1465 01:55:01,266 --> 01:55:02,300 take I think it's going to 1466 01:55:02,300 --> 01:55:04,866 be a lot of um a lot of it 1467 01:55:04,866 --> 01:55:07,933 is the words and those words are 1468 01:55:07,933 --> 01:55:10,133 triggering and in our areas we're 1469 01:55:10,133 --> 01:55:12,266 stopping we're not using DEI as 1470 01:55:12,266 --> 01:55:15,233 much of course I'm in Alabama where it's 1471 01:55:15,233 --> 01:55:15,766 a little bit different 1472 01:55:15,766 --> 01:55:17,666 than Michigan but we we 1473 01:55:18,266 --> 01:55:20,033 so we try to describe the things without 1474 01:55:20,033 --> 01:55:21,366 using the triggering words 1475 01:55:21,366 --> 01:55:23,333 you can because what you do 1476 01:55:23,800 --> 01:55:26,366 it's not as people don't really object to 1477 01:55:26,366 --> 01:55:26,966 the things you might 1478 01:55:26,966 --> 01:55:28,433 do they object to those 1479 01:55:28,933 --> 01:55:30,600 words that they have some other 1480 01:55:30,600 --> 01:55:31,533 connotation of what 1481 01:55:31,533 --> 01:55:32,733 you're really talking about 1482 01:55:33,633 --> 01:55:35,600 so some of it's going to be trying to 1483 01:55:35,600 --> 01:55:37,466 describe things in a different way 1484 01:55:38,199 --> 01:55:39,933 that accomplish what we want them to 1485 01:55:39,933 --> 01:55:41,233 accomplish for example you talk about 1486 01:55:41,233 --> 01:55:42,033 equitable design well 1487 01:55:42,066 --> 01:55:45,333 you can talk about universal design in 1488 01:55:45,333 --> 01:55:46,899 kind of doing the same 1489 01:55:46,899 --> 01:55:48,533 things how do you make this design 1490 01:55:48,933 --> 01:55:51,433 usable by more people well you're doing 1491 01:55:51,433 --> 01:55:53,666 that for equity but if you 1492 01:55:53,666 --> 01:55:55,666 call it equity then that may 1493 01:55:55,666 --> 01:55:57,966 trigger some people so you might call it 1494 01:55:57,966 --> 01:55:58,966 universal design you're also 1495 01:55:58,966 --> 01:56:02,199 doing it for everybody so you're 1496 01:56:02,199 --> 01:56:04,166 doing it for people who are introverts 1497 01:56:04,166 --> 01:56:07,066 people who are extroverts people from 1498 01:56:07,066 --> 01:56:08,533 different socio-economic 1499 01:56:08,866 --> 01:56:12,133 groups and it's everybody it's not 1500 01:56:12,133 --> 01:56:13,666 absolutely particular group 1501 01:56:13,666 --> 01:56:16,633 that gets you in trouble with 1502 01:56:16,833 --> 01:56:24,966 politics absolutely yeah something else 1503 01:56:24,966 --> 01:56:26,766 i'm really happy to hear about that 1504 01:56:26,766 --> 01:56:27,966 approach is that it's 1505 01:56:28,033 --> 01:56:30,666 about programs taking action not just 1506 01:56:30,666 --> 01:56:31,833 saying i have this 1507 01:56:31,833 --> 01:56:33,399 principle that i believe in i mean 1508 01:56:33,399 --> 01:56:35,333 it's nice to believe in principles but if 1509 01:56:35,333 --> 01:56:36,466 that's all you're doing is 1510 01:56:36,466 --> 01:56:37,366 believing in a principle and 1511 01:56:37,399 --> 01:56:39,166 not doing anything about it it's kind of 1512 01:56:39,166 --> 01:56:40,933 like so what so the fact 1513 01:56:40,933 --> 01:56:42,199 that you're saying what are you 1514 01:56:42,199 --> 01:56:43,699 doing to make your students more 1515 01:56:43,699 --> 01:56:47,466 successful in you know being a part of 1516 01:56:47,466 --> 01:56:48,266 this world that we're in 1517 01:56:48,266 --> 01:56:50,266 together i'm really happy to hear that 1518 01:56:50,266 --> 01:56:51,466 yeah in fact our 1519 01:56:51,466 --> 01:56:53,433 legislation that was passed in alabama 1520 01:56:54,366 --> 01:56:57,633 they um it it talks about the what 1521 01:56:57,633 --> 01:56:58,566 they're really objecting to 1522 01:56:58,566 --> 01:57:00,600 is forcing someone to believe 1523 01:57:00,600 --> 01:57:03,633 something which we don't do that we teach 1524 01:57:03,633 --> 01:57:04,800 this is the you know this 1525 01:57:04,800 --> 01:57:06,033 is and this is how you design 1526 01:57:06,033 --> 01:57:07,933 things this is why it's good but we don't 1527 01:57:07,933 --> 01:57:09,000 say you have to believe in 1528 01:57:09,000 --> 01:57:10,066 this principle which is what 1529 01:57:10,066 --> 01:57:12,633 they're kind of objecting to and some of 1530 01:57:12,633 --> 01:57:13,733 the exclusions are the 1531 01:57:13,733 --> 01:57:14,533 things like well if you're 1532 01:57:14,533 --> 01:57:16,300 supporting students even if you're 1533 01:57:16,300 --> 01:57:17,666 targeting a particular group 1534 01:57:17,666 --> 01:57:19,100 that's okay if it's just about 1535 01:57:19,566 --> 01:57:21,399 supporting students as long as you're not 1536 01:57:21,399 --> 01:57:23,166 making limiting 1537 01:57:23,166 --> 01:57:26,166 participation based on being in a group 1538 01:57:26,266 --> 01:57:30,866 which we don't do you know so um yeah i 1539 01:57:30,866 --> 01:57:32,733 agree and so it's when you think about 1540 01:57:32,733 --> 01:57:33,233 well what do you need 1541 01:57:33,233 --> 01:57:35,366 to do to accomplish what you might 1542 01:57:35,366 --> 01:57:37,133 describe as your dei goals well what do 1543 01:57:37,133 --> 01:57:37,766 you have to do to get 1544 01:57:37,766 --> 01:57:41,300 there you describe that and that that 1545 01:57:41,300 --> 01:57:42,066 you're not going to get 1546 01:57:42,066 --> 01:57:43,733 objection to so i think that's 1547 01:57:43,800 --> 01:57:45,166 probably the way abet's going to have to 1548 01:57:45,166 --> 01:57:46,300 do it they may have to change the 1549 01:57:46,300 --> 01:57:48,733 language to not make 1550 01:57:48,733 --> 01:57:52,033 it offensive but still force programs to 1551 01:57:52,033 --> 01:57:53,433 think about those things 1552 01:57:53,433 --> 01:57:55,266 and it's already in there i 1553 01:57:55,266 --> 01:57:57,199 mean with the teams right we talked about 1554 01:57:57,199 --> 01:57:58,333 work on diverse teams has 1555 01:57:58,333 --> 01:58:00,166 been there for years and that's 1556 01:58:00,233 --> 01:58:02,266 important and it's already in there but 1557 01:58:02,266 --> 01:58:04,133 then the kind of the design 1558 01:58:04,133 --> 01:58:05,833 part maybe it is in there too 1559 01:58:06,033 --> 01:58:09,166 but and communication to a range of 1560 01:58:09,166 --> 01:58:12,433 audiences it's there too 1561 01:58:12,433 --> 01:58:14,966 absolutely well jeff this has been 1562 01:58:15,000 --> 01:58:17,699 fantastic thank you so much i want to 1563 01:58:17,699 --> 01:58:19,633 mention that we're hosting 1564 01:58:19,633 --> 01:58:21,366 the north american materials 1565 01:58:50,366 --> 01:58:51,266 all the programs. 1566 01:58:54,500 --> 01:58:57,233 outcomes assessment process, but some of 1567 01:58:57,233 --> 01:58:57,966 those programs just 1568 01:58:57,966 --> 01:58:59,000 thought it was done and they 1569 01:58:59,000 --> 01:59:01,966 forgot to also document the continuous 1570 01:59:01,966 --> 01:59:02,833 improvement of their 1571 01:59:02,833 --> 01:59:04,133 program, which is really the whole 1572 01:59:04,133 --> 01:59:04,533 point. 1573 01:59:05,500 --> 01:59:08,000 And it's okay if measuring 1574 01:59:08,000 --> 01:59:10,966 the assessing the outcomes. 1575 01:59:11,500 --> 01:59:13,066 It's a really important thing to do 1576 01:59:13,066 --> 01:59:13,966 because it gives you 1577 01:59:13,966 --> 01:59:15,399 great insight into whether or 1578 01:59:15,399 --> 01:59:16,399 not you have a problem 1579 01:59:16,399 --> 01:59:18,366 anywhere and need to address it. 1580 01:59:19,566 --> 01:59:22,766 And one thing I've learned is by using 1581 01:59:22,766 --> 01:59:24,000 all the data and having 1582 01:59:24,000 --> 01:59:26,966 a histogram, I can't help 1583 01:59:26,966 --> 01:59:28,566 but look at the little 1584 01:59:28,566 --> 01:59:30,066 tick marks in the tails. 1585 01:59:31,366 --> 01:59:34,666 Even though, yeah, we have, you know, 80% 1586 01:59:35,066 --> 01:59:37,466 of our students are beyond our threshold, 1587 01:59:37,966 --> 01:59:40,566 you see that tail and now I'm wondering, 1588 01:59:40,566 --> 01:59:41,399 well, what about them? 1589 01:59:43,133 --> 01:59:46,066 And because we actually have all the 1590 01:59:46,066 --> 01:59:47,466 data, the actual unique 1591 01:59:47,466 --> 01:59:49,266 names, it's all secret. 1592 01:59:49,500 --> 01:59:53,433 I can't see who they are, but we have it. 1593 01:59:54,300 --> 01:59:57,800 And now we have a new AI tool that knows 1594 01:59:57,800 --> 01:59:59,699 how to look into SQL databases. 1595 02:00:00,800 --> 02:00:04,733 Again, our AI tool is private. 1596 02:00:05,266 --> 02:00:09,300 It's completely FERPA compatible because 1597 02:00:09,300 --> 02:00:11,300 we don't give any 1598 02:00:11,300 --> 02:00:13,800 information to OpenAI or Microsoft 1599 02:00:14,066 --> 02:00:14,566 or Google. 1600 02:00:15,633 --> 02:00:17,600 We've licensed their tools, we pay for 1601 02:00:17,600 --> 02:00:18,899 it, so that we can work 1602 02:00:18,899 --> 02:00:20,566 on our curated dataset. 1603 02:00:21,600 --> 02:00:22,766 Why not work on that? 1604 02:00:22,766 --> 02:00:25,166 We have 6,000 students in our system now. 1605 02:00:25,800 --> 02:00:27,866 Maybe we can do small longitudinal 1606 02:00:27,866 --> 02:00:29,899 studies with, you know, 1607 02:00:29,899 --> 02:00:31,000 here are these people in the 1608 02:00:31,266 --> 02:00:31,366 tails. 1609 02:00:31,600 --> 02:00:33,300 How did they do in the other criteria? 1610 02:00:33,866 --> 02:00:34,933 How did they do the next 1611 02:00:34,933 --> 02:00:35,933 year they were assessed? 1612 02:00:37,600 --> 02:00:40,466 And so I think this ABET process, 1613 02:00:40,466 --> 02:00:41,366 although this is not 1614 02:00:41,366 --> 02:00:45,566 explicitly part of ABET, it's 1615 02:00:45,633 --> 02:00:48,766 allowing us to view our data in ways we 1616 02:00:48,766 --> 02:00:49,500 probably never 1617 02:00:49,500 --> 02:00:51,166 dreamed we'd be able to do. 1618 02:00:51,766 --> 02:00:53,699 And it might be one of the advantages of 1619 02:00:53,699 --> 02:00:55,166 artificial intelligence. 1620 02:00:55,166 --> 02:00:57,100 It's certainly these large language 1621 02:00:57,100 --> 02:01:00,500 models to figure out what's going on. 1622 02:01:00,500 --> 02:01:03,500 So we're kind of excited about this, but 1623 02:01:03,500 --> 02:01:05,533 you still have to improve your program. 1624 02:01:06,399 --> 02:01:08,199 And we do that with lots of other ways. 1625 02:01:08,500 --> 02:01:10,300 Tim is creating a new math 1626 02:01:10,300 --> 02:01:11,866 course for all of our sophomores. 1627 02:01:12,500 --> 02:01:14,566 We had instituted a fifth math course in 1628 02:01:14,566 --> 02:01:15,233 our curriculum, 1629 02:01:15,866 --> 02:01:17,266 thinking it would help them 1630 02:01:17,266 --> 02:01:18,066 learn the math they 1631 02:01:18,066 --> 02:01:19,166 needed for the courses. 1632 02:01:19,833 --> 02:01:21,266 And the students all waited until their 1633 02:01:21,266 --> 02:01:22,399 last year, senior year 1634 02:01:22,399 --> 02:01:24,199 to take it, which defeated 1635 02:01:24,199 --> 02:01:24,733 the purpose. 1636 02:01:25,366 --> 02:01:26,300 So we reevaluated. 1637 02:01:26,566 --> 02:01:27,899 Now we're going to offer it as a 1638 02:01:27,899 --> 02:01:29,199 sophomore level course 1639 02:01:29,199 --> 02:01:31,666 and cover, you know, complex 1640 02:01:31,699 --> 02:01:33,666 variables or orthogonal series, 1641 02:01:34,199 --> 02:01:35,766 eigenvalue problems, all this 1642 02:01:35,766 --> 02:01:37,033 stuff that they don't really 1643 02:01:37,033 --> 02:01:38,800 get in their calculus sequence. 1644 02:01:39,866 --> 02:01:41,766 So these are all the things that good 1645 02:01:41,766 --> 02:01:43,600 programs will always do 1646 02:01:43,600 --> 02:01:45,199 to improve their program. 1647 02:01:45,199 --> 02:01:46,033 And it never ends. 1648 02:01:47,033 --> 02:01:49,033 We're constantly having to deal with it. 1649 02:01:49,466 --> 02:01:51,899 So anyway, any final 1650 02:01:51,899 --> 02:01:53,466 thoughts before we take off? 1651 02:01:58,866 --> 02:02:00,933 The only thing I guess I would say is, 1652 02:02:00,933 --> 02:02:01,966 you know, I know 1653 02:02:01,966 --> 02:02:03,000 people sometimes have issues 1654 02:02:03,166 --> 02:02:05,699 with specific things that ABED does, but 1655 02:02:05,699 --> 02:02:08,766 I think ideally the 1656 02:02:08,766 --> 02:02:10,366 idea that someone's coming 1657 02:02:10,366 --> 02:02:13,333 to look at your program and give you 1658 02:02:13,333 --> 02:02:15,966 feedback should be a positive thing. 1659 02:02:16,600 --> 02:02:18,000 And when we find that our program 1660 02:02:18,000 --> 02:02:18,866 developers are not doing 1661 02:02:18,866 --> 02:02:22,199 that, we either stop sending 1662 02:02:22,199 --> 02:02:24,933 them or try to encourage 1663 02:02:24,933 --> 02:02:27,699 them to follow that philosophy. 1664 02:02:27,699 --> 02:02:28,633 But that really is what it 1665 02:02:28,633 --> 02:02:29,966 should be, is meant to be. 1666 02:02:30,699 --> 02:02:32,833 And I think I'm still hopeful that we're 1667 02:02:32,833 --> 02:02:34,300 moving closer to that. 1668 02:02:35,800 --> 02:02:38,699 And I hope we look at it that way and try 1669 02:02:38,699 --> 02:02:40,633 to work towards that would be what I like 1670 02:02:40,633 --> 02:02:41,033 to do. 1671 02:02:41,833 --> 02:02:42,300 All right. 1672 02:02:42,300 --> 02:02:43,733 Well, thank you very much. 1673 02:02:43,733 --> 02:02:44,899 And I think we'll sign off. 1674 02:02:46,300 --> 02:02:48,166 So see you later. 1675 02:02:48,533 --> 02:02:48,833 Excellent. 1676 02:02:49,100 --> 02:02:49,699 Thanks so much. 1677 02:02:50,066 --> 02:02:50,866 See you next time. 1678 02:02:51,500 --> 02:02:51,566 Okay. 1679 02:02:51,566 --> 02:02:52,066 Thank you.