1
00:00:03,374 --> 00:00:05,334
Welcome to MakeEdTech100.

2
00:00:06,173 --> 00:00:10,493
I'm LindyHoc, educator, K-12
edtech advisor, and your host.

3
00:00:11,354 --> 00:00:15,964
This is a podcast where we keep it real
about what actually works in classrooms.

4
00:00:16,453 --> 00:00:21,433
No hype, no overwhelm, just practical
strategies, honest stories, and

5
00:00:21,433 --> 00:00:24,634
tools that make a real difference
for teachers and students.

6
00:00:25,134 --> 00:00:28,401
So come along with me on
a journey to MakeEdTech100

7
00:00:33,031 --> 00:00:38,360
Today's guest was already quoted
on the podcast a few episodes ago.

8
00:00:38,681 --> 00:00:44,140
Adam Sparks is the source of use
case not existence when it comes

9
00:00:44,140 --> 00:00:46,531
to technology use in classrooms.

10
00:00:46,881 --> 00:00:51,650
So to continue my series on screen
time in schools, I knew I had to get

11
00:00:51,650 --> 00:00:53,780
Adam on for a great conversation.

12
00:00:54,140 --> 00:00:58,210
Adam is an English and social studies
teacher who recently finished his

13
00:00:58,210 --> 00:01:02,560
master's degree in learning design
and technology, and as part of his

14
00:01:02,560 --> 00:01:07,291
master's coursework, he built a digital
writing tool called Short Answer.

15
00:01:07,640 --> 00:01:08,800
It's really amazing.

16
00:01:08,800 --> 00:01:10,471
We'll talk about that more in the episode.

17
00:01:10,771 --> 00:01:16,440
It's so powerful that he now works
full-time developing and sharing Short

18
00:01:16,440 --> 00:01:19,240
Answer with schools and educators.

19
00:01:19,631 --> 00:01:21,260
Welcome to the podcast, Adam.

20
00:01:21,460 --> 00:01:24,430
Hey thanks for having me I'm very
excited to be here and chat with you Andy

21
00:01:24,942 --> 00:01:27,372
Tell us, what else do listeners
need to know about you

22
00:01:27,572 --> 00:01:27,952
I think you?

23
00:01:27,952 --> 00:01:32,792
got it I taught for seven years I went
to grad school I built what was supposed

24
00:01:32,792 --> 00:01:36,492
to be a master's project We got grant
funding and launched it and here we

25
00:01:36,492 --> 00:01:41,132
are building it three years later So
feel lucky to do the work I do I do it

26
00:01:41,132 --> 00:01:45,092
with my wife Alexa which is special and
unique and fun and I'm really passionate

27
00:01:45,292 --> 00:01:50,412
about e-education and technology in our
current moment and so I feel like I'm

28
00:01:50,412 --> 00:01:53,672
in the right place I'm I'm really on
fire by the work right now which is fun

29
00:01:54,573 --> 00:01:58,643
I named the podcast Make Ed
Tech 100 because that meaning

30
00:01:58,643 --> 00:02:01,743
is open to interpretation
of whatever it means to you.

31
00:02:01,743 --> 00:02:06,623
So I like to ask every guest to tell us
what does Make Ed Tech 100 mean to you?

32
00:02:06,823 --> 00:02:12,403
Oh man Well I think of 100 it's like
keep it 100 Like it's like 100 like

33
00:02:12,603 --> 00:02:16,013
keep Yeah, keep ed tech good I don't
know if that's I I equate it in my

34
00:02:16,013 --> 00:02:18,913
mind to good I don't know if that's
right or or wrong but yeah so keep

35
00:02:18,913 --> 00:02:20,733
educational technology high quality

36
00:02:20,893 --> 00:02:21,093
Yeah

37
00:02:21,237 --> 00:02:23,007
which is fitting for our current moment

38
00:02:23,373 --> 00:02:25,803
W- it is very fitting
for the current moment.

39
00:02:26,417 --> 00:02:29,037
So yeah that's where my
mind goes I don't know

40
00:02:29,083 --> 00:02:33,623
The current conversation is focusing
on quantity and not quality,

41
00:02:34,457 --> 00:02:34,827
Yes

42
00:02:34,913 --> 00:02:35,343
right?

43
00:02:35,463 --> 00:02:39,243
So that is super, super fitting
amongst the screen time debate

44
00:02:39,323 --> 00:02:40,943
you're a former classroom teacher.

45
00:02:41,523 --> 00:02:43,423
I didn't realize you taught social
studies as well, by the way.

46
00:02:43,464 --> 00:02:47,193
I knew you taught English, obviously,
The Writing Connection, and you're

47
00:02:47,193 --> 00:02:50,883
building an education product to
assist with writing instruction.

48
00:02:50,883 --> 00:02:52,504
Build, built and building.

49
00:02:52,683 --> 00:02:55,074
Building education products never ends.

50
00:02:56,067 --> 00:02:56,407
Right

51
00:02:56,603 --> 00:02:59,754
and present and future
tense, all of the above.

52
00:03:00,054 --> 00:03:03,843
So I love that you have
both of those perspectives.

53
00:03:04,214 --> 00:03:08,894
When you hear Utah passed a screen
time bill this legislative season,

54
00:03:09,223 --> 00:03:13,984
something like 15 other states had
legislations considering some form of

55
00:03:14,343 --> 00:03:17,394
screen time blanket ban or limitation.

56
00:03:17,394 --> 00:03:24,784
LAUSD also passed some sort of policy
or something about screen time at

57
00:03:24,784 --> 00:03:28,824
LAUSD in particular, so that's a school
district level, not a state level.

58
00:03:29,504 --> 00:03:31,404
What's your gut reaction to hearing that?

59
00:03:31,604 --> 00:03:35,574
I have a lot First I wanna say I
think it it's all coming from a

60
00:03:35,574 --> 00:03:37,314
good place It's people are really

61
00:03:37,480 --> 00:03:38,110
Agree.

62
00:03:38,284 --> 00:03:44,444
and have been for a while a-about like
cell phones and and social media and the

63
00:03:44,444 --> 00:03:48,894
impact that that has on kids and their
social lives and their overall health and

64
00:03:48,894 --> 00:03:53,794
wellbeing I get that and so I think those
concerns have been around for a while what

65
00:03:53,794 --> 00:03:57,034
I think unfortunately what's happening
is we're taking our fear and frustration

66
00:03:57,034 --> 00:04:01,554
and all the emotion that comes with that
and we're we're just kind of increasing

67
00:04:01,554 --> 00:04:06,064
we're just kind of applying that fear and
frustration to all technology and I think

68
00:04:06,064 --> 00:04:11,004
that is misguided And so I think most of
these laws that are either passed or in

69
00:04:11,004 --> 00:04:15,744
the process of you know being proposed
and debated um are are generally misguided

70
00:04:15,744 --> 00:04:18,604
because I think they're falling for this
kind of moment of moral panic around

71
00:04:18,604 --> 00:04:24,444
technology that it's inherently bad that's
directly contradicted by research showing

72
00:04:24,444 --> 00:04:29,234
positive effect sizes of of technology
use in classrooms for different use

73
00:04:29,234 --> 00:04:33,474
cases in different contexts so I think
we're ignoring research in the name of

74
00:04:33,634 --> 00:04:38,684
emotion and fear and I think a lot of
politicians are all too willing to cash

75
00:04:38,684 --> 00:04:42,274
in on that to score some points with their
constituencies it's pretty low-hanging

76
00:04:42,274 --> 00:04:47,064
fruit and it's pretty like acro it it's
increasingly it's it's cutting across

77
00:04:47,064 --> 00:04:50,634
political divides of like this fear of
technology And again it comes from a good

78
00:04:50,634 --> 00:04:54,034
place We wanna do what's best for kids But
unfortunately I think that fear has has

79
00:04:54,034 --> 00:04:57,174
gotten the best of us and we're starting
to push the pendulum way too far in one

80
00:04:57,174 --> 00:05:01,934
direction And in the process we're we're
increasingly stripping valuable resources

81
00:05:01,934 --> 00:05:06,504
out of the hands of kids and teachers and
that is as a former educator who had to

82
00:05:06,504 --> 00:05:10,424
fight tooth and nail to get devices in
my classroom and when I got those devices

83
00:05:10,424 --> 00:05:16,194
it it completely changed my quality of
instruction in my classroom I find it so

84
00:05:16,194 --> 00:05:20,194
so frustrating And it's hard for me to
talk about it now cause I'm building an

85
00:05:20,194 --> 00:05:23,904
ed tech tool and so people automatically
assume the worst of like you're just y-you

86
00:05:23,984 --> 00:05:27,414
the only reason you care about this is
cause it it's gonna hurt your bottom line

87
00:05:27,564 --> 00:05:31,594
I promise you that's not it I promise
you it's it it's more growing out of this

88
00:05:31,594 --> 00:05:35,604
experience Like the reason I wanted to
build ed tech to begin with was because

89
00:05:35,604 --> 00:05:38,764
of how profound of a positive impact it
had on my classroom I think most of these

90
00:05:38,764 --> 00:05:43,854
laws are misguided I'm hoping that we can
bend them towards positive outcomes for ed

91
00:05:43,854 --> 00:05:47,514
tech and I'm happy to talk more about what
that might look like But generally I think

92
00:05:47,514 --> 00:05:49,564
they're misguided to answer your question

93
00:05:49,714 --> 00:05:50,244
Agreed.

94
00:05:50,654 --> 00:05:56,584
Lumping everything, all technology,
all screen time under one umbrella

95
00:05:56,584 --> 00:06:01,734
is, it's very misguided, and there's a
lot of misinformation, disinformation,

96
00:06:01,734 --> 00:06:06,024
malinformation out there that's
really pushing that narrative.

97
00:06:06,024 --> 00:06:06,974
And I agree with you.

98
00:06:06,974 --> 00:06:11,714
I said, I think in both the first and
second episode I did in this series was

99
00:06:11,714 --> 00:06:15,314
just me sharing my thoughts on this,
and I said, "I think that screen time

100
00:06:15,314 --> 00:06:22,204
should be being evaluated," because
there is a lot of bad use of screen time.

101
00:06:22,614 --> 00:06:29,104
You know, if you're only using screens
as a reward or the students that work

102
00:06:29,104 --> 00:06:34,294
fast and need something to do because
they plowed through your learning

103
00:06:34,294 --> 00:06:38,184
tasks really fast and it's used as
some sort of, like, reward, is-- if

104
00:06:38,184 --> 00:06:39,684
that's the only way you're using tech.

105
00:06:39,684 --> 00:06:44,934
Or there's a huge difference
between digital curriculums and

106
00:06:44,934 --> 00:06:46,994
instructional technology tools.

107
00:06:47,314 --> 00:06:52,304
If you are only having students
working in digital curriculums where

108
00:06:52,304 --> 00:06:56,474
they're watching videos and answering
questions, they're not creating.

109
00:06:56,474 --> 00:07:02,554
They're a very consumption-based tech
that's very, very, very, very different

110
00:07:02,794 --> 00:07:06,344
than in true instructional tech tools.

111
00:07:06,544 --> 00:07:10,934
No I I totally agree with with everything
you just said and I it's it's a part

112
00:07:10,934 --> 00:07:14,074
of A larger point that I think is
really important It's it's like ed ed

113
00:07:14,074 --> 00:07:19,424
tech quote-unquote is not one thing
It's thousands of things It's it's

114
00:07:19,424 --> 00:07:23,014
what you're describing It's like it's
digital curricula it's instructional

115
00:07:23,014 --> 00:07:25,184
tools it's administrative tools it's

116
00:07:25,370 --> 00:07:26,570
A large umbrella

117
00:07:26,934 --> 00:07:31,574
It's it's a It's a huge umbrella and
we're lumping them we're lumping all of

118
00:07:31,574 --> 00:07:36,194
these interventions literally thousands
of interventions under one label and we're

119
00:07:36,654 --> 00:07:41,564
saying it is inherently bad because of
the medium and researchers So the most

120
00:07:41,564 --> 00:07:44,644
influential of the researchers that I know
of that have talked about this is John

121
00:07:44,644 --> 00:07:47,834
Hattie I know that's a name that a lot
of folks in education will recognize just

122
00:07:47,834 --> 00:07:51,584
because his meta meta-analysis work in
visible learning is very influential and

123
00:07:51,584 --> 00:07:56,974
he had a chapter on technology influences
in that book And in his most recent one he

124
00:07:57,004 --> 00:08:01,744
he literally said something to the effect
of like If there is to be a third version

125
00:08:01,744 --> 00:08:05,094
of this book we're not gonna have a
chapter on technology because it's absurd

126
00:08:05,094 --> 00:08:10,004
to refer to technology as an intervention
in and of itself It's a medium through

127
00:08:10,004 --> 00:08:14,144
which thousands of interventions can be
implemented and so we need to instead

128
00:08:14,144 --> 00:08:19,164
measure that Like the use cases and the
context and and the tools themselves

129
00:08:19,564 --> 00:08:23,043
I think that's a healthy perspective
and that's coming from arguably the

130
00:08:23,043 --> 00:08:26,234
most influential education researcher
of the last 20 years we're ignoring

131
00:08:26,234 --> 00:08:27,764
that advice in our current moment and

132
00:08:27,825 --> 00:08:28,255
Mm-hmm

133
00:08:28,284 --> 00:08:30,734
I'm talking about when I talk
about we're ignoring researchers

134
00:08:30,974 --> 00:08:33,494
in our current of moral panic and

135
00:08:33,494 --> 00:08:34,524
I find it very frustrating

136
00:08:34,724 --> 00:08:38,234
It's like saying, this is a good
analogy I have right in front of me.

137
00:08:38,454 --> 00:08:42,174
Two learning tools that I use all
the time when I'm teaching: a pen or

138
00:08:42,174 --> 00:08:44,604
writing utensil and Post-it Notes.

139
00:08:44,914 --> 00:08:50,614
It's like saying all pens are terrible,
and we should get rid of every pen

140
00:08:51,034 --> 00:08:56,264
in every classroom in the world,
and there's no good use of a pen.

141
00:08:57,204 --> 00:08:59,804
The that's literally what
this argument comes down to.

142
00:09:00,004 --> 00:09:01,944
Or Post-it Notes, all
Post-it Notes are bad.

143
00:09:02,414 --> 00:09:03,834
They're terrible for the environment.

144
00:09:04,054 --> 00:09:06,564
They're using trees to create them.

145
00:09:06,954 --> 00:09:11,094
We should never, ever, ever, ever use
a Post-it Note as part of learning

146
00:09:11,262 --> 00:09:11,472
It

147
00:09:11,714 --> 00:09:12,454
ever again.

148
00:09:12,522 --> 00:09:15,652
it's like it Exactly It's it's like
it's like saying like What's the

149
00:09:15,652 --> 00:09:18,982
effect size of a chalkboard Like
well depends on what you're using the

150
00:09:18,982 --> 00:09:21,102
chalkboard for So if you The larger

151
00:09:21,204 --> 00:09:21,444
Yeah.

152
00:09:21,882 --> 00:09:26,002
if you strip if you strip down the
arguments being made here what you get

153
00:09:26,002 --> 00:09:32,682
what it what it reduces down to eventually
is 1950s and 60s media ecology philosophy

154
00:09:33,352 --> 00:09:37,912
like Marshall McLuhan and others where
basically like the medium is the message

155
00:09:38,112 --> 00:09:41,852
The the medium itself is what determines
the message And then in this case they're

156
00:09:41,852 --> 00:09:46,722
saying like the medium of technology Is
inherently distracting you use technology

157
00:09:46,722 --> 00:09:51,302
you will be distracted and that's that's
it Kids can't handle it that philosophical

158
00:09:51,302 --> 00:09:56,192
perspective was written off years ago
it is technologically deterministic

159
00:09:56,192 --> 00:10:01,062
Like humans have free will We can bend
technology to serve our interests and

160
00:10:01,062 --> 00:10:04,682
serve our needs and we do that every day
in classrooms through very simple things

161
00:10:04,732 --> 00:10:09,042
like content filters or through basic
classroom management processes that I'm

162
00:10:09,042 --> 00:10:13,432
happy to get into and talk about and I
mean this literally like I I I think I

163
00:10:13,432 --> 00:10:16,902
was aware of this anti-edtech movement
pretty early in the game because I I

164
00:10:16,902 --> 00:10:21,412
went to a conference in oh my almost
three years ago in Delaware and Jar

165
00:10:21,452 --> 00:10:24,762
Dr Jared Cooney Horvath who is I would
say is the leader of the anti-edtech

166
00:10:24,832 --> 00:10:28,962
movement His book The Digital Delusion
is what is testimony to the US Senate

167
00:10:28,962 --> 00:10:31,222
a couple months ago like is what has

168
00:10:31,569 --> 00:10:34,849
Is he the one that compared it to anthrax?

169
00:10:34,939 --> 00:10:37,429
Was that, was that the
guy that compared... Okay.

170
00:10:37,579 --> 00:10:37,929
All right.

171
00:10:38,099 --> 00:10:42,179
I mentioned that in the first or
second episode that, yeah, there

172
00:10:42,179 --> 00:10:45,949
was a guy, I didn't remember his
name, but that compared tech use

173
00:10:45,949 --> 00:10:51,179
to anthrax, basically saying that
like anthrax, none of it is good.

174
00:10:51,379 --> 00:10:51,549
Yep

175
00:10:52,365 --> 00:10:52,775
Yeah.

176
00:10:52,999 --> 00:10:56,189
uh uh just the fact that someone's
willing to make that comparison I think

177
00:10:56,189 --> 00:11:00,079
gives you a sense of I I don't think
they should be taken seriously if you're

178
00:11:00,079 --> 00:11:02,779
making those types of comparisons in
front of the US Senate Like that's a

179
00:11:03,529 --> 00:11:05,709
silly and dangerous analogy to make

180
00:11:05,935 --> 00:11:06,405
It is.

181
00:11:06,405 --> 00:11:07,275
It is dangerous.

182
00:11:07,475 --> 00:11:08,235
It really is.

183
00:11:08,349 --> 00:11:09,699
It's a dangerous analogy.

184
00:11:10,562 --> 00:11:14,362
is Um and and he So he was the keynote
speaker at a conference I went to in

185
00:11:14,362 --> 00:11:18,342
Delaware like three years ago It was
ResearchEd and was the he was like

186
00:11:18,382 --> 00:11:23,032
the final speaker of the conference he
sp Half of his talk was was literally

187
00:11:23,032 --> 00:11:29,312
referencing like 50s and 60s philosophers
on media ecology It's like interesting

188
00:11:29,312 --> 00:11:35,052
work and it it's certainly something we
can debate amongst you know uh uh higher

189
00:11:35,052 --> 00:11:40,032
ed circles and and folks interested in
like philosophy and theory not research

190
00:11:40,072 --> 00:11:44,332
and it shouldn't be used to guide
education policy And it's insane to me

191
00:11:44,332 --> 00:11:49,072
that that's what we're using basically
it's disappointing on so many levels

192
00:11:49,162 --> 00:11:52,502
so yeah I'm happy to get into any of
the details within there but it it it's

193
00:11:52,502 --> 00:11:55,912
it's it's been distracting for me as
someone like building edtech It's it's

194
00:11:55,912 --> 00:11:59,652
been hard to focus on the work w in this
when when you're living through this

195
00:11:59,652 --> 00:12:02,862
moment where people are almost talking
about the work you do as if you work

196
00:12:02,862 --> 00:12:05,382
in like like the tobacco industry or

197
00:12:05,429 --> 00:12:05,939
Yes.

198
00:12:06,289 --> 00:12:07,089
Yes!

199
00:12:07,289 --> 00:12:08,009
It's crazy

200
00:12:08,359 --> 00:12:14,619
I shared in a couple episodes ago too
that I have had some very interesting

201
00:12:14,619 --> 00:12:18,659
and negative interactions in the past
few months especially when I tell

202
00:12:18,659 --> 00:12:25,499
people what I do, because they read
The Anxious Generation, and they think

203
00:12:25,499 --> 00:12:30,099
that no matter what, all technology
is now, you know, frying the neurons

204
00:12:30,099 --> 00:12:31,959
in our brain and that kind of thing.

205
00:12:32,549 --> 00:12:36,479
I do wanna dig into research
more because multiple things.

206
00:12:36,479 --> 00:12:42,019
One, you and your wife, 'cause your... So
your wife's did y- master's degree too.

207
00:12:42,099 --> 00:12:44,519
You both did your master's
degrees at Stanford, right?

208
00:12:44,849 --> 00:12:45,819
Kind of at the same time.

209
00:12:45,899 --> 00:12:50,359
And your, and your wife's degree
is more research education?

210
00:12:50,359 --> 00:12:51,689
Tell... I can't remember for sure.

211
00:12:51,942 --> 00:12:55,152
No that's a good co so yeah we did So
we both did our master's at the Stanford

212
00:12:55,152 --> 00:12:58,182
Graduate School of Education Mine was
in learning design and technology so

213
00:12:58,492 --> 00:13:01,802
it's more I don't wanna say it's more
pedagogy-focused The the folks in

214
00:13:01,802 --> 00:13:05,802
EDF will be mad about that label But
it it's more focused on like solving

215
00:13:05,802 --> 00:13:10,052
problems in education using the unique
affordances of technology to help

216
00:13:10,202 --> 00:13:13,772
solve those problems Because There,
are unique affordances of technology

217
00:13:13,772 --> 00:13:15,242
that can help solve those problems

218
00:13:15,849 --> 00:13:19,129
There there's a whole degree at...
There wouldn't be a deg- a master's

219
00:13:19,129 --> 00:13:24,649
degree at Stanford University
if there wasn't evidence backing

220
00:13:25,219 --> 00:13:27,799
the use of technology in learning

221
00:13:28,772 --> 00:13:33,532
I y yes thank you Like w we're writing
off an entire field of validated research

222
00:13:33,562 --> 00:13:36,492
I I I met with my program director just
yesterday to talk about this Her name's

223
00:13:36,492 --> 00:13:41,462
Dr Karin Forssell She literally wrote
the book on efficacy and ed tech efficacy

224
00:13:41,462 --> 00:13:43,812
and like how you structure a learning.

225
00:13:43,812 --> 00:13:47,392
tool to actualize best practices in
research so that's LDT She's wonderful

226
00:13:47,392 --> 00:13:50,162
That's Karin Forssell She's the program
director But then my wife studied

227
00:13:50,192 --> 00:13:52,032
education data science so she studied

228
00:13:52,062 --> 00:13:52,122
Mm.

229
00:13:52,286 --> 00:13:55,526
it's a lot more obviously data
science intensive and so it is a bit

230
00:13:55,526 --> 00:13:58,856
more research-oriented as opposed
to like practitioner-oriented

231
00:13:58,942 --> 00:13:59,272
Yep.

232
00:13:59,456 --> 00:14:04,176
And so she did some original research in
Dr Dora Demsky's lab on AI and its ability

233
00:14:04,176 --> 00:14:08,356
to provide feedback on student writing
and of bending AI feedback towards best

234
00:14:08,356 --> 00:14:13,916
practices and and feedback more generally
and so I I think we bring a really

235
00:14:13,916 --> 00:14:17,806
healthy lens to this current moment of
like AI and ed tech in education because

236
00:14:17,806 --> 00:14:21,156
she has a deeply technical understanding
of AI and machine learning and natural

237
00:14:21,156 --> 00:14:24,816
language processing and and I think I
have the pedagogy side of things cause

238
00:14:24,816 --> 00:14:28,626
I taught for seven years and I'm know
I've worked in schools And so I think

239
00:14:28,956 --> 00:14:33,336
we bring a healthy perspective to it And
I I training and education we received

240
00:14:33,336 --> 00:14:36,206
at Stanford like I was talking to Karin
about this yesterday I feel like it

241
00:14:36,206 --> 00:14:37,896
prepared me perfectly for this moment

242
00:14:38,382 --> 00:14:43,552
cause there is a whole body of research
to support technology as a meaningful

243
00:14:43,952 --> 00:14:47,192
technology is a broad label but like
technology can be used meaningfully

244
00:14:47,192 --> 00:14:50,712
to improve learning outcomes for
kids So I'm happy to fight for it

245
00:14:50,812 --> 00:14:56,772
You just gave, I feel like, your
credentials as one of the top people I

246
00:14:56,772 --> 00:15:03,032
know to refer to, to actually say what
does the research say, and make sure

247
00:15:03,032 --> 00:15:06,712
reciting hi-high causal research as well.

248
00:15:06,762 --> 00:15:11,082
You dug into it a little bit but
tell us more of like what does

249
00:15:11,082 --> 00:15:14,492
the research actually say about
technology used as part of the

250
00:15:14,492 --> 00:15:18,092
learning process in K-12 in particular?

251
00:15:18,492 --> 00:15:24,282
And I know, again, I'm lumping a big
umbrella of technology but especially

252
00:15:24,282 --> 00:15:30,842
looking to-towards the, you know, is it
frying the neurons of our brain argument.

253
00:15:31,042 --> 00:15:34,422
Yeah it can if it's used
really poorly If if you're just

254
00:15:34,422 --> 00:15:34,872
parking a

255
00:15:34,872 --> 00:15:38,582
kid in front of YouTube and being like
Hey dude have some have some Chromebook

256
00:15:38,582 --> 00:15:42,652
time Like of course yeah but it can
also be used the exact opposite way

257
00:15:42,742 --> 00:15:46,012
and have extremely positive outcomes
especially when you lean into the unique

258
00:15:46,012 --> 00:15:50,342
affordances of technology So it depends
on how you use it and I that's not an

259
00:15:50,342 --> 00:15:55,322
answer people like They want a blanket
statement of like bad black or white It's

260
00:15:55,322 --> 00:15:56,892
neither It's gray It it depends on how

261
00:15:57,019 --> 00:15:57,289
Yep.

262
00:15:57,492 --> 00:16:01,802
It it technology This is a Sal Khan quote
I use it all the time Technology is an

263
00:16:01,802 --> 00:16:07,102
amplification of human intent If you're
a really great teacher can use technology

264
00:16:07,102 --> 00:16:09,062
to make you a super great teacher

265
00:16:09,459 --> 00:16:09,669
Yep.

266
00:16:10,019 --> 00:16:12,209
Give you superpowers, what I always say.

267
00:16:12,929 --> 00:16:13,429
Yeah

268
00:16:14,302 --> 00:16:17,552
Research supports that If if you're
if you're a burnout teacher you don't

269
00:16:17,552 --> 00:16:20,592
wanna be in the classroom anymore you're
just kinda doing it cause you whatever

270
00:16:20,592 --> 00:16:24,392
five more years and you can get to
retirement you just kinda roll in and and

271
00:16:24,392 --> 00:16:27,282
technology's a great way to just kinda
park the kids on something and keep them

272
00:16:27,282 --> 00:16:29,972
busy you can do that too And and Yeah.

273
00:16:29,972 --> 00:16:33,062
kids can fry their neurons just doing
whatever they want on the internet

274
00:16:33,702 --> 00:16:36,542
it depends on how the tech is used
and I think the research supports

275
00:16:36,542 --> 00:16:40,262
that i-in terms of like talking about
blanket research kind of writ large

276
00:16:40,262 --> 00:16:43,802
the closest thing you can get to that
are meta-analyses cause they look at

277
00:16:43,802 --> 00:16:49,892
like a a huge swath of studies within
different domains so that's why I cite

278
00:16:50,112 --> 00:16:54,802
the John Hattie research cause he's
looked at like for example he looked

279
00:16:54,802 --> 00:17:00,102
at the use of technology and its use to
provide feedback student work and found

280
00:17:00,102 --> 00:17:04,312
an effect size of I think 0.5 standard
deviations which is a large effect

281
00:17:04,564 --> 00:17:06,154
There's lo-- Yes, very large.

282
00:17:06,354 --> 00:17:11,114
Largely positive so that is one simple
use case a positive use case And and

283
00:17:11,114 --> 00:17:14,404
this is this You don't need research to
tell you this if you've used technology

284
00:17:14,404 --> 00:17:19,714
in the classroom It's like well I could
manually collect 120 papers that kids

285
00:17:19,714 --> 00:17:23,074
have handwritten and spend my weekend
reading through it and hopefully I

286
00:17:23,074 --> 00:17:26,844
can get them the feedback back by like
Next Thursday and they handed it in on

287
00:17:26,885 --> 00:17:30,885
Yeah

288
00:17:30,974 --> 00:17:35,294
more than likely because I'm having to
read through maybe like whatever like 20

289
00:17:35,294 --> 00:17:39,584
of like 120 of these and try to provide
comments and things and by the time I

290
00:17:39,584 --> 00:17:43,654
get them the feedback back will they
even remember what we were working on or

291
00:17:43,864 --> 00:17:48,234
I can use a wonderful new AI tool like
Short Answer or pick your favorite cause

292
00:17:48,234 --> 00:17:52,124
there's many tools that will do this
now And the kids submit their let's say

293
00:17:52,124 --> 00:17:55,814
exit ticket they can get feedback on the
quality of that of their response And,

294
00:17:55,814 --> 00:17:59,944
explanation before they leave the
classroom And so they they can then use

295
00:17:59,944 --> 00:18:03,174
that feedback to improve what they're
you know their understanding again before

296
00:18:03,174 --> 00:18:07,064
they leave the classroom that's one
simple use case that when you extrapolate

297
00:18:07,064 --> 00:18:11,164
that out and look at studies across many
different contexts again effect size

298
00:18:11,164 --> 00:18:15,714
of 0.5 like super positive are other
domains where we could look at There was

299
00:18:15,714 --> 00:18:20,294
recently a meta-analysis done by Rebecca
Silverman at Stanford She looked at This

300
00:18:20,294 --> 00:18:23,684
was in the context of early childhood
literacy so teaching kids how to read

301
00:18:23,714 --> 00:18:27,374
and write generally positive effect sizes
It was a very nuanced study and there's

302
00:18:27,374 --> 00:18:31,174
like literally it comes down to like
specific features and apps that do or d

303
00:18:31,204 --> 00:18:36,414
don't support early childhood literacy
outcomes but effect sizes ranging from

304
00:18:37,294 --> 00:18:43,304
like 0.2 to 0.3 which is like generally
positive now this is a whole other

305
00:18:43,304 --> 00:18:46,854
area that we could get into if you want
to around what qualifies as a large

306
00:18:46,854 --> 00:18:49,964
effect size Cause that's another thing
that the anti-ed tech movement is is

307
00:18:49,964 --> 00:18:55,894
doing at the movement sort of positive
research about technology cause I've

308
00:18:56,230 --> 00:19:00,230
And

309
00:19:00,604 --> 00:19:04,464
Those are large effect sizes So there's
a researcher at Brown his name's Matthew

310
00:19:04,464 --> 00:19:09,144
Craft he has pushed back on this narrative
that has grown up in education in the last

311
00:19:09,474 --> 00:19:14,744
15 20 years that came from John Hattie
that a large effect size is is anything

312
00:19:14,964 --> 00:19:19,174
0.4 standard deviations and above If
it's below 0.4 standard deviations and

313
00:19:19,174 --> 00:19:23,234
below it's a small effect size actually
Jared Cooney Horvath used that reference

314
00:19:23,234 --> 00:19:28,734
point in his testimony to the Senate
He he literally subtracted that hinge

315
00:19:28,734 --> 00:19:32,863
point to take away from edu educational
technologies effect size which is

316
00:19:32,863 --> 00:19:36,904
absolutely insane and total pseudoscience
But but people have pushed back on that

317
00:19:36,904 --> 00:19:41,253
0.4 standard deviation specifically
Matthew Craft of like if you look at

318
00:19:42,023 --> 00:19:46,043
just at meta-analyses like the average
effect size is really small for education

319
00:19:46,043 --> 00:19:50,633
interventions So actually like anything
that's I think is is 0.2 and above is

320
00:19:50,633 --> 00:19:54,153
actually quite large in education like in
terms of what a positive effect size would

321
00:19:54,153 --> 00:19:58,293
be And so I try to explain that to folks
and it's just like Well no no no just

322
00:19:58,293 --> 00:20:00,773
no That that that evidence doesn't count

323
00:20:01,580 --> 00:20:02,030
Mm.

324
00:20:02,313 --> 00:20:05,353
that's what makes me think that we're in
a moment of moral panic cause it's like

325
00:20:05,613 --> 00:20:09,813
these are researchers Like these these
are people that like Spend their time

326
00:20:09,813 --> 00:20:11,543
reading peer-reviewed research in journals

327
00:20:11,580 --> 00:20:14,080
Yeah, their life is
literally dedicated to this.

328
00:20:14,080 --> 00:20:17,130
Like, this is what they do
with their life every day.

329
00:20:17,923 --> 00:20:19,163
And they're

330
00:20:19,310 --> 00:20:21,500
to them, like, who do we listen to?

331
00:20:21,700 --> 00:20:21,770
Yeah.

332
00:20:22,103 --> 00:20:26,373
Exactly And man there's a whole other
post Substack post coming soon about

333
00:20:26,373 --> 00:20:30,143
like reform needs to come from all
sides Yes at a policy level Yes at an

334
00:20:30,143 --> 00:20:33,593
ed tech level implementation level It
also needs to come at a research level

335
00:20:33,593 --> 00:20:39,323
because research community in education
is a mess and leads to unclear guidance

336
00:20:39,323 --> 00:20:42,673
to schools on what is best practice
And it's partly why we're in the moment

337
00:20:42,673 --> 00:20:46,983
that we're in and arguing over what a
what qualifies as a large effect size

338
00:20:46,983 --> 00:20:51,033
is a perfect example of that So that's a
really long-winded way of saying research

339
00:20:51,033 --> 00:20:55,303
supports technology in schools when used
well it's just that we're ignoring it

340
00:20:55,503 --> 00:20:58,713
So Matthew Craft, just to make
sure I understand correctly, his

341
00:20:58,713 --> 00:21:04,623
argument is, so like Hattie's
saying 0.5 effect size and above,

342
00:21:04,823 --> 00:21:05,233
Point four

343
00:21:05,280 --> 00:21:07,860
saying actually... Oh, 0.4, sorry, 0.4.

344
00:21:08,230 --> 00:21:12,060
Matthew Craft is saying that
actually in education and the way

345
00:21:12,060 --> 00:21:17,780
education studies work, that we
can go down to a 0.2 effect size.

346
00:21:18,773 --> 00:21:22,993
Yeah and his argument there which I think
is real cause he's replicated it now twice

347
00:21:23,223 --> 00:21:28,723
is basically that Hattie threw everything
in including like correlational studies

348
00:21:28,723 --> 00:21:32,923
which tend to have higher effect sizes He
threw in studies that where the measures

349
00:21:32,923 --> 00:21:36,003
that were being used were created for
the specific test that was being done

350
00:21:36,003 --> 00:21:40,053
as opposed to like broader measures like
standardized tests and things like that

351
00:21:40,553 --> 00:21:44,013
Which tend so you tend to get higher
effect sizes So basically to to put it

352
00:21:44,013 --> 00:21:48,293
into layman's terms like Hattie threw in
all the research It's kind of like a hot

353
00:21:48,293 --> 00:21:51,803
dog It's easy to consume but you and and
tasty but you don't wanna know what's in

354
00:21:51,803 --> 00:21:55,833
it whereas Matthew Kraft only looked at
like high quality like randomized control

355
00:21:55,833 --> 00:22:00,533
trial-level studies And when you look
at that majority like almo I think it's

356
00:22:00,533 --> 00:22:06,083
like 33 it's like in the 30s percent of
studies have no effect on outcomes Cause

357
00:22:06,083 --> 00:22:07,633
it's really hard to have an effect on

358
00:22:07,655 --> 00:22:08,105
hard

359
00:22:08,673 --> 00:22:10,513
cause education's really hard.

360
00:22:10,513 --> 00:22:11,193
There's so

361
00:22:11,205 --> 00:22:11,475
Yeah.

362
00:22:11,863 --> 00:22:12,433
at play in

363
00:22:12,465 --> 00:22:16,455
So many fac- yeah, it's really hard
to say, like, this is the factor that

364
00:22:16,715 --> 00:22:19,995
improved student learning outcomes
or this is the factor that didn't.

365
00:22:20,295 --> 00:22:25,465
So that's another reason why just having
this blanket technology is the reason that

366
00:22:25,895 --> 00:22:29,815
test scores are down, and technology is
the reason that engagement is down, is

367
00:22:30,365 --> 00:22:33,635
really, really flawed thinking, because,

368
00:22:34,253 --> 00:22:34,643
Subscribe

369
00:22:34,675 --> 00:22:38,845
we're, we're not taking into
account testing becoming more

370
00:22:38,845 --> 00:22:40,575
and more a thing in schools.

371
00:22:40,895 --> 00:22:45,025
The pandemic causing
huge amounts of trauma.

372
00:22:45,055 --> 00:22:49,305
I just read a whole book on trauma,
and now I'm on the trauma train,

373
00:22:49,645 --> 00:22:54,775
because I knew this, but this book
was so enlightening to me at how much

374
00:22:55,075 --> 00:23:02,195
trauma affects our brain, and every
brain inputs trauma differently.

375
00:23:02,605 --> 00:23:06,485
Sometimes what is, you know, of like big
T trauma, little T trauma, people say.

376
00:23:06,805 --> 00:23:10,615
Sometimes what one brain interprets
as little T trauma, another

377
00:23:10,615 --> 00:23:12,425
brain interprets as big T trauma.

378
00:23:12,855 --> 00:23:17,835
And the pandemic, I believe, caused
trauma in every single one of us,

379
00:23:18,095 --> 00:23:24,195
adults that have fully developed
brains, and we haven't even begun to

380
00:23:24,645 --> 00:23:30,735
think about the trauma that happened
to the kids from 2020 to 2022.

381
00:23:31,365 --> 00:23:35,565
And I think there's a lot of
brains that had a lot of big T

382
00:23:35,565 --> 00:23:39,835
trauma, and when you have big T
trauma, your brain just shuts down.

383
00:23:40,455 --> 00:23:42,365
But that's not part of the conversation.

384
00:23:42,555 --> 00:23:42,705
Yeah

385
00:23:42,876 --> 00:23:46,006
The correlation does not equal
causation argument, right?

386
00:23:46,226 --> 00:23:49,066
Like, we're saying, "Oh,
I listened to this NPR."

387
00:23:49,096 --> 00:23:51,576
Actually, honestly, I
didn't make it through.

388
00:23:52,526 --> 00:23:54,846
I need to go back to it,
but I got so frustrated.

389
00:23:54,846 --> 00:23:56,106
It was an NPR segment.

390
00:23:56,486 --> 00:23:59,766
I can't remember the exact, And
usually I really-- I love NPR.

391
00:23:59,766 --> 00:24:01,496
My husband's an NPR addict.

392
00:24:01,526 --> 00:24:02,786
He's the one that sent it to me.

393
00:24:03,296 --> 00:24:06,086
I can't remember the name of the
segment, and usually I really like this

394
00:24:06,086 --> 00:24:11,226
journalist and I listen to it, but it
was so one-sided and it was the, this

395
00:24:11,226 --> 00:24:18,736
principal arguing that at the same
time that we put Devices in schools and

396
00:24:18,736 --> 00:24:22,936
that her middle school went one-to-one
Chromebooks was the same time that test

397
00:24:22,936 --> 00:24:25,136
scores and engagement started to go down.

398
00:24:25,256 --> 00:24:29,216
No, that's correlation-- Correlation
does not equal causation.

399
00:24:29,216 --> 00:24:33,566
You cannot say because this
happened, it's because of this.

400
00:24:33,985 --> 00:24:38,015
Yeah Yeah Yeah Yeah and it it's wild
that we're prioritizing correlational

401
00:24:38,015 --> 00:24:39,635
studies over causal studies

402
00:24:39,846 --> 00:24:40,756
Yes.

403
00:24:41,025 --> 00:24:44,715
that that that kinda shows me this is
about moral panic and emotion more than

404
00:24:44,715 --> 00:24:48,775
it is about evidence cause yeah I mean
what else correlates correlation we could

405
00:24:48,775 --> 00:24:52,315
draw is that around that same time kids
started spending seven to eight hours

406
00:24:52,395 --> 00:24:54,485
outside of school on their cell phone or

407
00:24:54,766 --> 00:24:55,036
Yep.

408
00:24:55,095 --> 00:24:55,725
devices

409
00:24:56,136 --> 00:24:56,526
Yep.

410
00:24:56,615 --> 00:25:00,185
of going home to read Harry Potter we're
going home to scroll through TikTok That

411
00:25:00,185 --> 00:25:05,225
may also play into it into this as as you
mentioned as does the trauma of a pandemic

412
00:25:05,305 --> 00:25:10,295
as does I mean a generation of kids
that was raised on reading methodology

413
00:25:10,295 --> 00:25:11,745
that has increasingly been questioned

414
00:25:11,745 --> 00:25:12,565
with the science of reading

415
00:25:12,565 --> 00:25:15,765
movement we could go down a list
of 1,000 different reasons why test

416
00:25:15,765 --> 00:25:18,835
scores are are likely declining cause
it is real that they're declining

417
00:25:19,356 --> 00:25:19,806
Yes.

418
00:25:20,145 --> 00:25:24,205
but drawing this simple correlation
of like we we bought computers

419
00:25:24,385 --> 00:25:25,795
and scores went down It's it's

420
00:25:25,946 --> 00:25:26,286
Yep.

421
00:25:26,825 --> 00:25:30,675
correlation I you can imagine So it's
it's not I don't think it's strong

422
00:25:30,675 --> 00:25:33,825
evidence and I think the people that
would tell you that they support this

423
00:25:33,825 --> 00:25:39,055
like science of insert your favorite part
of education science of movement tell

424
00:25:39,055 --> 00:25:42,425
you we need causal evidence To support
interventions if we're gonna put them

425
00:25:42,425 --> 00:25:45,975
in schools And simultaneously they're
ignoring their own advice in this moment

426
00:25:46,025 --> 00:25:49,565
and being like No actually correlational
evidence is enough it's it's pretty

427
00:25:49,565 --> 00:25:53,975
absurd So it it's frustrating but I
am hopeful and I wrote a Substack post

428
00:25:53,975 --> 00:25:57,925
about this just yesterday I am hopeful
that like it can this current moment

429
00:25:57,925 --> 00:26:02,525
can push for some positive reform cause
I do think edtech does need some reform

430
00:26:02,886 --> 00:26:03,446
Agreed.

431
00:26:03,906 --> 00:26:06,696
We wouldn't be here having
this conversation if it didn't

432
00:26:06,696 --> 00:26:09,416
need reform, is my opinion.

433
00:26:09,685 --> 00:26:12,685
no it does need reform Th-there are
some flaws in the industry that should

434
00:26:12,685 --> 00:26:17,375
be addressed but stripping resources
out of the hands of teachers is worst

435
00:26:17,375 --> 00:26:20,505
possible way to go about doing this but
unfortunately it's just the easiest thing

436
00:26:20,505 --> 00:26:21,935
to legislate so it's what we're doing

437
00:26:22,036 --> 00:26:22,376
Yep.

438
00:26:22,575 --> 00:26:22,935
here we are

439
00:26:23,035 --> 00:26:27,075
I also wanna talk about efficacy,
'cause I know this is something I've

440
00:26:27,075 --> 00:26:31,445
ta- I-- we've had conversations about
this, and ed tech efficacy in general.

441
00:26:31,445 --> 00:26:34,875
If you're not familiar with that
terminology, it basically, efficacy

442
00:26:34,875 --> 00:26:40,155
measures whether a tool, in this case
we're talking digital learning tools,

443
00:26:40,155 --> 00:26:43,425
improve student outcomes or not.

444
00:26:43,875 --> 00:26:48,505
There's a huge conversation in the
ed tech world about how do we measure

445
00:26:48,505 --> 00:26:52,865
ed tech effic-efficacy, and how
do we actually know if an ed tech

446
00:26:52,865 --> 00:26:55,625
tool is improving student outcomes?

447
00:26:55,775 --> 00:27:00,235
So that's what makes this really really
hard and that's why people go to school

448
00:27:00,235 --> 00:27:03,185
for eight years to study this and then
spend their lives trying to figure this

449
00:27:03,185 --> 00:27:09,195
out it's educational re education is a
social science Social science is hard

450
00:27:09,195 --> 00:27:13,785
cause there are so many variables that
you try to control are almost impossible

451
00:27:13,785 --> 00:27:18,475
to control in the way that if you wanna
compare it to medical science um like that

452
00:27:18,475 --> 00:27:22,205
that you can control in a in a human's
body as opposed to like a classroom

453
00:27:22,821 --> 00:27:23,281
Right

454
00:27:23,615 --> 00:27:27,125
speaking to answer your question how do
you measure efficacy The gold standard

455
00:27:27,125 --> 00:27:32,145
of efficacy in technology and a lot of
education in general is a randomized

456
00:27:32,145 --> 00:27:36,555
control trial or if below that it's
like a quasi-experimental study At a

457
00:27:36,555 --> 00:27:39,215
very broad level we're gonna paint with
a broad brush here cause we could get

458
00:27:39,215 --> 00:27:43,655
into nerdy specifics but I don't think
we need to turn this into a research

459
00:27:43,655 --> 00:27:48,025
methods grad school class So like at
a basic level it's you go to a a large

460
00:27:48,025 --> 00:27:52,135
subset of kids ideally Some of them get
the intervention So let's just go with

461
00:27:52,135 --> 00:27:55,605
Short Answer the platform we're building
These kids will use Short Answer during

462
00:27:55,605 --> 00:27:59,965
this period Let's And this subset of kids
that's been randomly selected a a large

463
00:27:59,965 --> 00:28:02,835
number of them they're not going to use
Short Answer They're gonna do business

464
00:28:02,835 --> 00:28:07,485
as usual will these We'll we'll come up
with a different plan et cetera et cetera

465
00:28:08,355 --> 00:28:12,325
We'll do like a pre/post let's say So
we'll look at their writing scores on the

466
00:28:12,325 --> 00:28:15,725
state test the year before and then we'll
look on the writing scores on the state

467
00:28:15,725 --> 00:28:18,955
test the year after and we'll compare the
control group the the kids that didn't

468
00:28:18,955 --> 00:28:22,855
use Short Answer to the kids that did the
experimental group we will subtract them

469
00:28:23,445 --> 00:28:27,935
and then we'll divide it by the standard
deviation within the test scores find

470
00:28:27,935 --> 00:28:30,875
an effect size So when I say effect size
that's what I'm referring to It's like a

471
00:28:30,875 --> 00:28:35,455
0.2 a 0.3 And like what qualifies as big
Well if I would say if you I'm gonna go

472
00:28:35,455 --> 00:28:40,675
with Matthew Kraft if the deviation is
0.2 or above you have a a medium to large

473
00:28:40,675 --> 00:28:45,495
effect size If it's 0.05 and above it's
it's a smaller effect size Sorry If it's

474
00:28:45,495 --> 00:28:50,795
0.2 and above it's large If it's 0.05
to 0.2 it's medium If it's below 0.05

475
00:28:50,795 --> 00:28:55,825
it's it's small and that's a randomized
control trial Now h I need to complicate

476
00:28:55,825 --> 00:29:00,015
that so hooray we got our standard you
know standard deviation and it's wonderful

477
00:29:00,335 --> 00:29:02,895
The problem with that in educational
technology is what you talked about at the

478
00:29:02,895 --> 00:29:06,755
beginning of the podcast which is edtech
moves fast and it changes a lot Like

479
00:29:06,755 --> 00:29:10,535
a product is never built You're always
building it so the problem with that is

480
00:29:10,865 --> 00:29:14,685
okay we we run a randomized control trial
It's great We we 0.5 standard deviation

481
00:29:14,685 --> 00:29:18,975
effect size Amazing You know Look at
this W we improved learning outcomes

482
00:29:19,585 --> 00:29:23,505
Well the problem with that is if in the
process of doing that research you also

483
00:29:23,505 --> 00:29:27,805
added 20 new features to your platform
that completely changed the what you do

484
00:29:28,411 --> 00:29:29,611
All, all the time.

485
00:29:30,245 --> 00:29:30,915
which happens all the

486
00:29:31,271 --> 00:29:31,791
Yeah.

487
00:29:31,925 --> 00:29:33,905
technology And that's a positive thing

488
00:29:34,021 --> 00:29:35,031
And it's a good thing.

489
00:29:35,111 --> 00:29:35,711
Yeah.

490
00:29:35,785 --> 00:29:39,815
a great thing Like if we have a new
whatever new research comes out that like

491
00:29:40,015 --> 00:29:44,295
actually if you do it this way it's better
we can quickly change it right If you

492
00:29:44,295 --> 00:29:48,655
have a textbook good luck Like that thing
ain't getting updated for a decade Like

493
00:29:48,961 --> 00:29:49,231
Yep.

494
00:29:49,495 --> 00:29:50,335
are so slow

495
00:29:50,901 --> 00:29:51,251
Yep.

496
00:29:51,315 --> 00:29:54,315
positive affordance One of the
downsides is that you end up

497
00:29:54,315 --> 00:29:57,765
measuring tools that no longer exist
because like it changed over time

498
00:29:58,085 --> 00:29:58,455
And so the

499
00:29:58,455 --> 00:30:02,695
best advice I've seen on this this comes
from who Molly Zimmerman and Jennifer

500
00:30:02,695 --> 00:30:06,215
Carolan They wrote an article that we read
in grad school at Stanford I think the

501
00:30:06,215 --> 00:30:10,425
healthiest approach to efficacy in edtech
is a portfolio approach where you build

502
00:30:10,425 --> 00:30:16,275
out multiple different studies of varying
sizes and quality over time because that

503
00:30:16,275 --> 00:30:20,955
better enables you to track your product
as it changes over time But just as

504
00:30:20,955 --> 00:30:25,885
importantly it allows you to measure your
impact in different contexts over time

505
00:30:26,095 --> 00:30:31,000
Cause the challenge of edtech that it's
used at scale Like we've got kids in rural

506
00:30:31,030 --> 00:30:36,420
West Texas using Short Answer and we've
got kids in urban Beijing using Short

507
00:30:36,420 --> 00:30:40,220
Answer Those are very different contexts
And so the way that tool works and is used

508
00:30:40,220 --> 00:30:44,370
in those contexts is probably completely
different you need a smattering of

509
00:30:44,370 --> 00:30:49,720
different research into a portfolio over
time to speak to those different contexts

510
00:30:49,720 --> 00:30:54,030
because education is deeply contextual so
like the randomized control trial is the

511
00:30:54,030 --> 00:30:59,170
gold level you know whatever the the high
the highest bar o of efficacy And then

512
00:30:59,170 --> 00:31:03,330
I suppose a meta-analysis of randomized
control trials would then be the only

513
00:31:03,330 --> 00:31:06,270
thing that would be above that but for an
ed tech company because you're constantly

514
00:31:06,270 --> 00:31:09,200
changing I think that portfolio approach
is healthy And that's why if you go to our

515
00:31:09,200 --> 00:31:12,900
website sorry for the shameless plug if
if you go to myshortanswer.com and click

516
00:31:12,900 --> 00:31:16,060
on the Our Pedagogy page there's a little
button that says See our portfolio of

517
00:31:16,060 --> 00:31:21,510
efficacy everything ranging from a logic
model which is basically just like Here's

518
00:31:21,510 --> 00:31:25,020
what our app does Here's the research
that supports every different part of

519
00:31:25,020 --> 00:31:29,100
our app and and why it does what it does
That doesn't mean that our app works

520
00:31:29,100 --> 00:31:33,190
it just means that there's at least a a
theoretical explanation of why it might

521
00:31:33,190 --> 00:31:36,070
work and then you'll see our learner
study that we ran as a part of my master's

522
00:31:36,070 --> 00:31:39,430
capstone project and then you'll see case
studies that we put together based on

523
00:31:39,430 --> 00:31:44,370
surveys and triangulation of interviews
and several other factors And what we're

524
00:31:44,370 --> 00:31:48,610
doing is slowly trying to build up towards
the causational evidence that I just

525
00:31:48,690 --> 00:31:51,980
talked about like a randomized control
trial but we haven't got there yet because

526
00:31:52,010 --> 00:31:54,370
RCTs are expensive and take forever

527
00:31:54,370 --> 00:31:54,960
and for a small

528
00:31:54,960 --> 00:31:58,380
company like us where we're still building
a product and changing things dramatically

529
00:31:58,380 --> 00:32:01,910
from day to day it doesn't make sense So
I think that portfolio approach is healthy

530
00:32:02,390 --> 00:32:07,410
and I think here's a healthy reform idea
require that an ed tech product at least

531
00:32:07,410 --> 00:32:11,780
have a logic model or theory of change
in order to be used in schools It's a

532
00:32:11,780 --> 00:32:16,090
very simple way to try to bend adoption
towards efficacy and towards quality

533
00:32:16,860 --> 00:32:18,770
opposed to what we were talking about
at the beginning of the conversation

534
00:32:19,080 --> 00:32:24,160
where it's like about the medium or just
about X number of minutes per day or

535
00:32:24,160 --> 00:32:27,890
like silly things that aren't aligned
with research that could be a very simple

536
00:32:27,890 --> 00:32:32,780
step forward towards improved focus on
quality that was a long-winded response

537
00:32:32,980 --> 00:32:34,680
but this was literally my master's so

538
00:32:35,006 --> 00:32:35,066
Yeah,

539
00:32:35,250 --> 00:32:35,370
y

540
00:32:35,476 --> 00:32:36,006
that

541
00:32:36,080 --> 00:32:36,390
hit on a

542
00:32:36,406 --> 00:32:37,466
was fantastic.

543
00:32:37,470 --> 00:32:38,610
I like talking about

544
00:32:39,416 --> 00:32:42,726
That was fantastic, and I love
that you noted that it, it's also

545
00:32:42,726 --> 00:32:44,396
so tricky with ed tech research.

546
00:32:44,426 --> 00:32:51,081
It's not like researching a
textbook-based curriculum That is static.

547
00:32:51,631 --> 00:32:56,661
And I, I'll, I tell teachers this,
I tell ed tech companies this: the

548
00:32:56,661 --> 00:33:00,771
small, tiny little things are what
make or break an ed tech product.

549
00:33:00,971 --> 00:33:03,361
Like, the teeny tiniest little details.

550
00:33:03,661 --> 00:33:10,081
So you could make a really small
change, a UI change, a UX change, a

551
00:33:10,081 --> 00:33:14,591
design change, a pedagogical change
in a product, and it can completely

552
00:33:14,591 --> 00:33:19,081
make or break the teacher using it,
how the teacher uses it, and that can

553
00:33:19,081 --> 00:33:26,671
completely change a study that was
done prior or post that change, or both

554
00:33:27,281 --> 00:33:28,331
You're 100 right?

555
00:33:28,371 --> 00:33:31,221
and I've never had more appreciation
for that concept than I do now

556
00:33:31,441 --> 00:33:32,531
as someone building this stuff at

557
00:33:32,903 --> 00:33:33,353
I bet.

558
00:33:34,331 --> 00:33:40,151
if we choose to make this little user
interface element a tab instead of a

559
00:33:40,151 --> 00:33:46,231
toggle that can have a profound impact at
scale So yeah it's it's fascinating And

560
00:33:46,231 --> 00:33:50,111
this is what I love about the work You
go from 10,000-foot level conversations

561
00:33:50,111 --> 00:33:54,931
about what qualifies as an effect size
a positive effect size in meta-analyses

562
00:33:54,931 --> 00:33:59,341
and and then you zoom all the way into
like what should the radius of this

563
00:34:00,211 --> 00:34:04,491
of the corner of this button like how
Should it be a three radius or a five

564
00:34:04,491 --> 00:34:10,531
radius when I design it in Figma So it's
like it's granular and it's it's it's

565
00:34:10,531 --> 00:34:14,021
universal all at the same time and it's
fascinating work cause it all matters

566
00:34:14,953 --> 00:34:21,553
Yeah, and the more you get in and use
and explore ed tech tools, the more you

567
00:34:21,553 --> 00:34:25,403
start to notice those little tiny details.

568
00:34:25,733 --> 00:34:31,513
For me, when I'm evaluating an ed tech
product, I look for those little tiny

569
00:34:31,513 --> 00:34:36,423
details, and that's a big indicator
to me of the quality of the product.

570
00:34:37,343 --> 00:34:39,763
Last question on this topic,
and then we'll move on.

571
00:34:39,963 --> 00:34:45,223
If you are a school leader, a teacher
listening to this, and you're like,

572
00:34:45,253 --> 00:34:52,433
"I wanna make sure that I am using
high-quality ed tech tools in my

573
00:34:52,433 --> 00:34:56,503
classroom," or a school leader that's
like, "I wanna make sure I'm putting

574
00:34:56,503 --> 00:35:00,733
high-quality ed tech tools in the
hands of my teachers and students,"

575
00:35:00,733 --> 00:35:02,133
what would be your advice to them?

576
00:35:02,133 --> 00:35:03,013
What would you tell them?

577
00:35:03,113 --> 00:35:05,733
First I would take matters
into your own hands and what I

578
00:35:05,733 --> 00:35:08,933
mean by that is if a teacher's
requesting to use a tool go to the

579
00:35:09,094 --> 00:35:09,184
Not

580
00:35:09,423 --> 00:35:13,523
look at it Do they have a page or
somewhere where they're explaining

581
00:35:13,693 --> 00:35:16,453
the research supporting their tool
And when I say research I mean

582
00:35:16,453 --> 00:35:20,783
research What what you'll find is
there are a lot of tools that they

583
00:35:21,104 --> 00:35:22,704
a white paper.

584
00:35:22,734 --> 00:35:25,664
No, not a white paper.

585
00:35:25,713 --> 00:35:29,753
a lot of vibey hand-waving that happens
with like our learning principles and

586
00:35:29,753 --> 00:35:33,293
it's just like this sort of like vague
explanation of like constructivism

587
00:35:33,453 --> 00:35:36,773
It's like okay that's nice but
that's not research That that's just

588
00:35:36,824 --> 00:35:37,124
Yeah

589
00:35:37,743 --> 00:35:37,883
That's

590
00:35:37,914 --> 00:35:38,794
That's a philosophy.

591
00:35:38,863 --> 00:35:38,943
I

592
00:35:39,154 --> 00:35:39,394
you're

593
00:35:39,413 --> 00:35:39,573
It's a

594
00:35:39,734 --> 00:35:43,214
aligning your tool to an
educational philosophy, which is

595
00:35:43,214 --> 00:35:45,174
not bad, but it's not research.

596
00:35:45,614 --> 00:35:45,854
Yeah.

597
00:35:46,098 --> 00:35:50,828
It's not bad But it's like what are the
specific things your app is doing and are

598
00:35:50,828 --> 00:35:54,458
those things supported by research And
that's called the it's called theory of

599
00:35:54,504 --> 00:35:54,624
Common

600
00:35:54,778 --> 00:35:57,438
it's called logic model But that
to me that's that's the baseline

601
00:35:57,438 --> 00:36:00,378
of efficacy And under the Every
Student Succeeds Act that was

602
00:36:00,434 --> 00:36:00,554
Sense

603
00:36:00,598 --> 00:36:02,068
by the Obama administration they

604
00:36:02,184 --> 00:36:02,344
Media.

605
00:36:02,358 --> 00:36:05,068
created these tiers of
efficacy and the first tier the

606
00:36:05,274 --> 00:36:10,384
Oh, they did?

607
00:36:10,384 --> 00:36:14,594
I didn't know that

608
00:36:14,638 --> 00:36:15,598
I'm at least looking for that.

609
00:36:15,598 --> 00:36:19,478
baseline Does it have a logic model
Does it have a theory of change Do they

610
00:36:19,478 --> 00:36:22,628
even have somewhere on their site where
they're talking about efficacy Because

611
00:36:22,628 --> 00:36:27,738
if not alarm bells should be ringing It
doesn't necessarily mean that a tool isn't

612
00:36:27,738 --> 00:36:31,248
research-supported or backed or aligned
to best practice if it doesn't have it

613
00:36:31,658 --> 00:36:34,538
but it's a pretty strong indication that
the people building it haven't considered

614
00:36:34,538 --> 00:36:39,598
that which to me is a sign that you
know it's a strong signal I that would

615
00:36:39,598 --> 00:36:44,238
be my simplest advice There are other
places though that schools can go to get

616
00:36:44,238 --> 00:36:47,948
advice on this stuff ISTE has a product
index that's pretty good where they

617
00:36:47,948 --> 00:36:54,168
kind of rate and review and kind of vet
technology EdSurge has one Common Sense

618
00:36:54,168 --> 00:36:56,178
Media used to have one but they gave up

619
00:36:56,884 --> 00:36:57,434
Okay.

620
00:36:58,478 --> 00:37:03,688
I they did so their product index is
I think it's still around maybe but

621
00:37:03,688 --> 00:37:04,918
they're not updating It I don't know

622
00:37:05,424 --> 00:37:07,964
I know Johns Hopkins University
has one It's called like

623
00:37:08,081 --> 00:37:08,331
Okay.

624
00:37:09,224 --> 00:37:12,384
ooh something but the the problem
with all of these is like a a lot

625
00:37:12,384 --> 00:37:15,254
of administrators don't know they
exist and a lot of them also just

626
00:37:15,364 --> 00:37:18,084
like not up to date cause it's hard

627
00:37:18,084 --> 00:37:18,544
to keep up to

628
00:37:18,544 --> 00:37:19,564
date with the edtech market

629
00:37:20,091 --> 00:37:22,001
They change all the time.

630
00:37:22,231 --> 00:37:22,291
Yeah.

631
00:37:22,464 --> 00:37:25,384
all the time right So it's like
I applied last year and I was

632
00:37:25,384 --> 00:37:26,934
approved and got a good score

633
00:37:27,451 --> 00:37:27,731
Mm-hmm.

634
00:37:27,731 --> 00:37:27,971
Mm.

635
00:37:28,264 --> 00:37:32,844
if I added 15 new features I guess I'll
have to apply again And then like even

636
00:37:32,924 --> 00:37:36,414
large organizations like Common Sense
Media can't keep up and that's why I do

637
00:37:36,414 --> 00:37:40,394
think we need some sort of regulatory
body that can like actually be the stamp

638
00:37:40,394 --> 00:37:45,164
of approval just like the way we have the
FDA and like food and drug need we need

639
00:37:45,164 --> 00:37:50,024
the equivalent of that in in education of
like a stamp of approval from a verified

640
00:37:50,024 --> 00:37:53,994
government source that's like This is
good and okay cause that is w going back

641
00:37:53,994 --> 00:37:58,604
to like the flaws of the industry one of
them is there's a lot of garbage efficacy

642
00:37:58,864 --> 00:38:03,074
numbers being thrown around that are based
on not real research It's just kind of

643
00:38:03,074 --> 00:38:08,729
like testimonials and sort of like we're
we put together our little case study and

644
00:38:08,729 --> 00:38:12,429
we're pretending it's like a peer-reviewed
randomized control trial and in reality

645
00:38:12,429 --> 00:38:15,529
it's just like an interview with one
teacher that had a good experience Like

646
00:38:15,789 --> 00:38:19,889
that type of stuff probably should not
be allowed or if it is it needs to come

647
00:38:19,889 --> 00:38:23,509
with a label that's like This statement
has not been approved by the Food and

648
00:38:23,509 --> 00:38:26,929
Drug Administration You know what I mean
so th-there is a lot of fake efficacy

649
00:38:26,929 --> 00:38:29,779
going around That's one of the major
flaws that hopefully can be addressed in

650
00:38:29,779 --> 00:38:33,429
this current moment of angst but now I've
gotten off topic So to go back to your

651
00:38:33,429 --> 00:38:36,819
original question look for a theory of
change look for a portfolio of efficacy

652
00:38:36,819 --> 00:38:41,839
on the website then in lieu of that
maybe go to one of these product indexes

653
00:38:41,839 --> 00:38:43,279
and see what other people are saying

654
00:38:43,976 --> 00:38:47,976
Yeah.

655
00:38:48,329 --> 00:38:51,269
if you're looking at using an ed tech
tool call the district down the street

656
00:38:51,269 --> 00:38:55,369
from you that maybe already is using it
or had teachers and get their perspective

657
00:38:56,079 --> 00:38:59,489
Unfortunately we A lot of times in school
districts we live in silos where like

658
00:38:59,899 --> 00:39:03,189
even we're like right next to each other
but we don't talk to the district over

659
00:39:03,189 --> 00:39:06,259
about what's working and what's not so
I think those conversations could be

660
00:39:06,259 --> 00:39:12,219
healthy too But just even just taking a
more proactive stance of like I'm gonna

661
00:39:12,219 --> 00:39:15,739
look into the efficacy before I approve
a tool that in and of itself would be

662
00:39:15,739 --> 00:39:19,759
a step forward cause oftentimes that's
not happening So maybe this current

663
00:39:19,759 --> 00:39:21,429
moment could be good for that at least

664
00:39:21,579 --> 00:39:26,429
Switching gears, I wanna make sure we
talk more about Short Answer, 'cause

665
00:39:26,429 --> 00:39:28,979
it is a really, really amazing tool.

666
00:39:28,979 --> 00:39:35,389
Pretend I am a skeptical English
teacher, and walk me through Short

667
00:39:35,389 --> 00:39:40,599
Answer and why should I spend my
very limited time to explore Short

668
00:39:40,599 --> 00:39:42,504
Answer and try it with my students?

669
00:39:43,413 --> 00:39:46,543
So if you're a skeptical English teacher
then I'm gonna start with the why I'm not

670
00:39:46,543 --> 00:39:50,603
gonna start with the product Let's look
at our current moment We now live in a

671
00:39:50,603 --> 00:39:57,143
world where AI can write for you like a
machine can can write it's a momentous

672
00:39:57,873 --> 00:40:03,323
it it's a crazy moment in human history
that completely upends I think how we need

673
00:40:03,323 --> 00:40:09,683
to approach we teach and assess writing
It doesn't devalue teaching kids how to

674
00:40:09,683 --> 00:40:13,403
write It is still critically important
to learn how to write But I do think it

675
00:40:13,403 --> 00:40:18,283
changes how we go about teaching writing
the first thing that I would say once

676
00:40:18,283 --> 00:40:21,793
writing leaves the classroom which it
necessarily needs to because oftentimes

677
00:40:21,793 --> 00:40:26,363
it just takes a long time to produce
longer form writing writing leaves the

678
00:40:26,363 --> 00:40:30,073
classroom it's hard to know how much of an
influence AI is having on that student's

679
00:40:30,073 --> 00:40:35,593
process a very simple step forward is
we need to do more writing in class will

680
00:40:35,593 --> 00:40:39,253
probably look like shorter form writing
I also think you know the first thing

681
00:40:39,253 --> 00:40:42,743
I see when I log into Gmail in twenty
twenty-six is a thing that says Hey want

682
00:40:42,743 --> 00:40:44,733
me to write the email for you Our kids are

683
00:40:44,859 --> 00:40:45,459
Wait for me.

684
00:40:45,463 --> 00:40:45,573
and

685
00:40:45,959 --> 00:40:46,369
Yep.

686
00:40:47,353 --> 00:40:50,743
and that button's gonna go everywhere
So that's gonna result in less writing

687
00:40:50,743 --> 00:40:54,263
practice Our kids already struggle with
writing The less you practice the more

688
00:40:54,263 --> 00:40:59,753
your skills atrophy Troubling situation
We can't just say Well the English

689
00:40:59,753 --> 00:41:03,443
teacher needs to do more writing We
don't they don't have the bandwidth It

690
00:41:03,473 --> 00:41:06,843
it's gonna take a team It's gonna take
have to embed more writing across the

691
00:41:06,843 --> 00:41:10,663
curriculum I think that's kind of step
number two where it can't just fall on

692
00:41:10,663 --> 00:41:13,783
the English teacher anymore It needs to
be embedded across the curriculum That's

693
00:41:13,783 --> 00:41:17,203
been advocated for for decades It's been
historically really challenging to pull

694
00:41:17,203 --> 00:41:22,473
off but I think any writing tool needs
to be built with that in mind and and

695
00:41:22,473 --> 00:41:25,743
that's partly why we built Short Answer
And then the third piece that I would say

696
00:41:25,743 --> 00:41:30,073
to address this challenge is it has to
be an engaging experience It has to be an

697
00:41:30,073 --> 00:41:34,273
experience where the kid is gonna fall in
love with writing cause how on earth are

698
00:41:34,273 --> 00:41:37,963
we gonna motivate kids to want to learn
how to do this really really challenging

699
00:41:38,013 --> 00:41:40,903
thing arguably one of the hardest
things you learn how to do in school

700
00:41:40,933 --> 00:41:43,303
Cause writing is so cognitively complex

701
00:41:43,489 --> 00:41:43,929
hard.

702
00:41:43,973 --> 00:41:47,163
How are we gonna motivate a kid to wanna
to wrestle with this and struggle and

703
00:41:47,163 --> 00:41:50,743
fail at it where they can press a button
and it'll do it for them in ten seconds

704
00:41:51,573 --> 00:41:55,733
So the third piece is I struggle to
say engagement cause then we fall into

705
00:41:55,733 --> 00:41:58,783
like well it's just gonna be a game and
silliness But no but it needs to be an

706
00:41:58,783 --> 00:42:01,573
experience where a kid falls in love
with writing and reaches a flow state and

707
00:42:01,573 --> 00:42:06,603
just like loves doing it inherently And
so those three pieces more in class more

708
00:42:06,603 --> 00:42:11,043
across the curriculum and more engaging
I think are critical and that's why we

709
00:42:11,043 --> 00:42:13,683
have built Short Answer So I like to
start with the why especially when talking

710
00:42:13,683 --> 00:42:17,573
about cynical English teachers and cynical
teachers in general about why should I

711
00:42:17,573 --> 00:42:21,053
use your ed tech tool That's our why The
way that we do it with Short Answer is

712
00:42:21,053 --> 00:42:25,393
through short form social writing practice
that's intended to be em-embedded across

713
00:42:25,393 --> 00:42:29,613
the curriculum combining daily formative
assessment of whatever skill you're

714
00:42:29,613 --> 00:42:34,943
teaching with writing instruction So
teacher pulls up code kids sign in kids

715
00:42:34,943 --> 00:42:40,443
see prompt Prompt could be anything from
if you're in English class copy-paste in

716
00:42:40,443 --> 00:42:44,453
your opening paragraph to the research
paper we're writing if you're in science

717
00:42:44,453 --> 00:42:48,823
class explain the process of cell division
as an exit ticket right It it could be

718
00:42:48,823 --> 00:42:51,553
anything It could be assessing content it
could be doing literal writing instruction

719
00:42:52,323 --> 00:42:56,863
Kids do it submit it then teacher clicks
begin feedback Going back to Hattie's

720
00:42:56,863 --> 00:43:00,903
effect size point five effect size of
of using technology to improve feedback

721
00:43:00,903 --> 00:43:05,263
processes in in classrooms kids either
get feedback from their peers and that's

722
00:43:05,263 --> 00:43:09,453
a huge part of what we try to do cause
we think inherently motivating for kids

723
00:43:09,553 --> 00:43:12,333
because it creates an authentic audience
for their work of like Oh my gosh my

724
00:43:12,333 --> 00:43:15,213
classmates are gonna see what I wrote
I should try hard And it's anonymous so

725
00:43:15,213 --> 00:43:18,813
we do it safely but I think that's a key
part of it Or they get feedback from AI

726
00:43:19,463 --> 00:43:22,183
or they get feedback from both And I think
there's some really interesting things

727
00:43:22,183 --> 00:43:25,613
you can do with augmenting AI and and peer
feedback so they get that feedback they

728
00:43:25,613 --> 00:43:30,213
reflect on it And then as a class then the
final step is discussion and reflection

729
00:43:30,893 --> 00:43:34,473
So every Short Answer activity follows
that same kind of core structure of do

730
00:43:34,473 --> 00:43:39,793
some writing get some feedback and reflect
on the feedback and then ideally go

731
00:43:39,793 --> 00:43:42,713
back to the do some more writing because
we're using that feedback to improve our

732
00:43:42,713 --> 00:43:46,013
understanding and using that discussion to
improve our understanding so that's Short

733
00:43:46,013 --> 00:43:50,403
Answer as it currently exists We have a
whole new host of features coming soon

734
00:43:50,864 --> 00:43:51,704
Oh.

735
00:43:51,853 --> 00:43:56,333
are gonna completely augment that
entire process it's gonna lead to some

736
00:43:56,333 --> 00:44:00,213
massive changes to Short Answer this
summer So stay tuned for that just

737
00:44:00,213 --> 00:44:01,253
gonna tease them I don't wanna say

738
00:44:01,284 --> 00:44:02,034
Exciting.

739
00:44:02,533 --> 00:44:06,753
But that's the core idea and it's very
straightforward Our kind of philosophy

740
00:44:06,753 --> 00:44:10,323
going back to your point on design is
like we wanna do a f only a few things

741
00:44:10,323 --> 00:44:15,053
but we wanna do them extremely well as
opposed to trying to be like the we do

742
00:44:15,053 --> 00:44:18,973
a bajillion different things type of
tool so it's very simple and I think

743
00:44:18,973 --> 00:44:21,643
in that way it serves us well And so we
have about a hundred thousand monthly

744
00:44:21,643 --> 00:44:23,743
active teachers and students across all

745
00:44:23,784 --> 00:44:23,864
Wow.

746
00:44:24,023 --> 00:44:27,573
states and several countries at
this point And but that's that's

747
00:44:27,573 --> 00:44:32,293
kind of the core of it both what
it does and the why behind we do

748
00:44:33,254 --> 00:44:33,724
I love it.

749
00:44:33,754 --> 00:44:35,514
It is-- it's so engaging.

750
00:44:35,624 --> 00:44:37,134
Students love it.

751
00:44:38,074 --> 00:44:39,054
Like, they really do.

752
00:44:39,294 --> 00:44:44,594
It really gets them
excited to want to write.

753
00:44:44,764 --> 00:44:52,154
And that anonymous peer feedback element
really motivates them to try because they

754
00:44:52,154 --> 00:44:57,444
don't wanna submit something terrible
That their peers are gonna look at and

755
00:44:57,444 --> 00:45:01,164
tear apart and get horrible feedback on.

756
00:45:01,164 --> 00:45:03,294
So it really does motivate.

757
00:45:03,294 --> 00:45:06,254
And I love that you hit
on the engagement point.

758
00:45:07,164 --> 00:45:11,664
I, I talk about... And I talk to teachers
about this a lot when I'm training

759
00:45:11,664 --> 00:45:13,754
them about our educational philosophy.

760
00:45:14,114 --> 00:45:17,584
I'm not having them write like they're,
you know, I'm sure you had to do it in

761
00:45:17,584 --> 00:45:22,134
your teacher prep program where you had
to write your philosophy on education.

762
00:45:22,134 --> 00:45:23,354
We don't go that far.

763
00:45:23,654 --> 00:45:26,694
But I do like to get them to
really think, like, what is

764
00:45:26,694 --> 00:45:27,904
your philosophy as a teacher?

765
00:45:27,904 --> 00:45:29,164
What is your role?

766
00:45:29,464 --> 00:45:32,014
What is your purpose as a teacher?

767
00:45:32,044 --> 00:45:40,104
And some teachers really think that
they are there to get kids to do

768
00:45:40,144 --> 00:45:44,734
hard stuff, and that kids should
just be able to, that grit, just

769
00:45:44,734 --> 00:45:47,114
plow through and do the hard thing.

770
00:45:47,294 --> 00:45:49,524
I don't need to try to engage them.

771
00:45:50,484 --> 00:45:53,114
I just feel so strongly against that.

772
00:45:53,114 --> 00:45:59,614
And the more that I teach, work with,
talk to Gen Z and Gen Alpha learners,

773
00:45:59,654 --> 00:46:04,684
the more I realize that they are so
different, as in every generation.

774
00:46:04,874 --> 00:46:08,914
Every generation is significantly
different than the previous

775
00:46:08,914 --> 00:46:10,664
generations on a whole, right?

776
00:46:11,204 --> 00:46:12,364
Lumping them all together.

777
00:46:12,394 --> 00:46:17,564
Because we have different experiences
growing up Now technology has just

778
00:46:18,024 --> 00:46:23,334
changed the generational experience
immensely, and I really feel that if

779
00:46:23,334 --> 00:46:28,564
we don't engage Gen Z and Gen Alpha
learners in a different way than we

780
00:46:28,564 --> 00:46:34,804
have previous generations, that we are
just gonna see test scores just plummet

781
00:46:35,048 --> 00:46:35,148
Yeah

782
00:46:35,354 --> 00:46:36,254
and down and down and down.

783
00:46:36,534 --> 00:46:41,504
And Short Answer just is really great
at hitting that, the combination of a

784
00:46:41,504 --> 00:46:45,414
little bit of, like, gamification with
a little bit of peer feedback, with the

785
00:46:45,414 --> 00:46:47,564
AI feedback, with the teacher leading.

786
00:46:47,854 --> 00:46:53,694
The teacher leads the entire activity,
that it is very engaging for Gen Z and

787
00:46:53,694 --> 00:46:55,694
Gen Alpha learners in my experience.

788
00:46:55,844 --> 00:46:59,064
More Gen Z. I've, I've seen it
more with Gen Z than Gen Alpha.

789
00:46:59,744 --> 00:47:05,034
On that note, is there a-- Do you say
with Short Answer middle school, high

790
00:47:05,034 --> 00:47:07,434
school only, high school only, all K12?

791
00:47:07,434 --> 00:47:12,034
Can I use it in higher ed if I'm
a higher ed person listening here?

792
00:47:12,158 --> 00:47:14,058
Good question It's generally
second grade through 12th grade

793
00:47:14,714 --> 00:47:16,494
Okay.

794
00:47:16,638 --> 00:47:20,318
I'll be honest we get some classes it goes
like gangbusters and they're like Heck

795
00:47:20,318 --> 00:47:23,238
yeah this is amazing Our second graders
are writing And then in some classes

796
00:47:23,238 --> 00:47:24,808
it's like This is way too advanced So

797
00:47:24,808 --> 00:47:25,098
it d it

798
00:47:25,098 --> 00:47:25,738
just Depends on the

799
00:47:25,854 --> 00:47:27,114
Depends on where they are.

800
00:47:27,334 --> 00:47:27,734
Yeah.

801
00:47:28,278 --> 00:47:29,918
spec that's specific with second grade

802
00:47:30,424 --> 00:47:30,704
Yep

803
00:47:31,028 --> 00:47:36,018
it's third th third through 12th at the
higher ed level it's a bit we've had some

804
00:47:36,058 --> 00:47:39,368
some folks use it with some success but
it's it tends to be K-12 oriented And

805
00:47:39,368 --> 00:47:42,298
especially when you use the tool you'll
kind of understand why It you can tell

806
00:47:42,298 --> 00:47:45,898
from the design it's oriented towards
K-12 going back you said something that

807
00:47:45,898 --> 00:47:48,528
was really interesting and something I
think about a lot when you're talking

808
00:47:48,528 --> 00:47:51,588
about engagement and what does it
look like and I think that's like what

809
00:47:51,728 --> 00:47:56,698
partly what makes designing ed tech
so interesting is like wanna create an

810
00:47:56,698 --> 00:48:00,868
engaging experience that a kid wants
to do But you don't want it to spill so

811
00:48:00,868 --> 00:48:04,998
far that the kids are just conking out
kind of playing games and not really

812
00:48:05,009 --> 00:48:06,019
Yes.

813
00:48:06,929 --> 00:48:07,709
Yes.

814
00:48:07,908 --> 00:48:12,518
I I do think in the in the larger context
of this conversation around ed tech reform

815
00:48:12,518 --> 00:48:16,588
and why people are skeptical of ed tech
is there are a lot of tools that have

816
00:48:16,588 --> 00:48:18,698
kind of leaned really heavily into sort

817
00:48:18,719 --> 00:48:18,809
There's not

818
00:48:18,848 --> 00:48:22,248
the candy side of things where it's like
yeah the kids beg the teacher to do it but

819
00:48:22,248 --> 00:48:26,548
the kids beg the teacher to do it because
just playing games for 20 minutes Like

820
00:48:26,659 --> 00:48:29,479
a huge amount of learning value.

821
00:48:29,479 --> 00:48:29,539
Yeah.

822
00:48:29,539 --> 00:48:29,719
Yeah.

823
00:48:29,919 --> 00:48:33,009
So it's this really fine line to walk
and I always get nervous when I even

824
00:48:33,009 --> 00:48:34,439
say the word gamification because I

825
00:48:34,491 --> 00:48:34,981
Yes.

826
00:48:35,309 --> 00:48:35,599
kind of

827
00:48:36,031 --> 00:48:36,281
Yes.

828
00:48:36,289 --> 00:48:40,479
go off for people like no no no
it's I mean if you use our tool

829
00:48:40,479 --> 00:48:43,809
you'll get a sense of it It's like
pr the gamification is Very light.

830
00:48:43,951 --> 00:48:44,641
Very light

831
00:48:45,099 --> 00:48:48,269
that we're hoping the core engagement
comes from what you just talked about

832
00:48:48,269 --> 00:48:52,129
which is like Oh my gosh my classmates
are gonna see what I wrote And and

833
00:48:52,129 --> 00:48:55,249
there's something just intrinsically
motivating about like creating something

834
00:48:55,249 --> 00:48:59,359
and sharing it with people I'm hoping
that W-which I think is it's not

835
00:48:59,359 --> 00:49:02,749
extrinsic it's not points and bells
and whistles and badges it's creative

836
00:49:02,749 --> 00:49:06,069
and and sharing and and social in
ways that I think are more healthy and

837
00:49:06,741 --> 00:49:11,641
Which is how Gen Z and Gen Alpha,
that's the world that they live in.

838
00:49:11,721 --> 00:49:14,431
They live in a world of gamification.

839
00:49:14,601 --> 00:49:18,721
I actually have an episode coming up
here in a few weeks, closer to end of

840
00:49:18,721 --> 00:49:23,061
June it'll come out, where we're really
diving into gamification versus game-based

841
00:49:23,061 --> 00:49:25,811
learning and the misconceptions.

842
00:49:26,011 --> 00:49:28,461
There's a huge misconceptions of
not understanding the difference.

843
00:49:28,791 --> 00:49:31,501
Gamification is... Well, actually,
let me start with game-based learning.

844
00:49:31,561 --> 00:49:35,701
Game-based learning is you're
learning through a game.

845
00:49:35,991 --> 00:49:39,231
You are playing a game, and
it's the learning process.

846
00:49:39,231 --> 00:49:44,111
Gamification is we're adding
elements of gaming to learning.

847
00:49:44,401 --> 00:49:47,881
So points, that's
gamification right there.

848
00:49:48,171 --> 00:49:52,441
Badges, leaderboards, XP,
that's s- kinda same as points.

849
00:49:52,751 --> 00:49:54,001
That's gamification.

850
00:49:54,001 --> 00:49:58,251
So huge misunderstandings between
the differences in the two, and

851
00:49:58,251 --> 00:50:04,211
I do think that both have gotten
also a bad rap 'cause of a lot

852
00:50:04,211 --> 00:50:06,191
of misinformation around it.

853
00:50:06,191 --> 00:50:09,721
And there are products out there
that have gone too far with

854
00:50:09,779 --> 00:50:10,009
Yeah

855
00:50:10,051 --> 00:50:10,981
gamification.

856
00:50:11,031 --> 00:50:11,411
100

857
00:50:11,482 --> 00:50:17,707
think that that hasn't helped the bad,
but The way that Gen Z and Gen Alpha learn

858
00:50:17,707 --> 00:50:23,007
and the way their brains think and the
world that they live in of social media

859
00:50:23,007 --> 00:50:30,397
and interactive internet experiences,
they really do engage to those small

860
00:50:30,397 --> 00:50:32,737
gamification elements in my experiences.

861
00:50:33,077 --> 00:50:37,187
And that's where ed tech can,
if done right, can really help

862
00:50:37,727 --> 00:50:39,597
with that engagement factor.

863
00:50:39,817 --> 00:50:43,067
And it's so ironic because one point
we're saying out of this side of our mouth

864
00:50:43,187 --> 00:50:46,007
that Gen Z and Gen Alpha aren't engaged.

865
00:50:46,057 --> 00:50:47,857
It's really hard to engage them.

866
00:50:48,367 --> 00:50:49,757
They're checked out of learning.

867
00:50:49,897 --> 00:50:53,497
And the other side of our mouth
we're saying, "Oh, gamification

868
00:50:53,497 --> 00:50:55,937
is terrible and never use it."

869
00:50:56,137 --> 00:50:56,427
Right

870
00:50:56,974 --> 00:51:01,214
Super ironic because if you do it
right, gamification is a really

871
00:51:01,214 --> 00:51:04,854
powerful way to engage Gen Z and
Gen Alpha learners in my experience.

872
00:51:05,474 --> 00:51:07,954
And I do think Short
Answer is doing that right.

873
00:51:07,954 --> 00:51:10,824
And I'm not just saying that
'cause Adam is on the podcast here.

874
00:51:10,824 --> 00:51:13,864
Believe me, if anybody knows me,
they know that I speak my mind

875
00:51:14,144 --> 00:51:17,734
and I tell it how it is, even if
people don't want to hear the truth.

876
00:51:17,934 --> 00:51:20,484
So I wouldn't be saying that
if I didn't really think that.

877
00:51:21,337 --> 00:51:25,207
Well thank you and I also think this
generation of kids For that matter

878
00:51:25,207 --> 00:51:29,267
millennials and yeah Gen Z and anyone
that was raised in this like tech

879
00:51:29,267 --> 00:51:33,037
era th-there's a desp desperation
maybe is a strong word but there's a

880
00:51:33,037 --> 00:51:34,777
strong desire for social connection

881
00:51:35,489 --> 00:51:36,039
Yes.

882
00:51:36,207 --> 00:51:40,317
patterns in society And I think
really good ed tech going forward will

883
00:51:40,317 --> 00:51:43,107
be tech that finds ways to sort of

884
00:51:45,417 --> 00:51:49,257
human interaction in the classroom and I'm
I'm hoping that's what Short Answer can be

885
00:51:50,207 --> 00:51:50,447
Yep.

886
00:51:50,576 --> 00:51:55,356
example we have an activity where okay
kids submit writing get feedback Did

887
00:51:55,356 --> 00:51:58,386
the class hit a score goal together
That's kind of the gamification elements

888
00:51:58,386 --> 00:52:01,726
team-based But then there's an option
the teacher can click that's like reflect

889
00:52:01,726 --> 00:52:05,946
together and it pulls up names on the
board and it's like Go sit Adam go sit

890
00:52:05,946 --> 00:52:10,076
with Lindy and you know Jimbo go sit
with Kearney the kids have to go then

891
00:52:10,076 --> 00:52:14,456
sit together then they they together go
through a structured reflection where

892
00:52:14,456 --> 00:52:17,286
they're talking through what they wrote
about why they wrote about it the feedback

893
00:52:17,286 --> 00:52:21,216
they got if they agreed with the feedback
that's finding a way to use technology to

894
00:52:21,216 --> 00:52:25,976
like get kids sitting together talking to
each other socially i-in a structured way

895
00:52:26,016 --> 00:52:30,686
that supports learning through this like
think pair share model So I wanna really

896
00:52:30,686 --> 00:52:34,466
lean into that as we continue to build and
and scale Short Answer and that's gonna

897
00:52:34,466 --> 00:52:38,316
be part of the updates that you'll see
later this summer I keep teasing those

898
00:52:38,567 --> 00:52:39,197
Yay.

899
00:52:39,377 --> 00:52:40,077
I'm excited.

900
00:52:40,077 --> 00:52:41,117
I'm excited to see them.

901
00:52:41,377 --> 00:52:45,977
You know, Short Answer, if you're one
of those that are in the camp of you

902
00:52:45,977 --> 00:52:51,487
think AI is the end of humanity and it's
getting rid of critical thinking and,

903
00:52:51,487 --> 00:52:56,592
you know, all the terrible things, that,
that end of the AI spectrum If you're

904
00:52:56,722 --> 00:53:02,112
at that end, you should go check out
Short Answer because it is developed

905
00:53:02,112 --> 00:53:08,162
in a way that it pulls the very great
powerful parts of the technology.

906
00:53:08,162 --> 00:53:13,292
And not just AI in general, but if you
are on the spectrum of ed tech where you

907
00:53:13,292 --> 00:53:16,292
do think that tech is like, you know,
ruining the neurons in our brain and

908
00:53:16,292 --> 00:53:21,852
there's nothing good about it, as well as
AI is the end of humanity, Sh- my Short

909
00:53:21,852 --> 00:53:28,352
Answer is a great example of how you can
pull the really powerful parts of the

910
00:53:28,352 --> 00:53:34,002
technology to do things like anonymous
peer feedback that would-- That super

911
00:53:34,002 --> 00:53:41,062
powerful engagement element that would be
very hard to do without the technology.

912
00:53:41,422 --> 00:53:42,242
So there you go.

913
00:53:42,442 --> 00:53:45,532
I was just getting th that you've
you've identified an affordance like w

914
00:53:45,882 --> 00:53:49,142
w going back to this larger conversation
about ed tech It does have unique

915
00:53:49,142 --> 00:53:53,352
affordances that that enable it to improve
learning in unique ways One of them is

916
00:53:53,352 --> 00:53:55,112
anonymity when it comes to writing It's

917
00:53:55,249 --> 00:53:55,579
Yeah.

918
00:53:55,662 --> 00:54:00,362
for stuff to be anonymous which can be
obviously also has has some downsides But

919
00:54:00,502 --> 00:54:04,072
in the context of providing peer feedback
on writing that anonymity is much easier

920
00:54:04,072 --> 00:54:07,222
to pull off digitally which is partly
why we've built Short Answer the way it

921
00:54:07,222 --> 00:54:08,522
is So just wanna double-click on that

922
00:54:08,909 --> 00:54:12,659
Two more questions that I know teachers,
educators listening probably have

923
00:54:12,689 --> 00:54:14,269
if they're interested in trying it.

924
00:54:14,599 --> 00:54:16,489
Number one, go to myshortanswer.com.

925
00:54:17,279 --> 00:54:18,899
Is this only for English teachers?

926
00:54:18,899 --> 00:54:20,939
I feel like you already kind
of answered this, but I wanna

927
00:54:20,939 --> 00:54:23,089
specifically hit that point.

928
00:54:24,087 --> 00:54:26,347
Yeah and that's why we called it
Short Answer is cause we we didn't

929
00:54:26,347 --> 00:54:28,337
wanna call it like The Writing Tool

930
00:54:28,389 --> 00:54:28,919
Yes.

931
00:54:28,919 --> 00:54:29,479
Yes.

932
00:54:29,577 --> 00:54:33,277
wh when when a when a science teacher
sees The Writing Tool they're like Ah

933
00:54:33,277 --> 00:54:36,487
it's not for me we were like Ah Short
Answer that's kind of something that

934
00:54:36,687 --> 00:54:40,857
is everybody uses but yeah no it's
not just for English teachers In fact

935
00:54:40,857 --> 00:54:42,767
it's it's built from the ground up to

936
00:54:42,939 --> 00:54:46,939
And

937
00:54:47,357 --> 00:54:52,617
for for English it does work in math
works in math with specific use cases

938
00:54:52,927 --> 00:54:57,167
the best being like error analysis
tasks where it's like teacher uploads

939
00:54:57,167 --> 00:55:00,197
picture of incorrectly worked equation
and using writing kids explain

940
00:55:00,379 --> 00:55:01,429
Oh,

941
00:55:01,477 --> 00:55:01,687
and

942
00:55:01,799 --> 00:55:02,289
okay.

943
00:55:03,167 --> 00:55:05,797
It's not great I'll just be transparent
It's not great for writing if you're

944
00:55:05,797 --> 00:55:10,367
trying to get kids to like work through
a problem because it's made for text and

945
00:55:10,577 --> 00:55:10,947
Yep.

946
00:55:11,066 --> 00:55:12,736
have a digital whiteboard functionality

947
00:55:13,097 --> 00:55:16,027
I could see it being used for
word problems though in math.

948
00:55:16,346 --> 00:55:16,786
100

949
00:55:17,167 --> 00:55:17,627
Yeah

950
00:55:17,716 --> 00:55:22,216
for concepts and and and for like
definitions and yeah No it it definitely

951
00:55:22,216 --> 00:55:26,536
can be used in math I'm just being
transparent Like of the use cases I

952
00:55:26,536 --> 00:55:30,696
would say generally it goes English
and then social studies science is the

953
00:55:30,696 --> 00:55:34,806
next most common use case and then down
into the electives and that's probably

954
00:55:34,806 --> 00:55:37,766
just a product of there's teachers th
in in the electives than there are in

955
00:55:37,766 --> 00:55:42,866
the the core content area and then math
So um that's kind of the flow and but

956
00:55:42,866 --> 00:55:45,766
we've we've built it from the ground
up to work across the curriculum and

957
00:55:45,956 --> 00:55:48,576
sorry to keep doing this but we have
some features coming this summer that

958
00:55:48,576 --> 00:55:53,486
are gonna help even more to embed it in
your content area So I'm excited to roll

959
00:55:53,486 --> 00:55:55,266
those out and stop teasing them like this

960
00:55:55,366 --> 00:55:59,656
When I was teaching career and tech
ed, I would 100% used Short Answer.

961
00:55:59,816 --> 00:56:04,726
I taught personal finance, and my
students told me that they wrote

962
00:56:04,756 --> 00:56:07,746
more in their personal finance
class than they did in English.

963
00:56:07,946 --> 00:56:08,336
Yeah

964
00:56:08,766 --> 00:56:09,046
'
Cause I--

965
00:56:09,406 --> 00:56:11,306
that's a great that's a great thing

966
00:56:11,596 --> 00:56:11,836
Yeah.

967
00:56:11,866 --> 00:56:15,096
No, they, I literally had students
tell me that multiple times.

968
00:56:15,096 --> 00:56:18,846
Like, "We write..." And of course, they're
like, "We write more in this class than

969
00:56:18,846 --> 00:56:23,666
we do in English class." But that's a
perfect example, like career and tech ed.

970
00:56:23,856 --> 00:56:29,026
Hey, no, 100% would've, would've used
Short Answer a lot actually, and it

971
00:56:29,026 --> 00:56:32,646
works really great if you're like an exit
ticket person, really great for that.

972
00:56:32,676 --> 00:56:35,026
I think you can really use it
as that anticipatory set at

973
00:56:35,026 --> 00:56:36,126
the beginning of the lesson.

974
00:56:37,016 --> 00:56:41,946
Second question that I'm sure people
have is pricing to use Short Answer.

975
00:56:42,846 --> 00:56:46,576
It's interesting that you asked that
question today Just last Friday we changed

976
00:56:46,576 --> 00:56:51,596
some things So straightforward response
beginning June 15th we will no longer

977
00:56:51,596 --> 00:56:56,276
offer individual teacher licenses We
used to offer individual teacher licenses

978
00:56:56,276 --> 00:57:00,906
for $75 per year We just announced last
Friday that in the upcoming month we're

979
00:57:00,906 --> 00:57:04,676
gonna get rid of that option was a
controversial decision and not an easy

980
00:57:04,676 --> 00:57:09,066
one and it was frustrating for a lot
of our teachers if I'm happy to talk

981
00:57:09,066 --> 00:57:12,486
about why we chose to make that decision
if we wanna get into it and in fact I

982
00:57:12,536 --> 00:57:13,876
would kinda like to cause it gives me a

983
00:57:14,062 --> 00:57:16,442
Yeah, go for it

984
00:57:16,496 --> 00:57:20,546
But j-j-just quickly to answer your
question So otherwise it's we do

985
00:57:20,546 --> 00:57:25,386
site-wide licenses or district-wide
licenses We price that per student it.

986
00:57:25,386 --> 00:57:28,266
depends on the size of your district
but right now we tend to charge anywhere

987
00:57:28,266 --> 00:57:32,916
between $2 to $4 per student depending
on the size and level of adoption so

988
00:57:32,916 --> 00:57:36,846
that's general pricing and pricing
structure we're happy to do some pilots

989
00:57:37,006 --> 00:57:41,466
of the premium version of our tool in a
structured way to to try it out before you

990
00:57:41,466 --> 00:57:46,156
buy it I think that's a healthy process in
edtech adoption said we also have a free

991
00:57:46,156 --> 00:57:49,976
plan So if you're just like Dude I don't
have budget but this sounds cool and I

992
00:57:49,976 --> 00:57:54,756
wanna use it fine we still have a really
robust free plan that in-includes many of

993
00:57:54,756 --> 00:57:57,286
the all the el in fact all the elements
that you and I have talked about on this

994
00:57:57,286 --> 00:58:01,956
podcast like the the peer-to-peer feedback
all those things So we do have a free plan

995
00:58:01,956 --> 00:58:04,706
and that will continue to be available
It's just that literally last Friday we

996
00:58:04,706 --> 00:58:09,396
announced that we're removing individual
purchases of licenses I'm happy to talk

997
00:58:09,396 --> 00:58:12,036
about why but that's the pricing structure

998
00:58:12,186 --> 00:58:16,106
We could've done a whole episode
just on, like, research and efficacy

999
00:58:16,106 --> 00:58:19,006
and the screen time debate, and then
we could've done a whole episode on

1000
00:58:19,006 --> 00:58:22,336
short answer, and then we could also
have done a whole episode on writing

1001
00:58:22,336 --> 00:58:29,826
instruction in the age of AI, because
obviously you are a expert in that area.

1002
00:58:29,856 --> 00:58:34,906
I was lucky to get to go to your
workshop at TCA this year, yeah.

1003
00:58:34,994 --> 00:58:35,494
to have you

1004
00:58:35,714 --> 00:58:36,274
It was cool to see

1005
00:58:36,274 --> 00:58:36,544
you there

1006
00:58:36,596 --> 00:58:40,846
all right, k- chef's kiss to the
Matthew McConaughey reference there.

1007
00:58:41,116 --> 00:58:46,816
It was fantastic, seriously, I
am very critical of my PD time,

1008
00:58:47,056 --> 00:58:48,876
especially conference sessions.

1009
00:58:49,106 --> 00:58:52,056
A, A, don't have a lot of time to go to
conference sessions, 'cause I'm always

1010
00:58:52,056 --> 00:58:56,646
presenting and doing a million other
things, and B, I'm always very careful at

1011
00:58:56,646 --> 00:58:59,706
the conference sessions that I do choose
to go to, 'cause I have such little time.

1012
00:58:59,936 --> 00:59:01,196
It was very, very good.

1013
00:59:01,196 --> 00:59:04,266
I actually have gone back and referenced
your slide deck several times,

1014
00:59:04,474 --> 00:59:04,534
Yeah

1015
00:59:04,566 --> 00:59:06,856
which I also would say
that I, I rarely do.

1016
00:59:06,856 --> 00:59:11,776
It was very helpful for me,
'cause I get tons of questions on

1017
00:59:11,974 --> 00:59:12,034
Yeah

1018
00:59:12,196 --> 00:59:15,156
The writing in the age of AI debate.

1019
00:59:15,626 --> 00:59:18,266
A few things to hit on here.

1020
00:59:18,346 --> 00:59:24,696
You said that you think that writing
instruction in the age of advanced AI,

1021
00:59:24,696 --> 00:59:28,856
generative AI, agentic AI that we're
in right now, does need to change.

1022
00:59:28,856 --> 00:59:30,556
You hit on that a little bit.

1023
00:59:31,026 --> 00:59:33,946
What else would you add
to that for teachers?

1024
00:59:34,146 --> 00:59:38,016
Yeah no We lead four-hour workshops
on this so I want to preface it

1025
00:59:38,016 --> 00:59:38,136
that

1026
00:59:38,167 --> 00:59:38,507
know

1027
00:59:38,616 --> 00:59:38,756
not

1028
00:59:38,817 --> 00:59:39,277
I know,

1029
00:59:39,696 --> 00:59:40,406
not we're not

1030
00:59:40,437 --> 00:59:40,737
Oh,

1031
00:59:40,756 --> 00:59:41,376
justice And

1032
00:59:41,387 --> 00:59:41,757
all right,

1033
00:59:41,826 --> 00:59:42,016
Oh go

1034
00:59:42,077 --> 00:59:42,477
all right.

1035
00:59:42,777 --> 00:59:42,917
I

1036
00:59:42,966 --> 00:59:43,076
All

1037
00:59:43,157 --> 00:59:43,267
I

1038
00:59:43,366 --> 00:59:43,536
right

1039
00:59:43,607 --> 00:59:43,957
know,

1040
00:59:44,866 --> 00:59:47,986
Yeah exactly It was funny at the
beginning of that session to say like

1041
00:59:48,136 --> 00:59:50,616
How many people are here just for the
McConaughey pun And like half the people

1042
00:59:50,616 --> 00:59:54,656
raised their hands so But I was glad
they were there okay quickly I'll just

1043
00:59:54,656 --> 00:59:58,196
rattle them off Don't use AI detection
software It doesn't work that's not

1044
00:59:58,196 --> 01:00:02,076
gonna be a solution unfortunately
especially in a world of agentic AI even

1045
01:00:02,076 --> 01:00:05,646
revision tracking tools which I would
have suggested even just six months ago

1046
01:00:05,846 --> 01:00:10,986
moving forward it's not gonna be reliable
necessarily because you can use AI agents

1047
01:00:10,986 --> 01:00:15,146
to autonomously write a paper for you
and kind of simulate a revision history

1048
01:00:15,786 --> 01:00:17,056
We're pretty much already in that world

1049
01:00:17,551 --> 01:00:17,801
Yep.

1050
01:00:17,946 --> 01:00:22,346
So I think that the healthiest forward
addition to more writing in class and

1051
01:00:22,346 --> 01:00:26,376
more writing across the curriculum more
layered assessment And I talked about

1052
01:00:26,376 --> 01:00:31,376
this in the the TCA presentation This is
from Philip Dawson and his interpretation

1053
01:00:31,376 --> 01:00:36,566
of Dr Kiara Rundle's research But it's
basically thinking about any layer of

1054
01:00:36,566 --> 01:00:40,926
assessment that you come up with to
measure how well a kid can write or do

1055
01:00:40,926 --> 01:00:44,886
anything really is gonna have holes in
it that AI plagiarism can easily get

1056
01:00:44,886 --> 01:00:49,436
through cause AI can do just about any
task you throw at a kid where one layer

1057
01:00:49,436 --> 01:00:52,726
might have a hole another layer might
not It's called the Swiss cheese model

1058
01:00:52,726 --> 01:00:56,956
sometimes you wanna stack the layers of
Swiss cheese you know where one has a hole

1059
01:00:56,956 --> 01:01:00,846
another might not you're gonna eventually
block that AI plagiarism for example an

1060
01:01:00,846 --> 01:01:04,416
example I talk about in writing is like
okay if it's a five-page research paper

1061
01:01:04,416 --> 01:01:09,246
it's big summative for my eighth graders
let's say or freshmen I don't know we'll

1062
01:01:09,246 --> 01:01:13,066
do more in-class writing but then we're
also gonna do peer conferencing That's

1063
01:01:13,066 --> 01:01:16,126
gonna be a layer where they have to bring
their writing into class and and defend

1064
01:01:16,126 --> 01:01:20,126
what they wrote to small groups let's
say then maybe while they do that another

1065
01:01:20,126 --> 01:01:22,826
layer is gonna be I'm gonna go around
and have a one-on-one conversation with

1066
01:01:22,826 --> 01:01:26,226
students and talk to them while they're
doing the peer conferencing where they

1067
01:01:26,226 --> 01:01:30,646
walk me through what they've written And
then maybe we're also gonna use revision

1068
01:01:30,646 --> 01:01:33,976
tracking That can be a layer too Like it's
it's flawed but there's some potential

1069
01:01:34,006 --> 01:01:37,156
there Like it's still kinda helpful cause
a lot of kids don't have access to agentic

1070
01:01:37,156 --> 01:01:42,146
AI quite yet maybe if it's a big summative
big enough summative assessment maybe

1071
01:01:42,146 --> 01:01:45,546
we're gonna do an in-class presentation
of your paper where you walk the class

1072
01:01:45,546 --> 01:01:49,031
through your process and talk through Why
you structured it the way you did or what

1073
01:01:49,031 --> 01:01:53,481
your thesis was or you know defend what
you wrote so you're adding like multiple

1074
01:01:53,481 --> 01:01:57,611
different layers to this assessment such
that plagiarism might get through one or

1075
01:01:57,611 --> 01:02:00,751
two of them but it's not gonna get through
all of them how you go about structuring

1076
01:02:00,751 --> 01:02:06,111
those layers depends on the assignment It
depends on what you're teaching like your

1077
01:02:06,111 --> 01:02:10,401
content area I mean what age group you're
working with a lot of different things but

1078
01:02:10,431 --> 01:02:14,251
I think that lens is a really healthy one
of assessment layering cause that's just

1079
01:02:14,251 --> 01:02:20,011
kind of general good practice in formative
assessment a-anyway so we have this

1080
01:02:20,011 --> 01:02:23,331
whole resource and if people reach out
to me my email is adam@myshortanswer.com

1081
01:02:23,441 --> 01:02:29,721
I can share it about like what might a
layer look like and like do layers look

1082
01:02:29,721 --> 01:02:33,031
like before writing during writing after
writing There's a lot of different things

1083
01:02:33,031 --> 01:02:37,451
we can do But just that lens of layering
assessment I would say to keep my response

1084
01:02:37,451 --> 01:02:41,411
as succinct as I can that would be my
answer to like structural reform that

1085
01:02:41,411 --> 01:02:46,361
needs to happen to continue to accurately
and effectively teach and assess writing

1086
01:02:46,551 --> 01:02:48,271
in a world where machines can do it for us

1087
01:02:49,087 --> 01:02:51,187
I love that answer from
multiple perspectives.

1088
01:02:51,377 --> 01:02:53,787
One, because I 100% agree.

1089
01:02:53,837 --> 01:02:55,997
There is no silver bullet answer

1090
01:02:56,031 --> 01:02:56,891
Yeah

1091
01:02:56,957 --> 01:03:00,557
and I know that's super
frustrating for educators, and

1092
01:03:00,557 --> 01:03:02,207
that's not what you wanna hear.

1093
01:03:02,237 --> 01:03:07,297
And you want to hear that, "Oh, I can use
revision tracking or AI content detectors,

1094
01:03:07,297 --> 01:03:08,727
and that'll just solve this problem."

1095
01:03:08,727 --> 01:03:09,387
Silver bullet.

1096
01:03:09,387 --> 01:03:13,467
But the evidence is huge
It just does not work.

1097
01:03:13,907 --> 01:03:16,917
And exactly what Adam said, the
more advanced, especially the more

1098
01:03:16,917 --> 01:03:21,847
we're getting into agentic AI, even
that tracking revision history,

1099
01:03:21,847 --> 01:03:24,347
it, it just is not gonna work.

1100
01:03:24,477 --> 01:03:28,107
It could be one teeny, teeny
little tool in your layers.

1101
01:03:28,447 --> 01:03:31,407
One, one teeny little layer in
your layers maybe, but it can't be

1102
01:03:31,407 --> 01:03:33,267
the only thing that you're doing.

1103
01:03:33,447 --> 01:03:36,537
I also love that answer because
I loved that you introduced

1104
01:03:36,597 --> 01:03:37,957
that Swiss cheese model.

1105
01:03:37,957 --> 01:03:40,707
That, I think that's actually what
I've gone back and referenced a few

1106
01:03:40,707 --> 01:03:45,687
times in the slide deck because I have
the same analogy, but I use puzzle

1107
01:03:45,687 --> 01:03:49,347
pieces instead when I'm talking to
teachers about assessing in the age

1108
01:03:49,347 --> 01:03:53,597
of AI, and this idea that if you
only have one puzzle piece, even two

1109
01:03:53,597 --> 01:03:55,387
puzzle pieces, that's not a puzzle.

1110
01:03:55,677 --> 01:03:57,587
You've gotta have at
least three puzzle pieces.

1111
01:03:57,647 --> 01:04:01,417
And ideally, the more puzzle pieces
you have, the harder the puzzle.

1112
01:04:01,837 --> 01:04:05,847
A 1,000-piece puzzle is a lot harder
than a 500-piece puzzle to do.

1113
01:04:06,047 --> 01:04:09,707
So it's the same idea, like layering
Swiss cheese and covering the

1114
01:04:09,707 --> 01:04:12,217
holes, adding more puzzle pieces.

1115
01:04:12,797 --> 01:04:15,327
It has to be a multi-layered approach.

1116
01:04:15,327 --> 01:04:17,607
And that is not just writing, by the way.

1117
01:04:17,817 --> 01:04:21,067
It's essential to writing
because that's what AI large

1118
01:04:21,067 --> 01:04:25,417
language models in particular
are best at right now is writing.

1119
01:04:25,467 --> 01:04:31,577
It's also, I think, what we have to do
across really all assessment is take

1120
01:04:31,577 --> 01:04:34,887
this really multi-layered approach.

1121
01:04:35,287 --> 01:04:39,457
I will link to, of course,
myshortanswer.com in the show notes.

1122
01:04:39,507 --> 01:04:40,397
We'll link to your Substack.

1123
01:04:40,397 --> 01:04:45,347
Uh, I linked to it in a couple episodes
ago, the specific article you wrote.

1124
01:04:45,347 --> 01:04:48,377
It was a two-part article
on, you know, edtech is worth

1125
01:04:48,487 --> 01:04:49,727
saving or something like that.

1126
01:04:49,727 --> 01:04:50,887
But we'll link to your Substack.

1127
01:04:51,107 --> 01:04:52,787
I haven't read the one you just put out.

1128
01:04:53,097 --> 01:04:54,207
I'll have to go back and read that.

1129
01:04:54,207 --> 01:05:00,447
I'll put your email in there as well
so people will have all of that.

1130
01:05:00,647 --> 01:05:01,787
Wrapping up here,

1131
01:05:01,841 --> 01:05:04,601
Anything else you wanna share
with listeners in terms of

1132
01:05:04,601 --> 01:05:05,761
how they can connect with you?

1133
01:05:06,574 --> 01:05:10,604
I mean my email's adam@myshortanswer
Anybody can reach out to me I'm

1134
01:05:10,604 --> 01:05:13,954
happy to chat I genuinely enjoyed
this conversation and could continue

1135
01:05:13,954 --> 01:05:15,504
having it for hours because I'm

1136
01:05:15,811 --> 01:05:16,411
Same.

1137
01:05:16,504 --> 01:05:21,194
work and I think we're in a really
important moment for education but also

1138
01:05:21,194 --> 01:05:25,944
educational technology specifically
And the decisions we make in the next

1139
01:05:26,144 --> 01:05:29,214
six to 12 months feel like they're
gonna reverberate for the next five to

1140
01:05:29,214 --> 01:05:33,524
10 years So it it feels like a really
important moment and I appreciate you

1141
01:05:33,524 --> 01:05:37,654
giving me a chance to share my voice
on this I hope that you will and I

1142
01:05:37,654 --> 01:05:41,404
know you do cause you're a rock star
w will continue to lift up the voices

1143
01:05:41,404 --> 01:05:45,304
of educators cause I think their
voices are kind of being lost in this

1144
01:05:45,391 --> 01:05:46,011
Yes.

1145
01:05:46,354 --> 01:05:49,454
There's just like this assumption of
like No one ever wanted technology and we

1146
01:05:49,454 --> 01:05:53,134
never we There's no Why did we ever even
adopt it to begin with I'm like Well you

1147
01:05:53,134 --> 01:05:58,014
should go talk to the 10,000 teachers that
I see every year at ed tech conferences

1148
01:05:58,201 --> 01:05:58,561
Yep.

1149
01:05:58,704 --> 01:06:01,404
this stuff because of the positive
impact it's having in their classroom

1150
01:06:01,484 --> 01:06:05,254
So please keep doing what you do and
please keep lifting up those voices

1151
01:06:05,254 --> 01:06:06,404
cause I think they're really important

1152
01:06:07,221 --> 01:06:08,391
Yeah, absolutely.

1153
01:06:08,391 --> 01:06:14,411
I might have to have you back on to
dig more into writing instruction,

1154
01:06:14,451 --> 01:06:18,611
especially in the age of AI, 'cause
that is such a hot button topic.

1155
01:06:18,611 --> 01:06:22,551
And you said it, but I know y- you
guys do, like, four-hour workshops

1156
01:06:22,891 --> 01:06:24,321
specifically focused on this.

1157
01:06:24,611 --> 01:06:27,531
Do you have that on the
Short Answer website?

1158
01:06:28,414 --> 01:06:31,604
Yeah yeah If you go to our at the top
there's like a Getting Started button

1159
01:06:31,604 --> 01:06:34,524
and then there'll be an option for like
professional development and workshops

1160
01:06:34,524 --> 01:06:35,724
and that'll give you the information

1161
01:06:35,851 --> 01:06:36,031
Okay.

1162
01:06:36,031 --> 01:06:36,161
Okay.

1163
01:06:36,474 --> 01:06:40,424
We will be leading it Oh man what's the
next conference We lead a a lot with

1164
01:06:40,424 --> 01:06:45,184
school districts but the next conference
I think we're leading it at is will be

1165
01:06:45,984 --> 01:06:49,024
Dakota Teachers of English Conference
Huge conference that I'm sure all your

1166
01:06:49,024 --> 01:06:50,384
listeners will be at Just kidding and

1167
01:06:50,411 --> 01:06:51,801
Where's that at, by the way?

1168
01:06:51,961 --> 01:06:53,871
Is that in Rapid City or Sioux Falls

1169
01:06:53,939 --> 01:06:54,889
No it's in Sioux Falls?

1170
01:06:54,889 --> 01:06:55,009
It's

1171
01:06:55,021 --> 01:06:55,361
Sioux Falls.

1172
01:06:55,361 --> 01:06:56,991
Okay, yeah, that's a long ways from me.

1173
01:06:57,019 --> 01:06:57,909
that's a long way for you

1174
01:06:57,941 --> 01:06:58,241
Yeah.

1175
01:06:58,349 --> 01:07:01,099
But we'll be in we'll be in Philadelphia
for the National Council of Teachers

1176
01:07:01,099 --> 01:07:02,129
of English Conference and we're

1177
01:07:02,261 --> 01:07:02,491
Okay.

1178
01:07:02,599 --> 01:07:05,159
the workshop It's it's gonna be the
full four-hour workshop and we're

1179
01:07:05,159 --> 01:07:08,529
gonna lead it with one of our schools
Kimberly Sosbe who's an awesome director

1180
01:07:08,529 --> 01:07:11,969
of secondary ELA at Grand Prairie ISD
in Texas her and I are gonna lead it

1181
01:07:11,969 --> 01:07:15,939
together if you've got English teachers
that are on the East and Seaboard or

1182
01:07:15,939 --> 01:07:18,369
who are members of the National Council
of Teachers of English we'll be in

1183
01:07:18,369 --> 01:07:22,419
Philadelphia leading it this this fall
any conference we go to we try to lead

1184
01:07:22,419 --> 01:07:26,769
a version of it So yeah please reach out
I'm happy to talk about that or anything

1185
01:07:27,151 --> 01:07:27,471
Okay.

1186
01:07:28,039 --> 01:07:30,819
technology efficacy short
answer writing whatever

1187
01:07:31,261 --> 01:07:35,761
I will put a link to the writing
workshops in the show notes as well,

1188
01:07:35,761 --> 01:07:38,951
if there's any teachers that are
specifically interested in that.

1189
01:07:38,981 --> 01:07:44,011
I wanna try to go-- I got your
50-minute version of your four hours.

1190
01:07:44,011 --> 01:07:49,011
I would love to, to get
the full four hours myself.

1191
01:07:49,409 --> 01:07:54,429
For real wrapping up, we end every episode
with a Make a Tech100 Moment, which

1192
01:07:54,429 --> 01:07:59,169
is like what is your final thoughts of
what you want to leave listeners with?

1193
01:07:59,369 --> 01:08:01,259
Oh man

1194
01:08:01,309 --> 01:08:03,669
We covered a lot of
ground in this episode.

1195
01:08:03,828 --> 01:08:08,258
so much ground If you're an educator
listening to this you probably like

1196
01:08:08,258 --> 01:08:12,648
educational technology because you're
you know what you do Lindy and you're

1197
01:08:12,648 --> 01:08:19,098
really good at it So I guess my thing
to share would be share your voice and

1198
01:08:19,098 --> 01:08:23,388
when I say share your voice I just mean
share your experience share how you use

1199
01:08:23,388 --> 01:08:27,898
technology and how it has positively
improved outcomes in your classroom in

1200
01:08:27,898 --> 01:08:33,658
this But I think those stories are being
lost in our current echo chamber around

1201
01:08:33,728 --> 01:08:38,498
how awful technology is and we're losing
sight of the really positive use cases

1202
01:08:38,498 --> 01:08:42,058
and affordances in education And the
most important voices we need to hear

1203
01:08:42,058 --> 01:08:44,718
those from are teachers like the people
that might be listening to this podcast

1204
01:08:44,718 --> 01:08:49,468
So I'd just say your voice share your
stories on social media on Substack with

1205
01:08:49,468 --> 01:08:54,798
your with your principal with your school
board with your parents like any audience

1206
01:08:54,898 --> 01:08:58,578
to sort of help com I don't wanna say
combat but just to help put some balm

1207
01:08:58,578 --> 01:09:04,168
on the the the current moment that would
be my my I would leave with I guess

1208
01:09:04,599 --> 01:09:05,449
100% agree.

1209
01:09:05,449 --> 01:09:11,229
The naysayers are loud, so we have
to be loud in response to show

1210
01:09:11,599 --> 01:09:12,719
the other end of the spectrum.

1211
01:09:12,929 --> 01:09:16,609
By the way, I loved in your
workshop the, I think you

1212
01:09:16,609 --> 01:09:19,139
called it radical cen-centrism.

1213
01:09:19,139 --> 01:09:19,899
What did you call it?

1214
01:09:20,008 --> 01:09:21,268
Radical centrism

1215
01:09:21,409 --> 01:09:25,919
so radical centrism this idea of there's
a spectrum on one end of the other.

1216
01:09:26,169 --> 01:09:29,659
One spectrum is, you know, it's
great and everything's great.

1217
01:09:29,659 --> 01:09:31,049
Like right now, AI.

1218
01:09:31,049 --> 01:09:32,509
AI's gonna save humanity, da, da, da, da.

1219
01:09:32,509 --> 01:09:35,949
And then the other end of the spectrum
is it's all terrible, it's ruining our

1220
01:09:35,949 --> 01:09:38,299
brains, AI's the end of humanity, right?

1221
01:09:38,519 --> 01:09:42,699
And your argument was be a
radical centrist in the middle.

1222
01:09:43,269 --> 01:09:44,279
I love that for multiple reasons.

1223
01:09:44,279 --> 01:09:47,479
One of the reasons is that's exactly
what I tell teachers, especially when

1224
01:09:47,479 --> 01:09:49,499
it comes to the AI debate right now.

1225
01:09:49,819 --> 01:09:52,154
I always say- You wanna be in the center.

1226
01:09:52,184 --> 01:09:57,384
You wanna pull the powerful parts of
the technology and leverage the powerful

1227
01:09:57,384 --> 01:10:00,924
parts and mitigate the bad parts.

1228
01:10:01,074 --> 01:10:01,524
So

1229
01:10:01,817 --> 01:10:02,337
Agreed

1230
01:10:02,644 --> 01:10:08,994
I loved... I've been using your radical
centrist terminology ever since then.

1231
01:10:09,374 --> 01:10:10,504
Be a radical centrist.

1232
01:10:10,657 --> 01:10:12,467
now but yeah Yeah

1233
01:10:12,484 --> 01:10:12,924
Is there?

1234
01:10:12,924 --> 01:10:13,674
Okay.

1235
01:10:14,107 --> 01:10:15,017
Yeah Yeah

1236
01:10:15,264 --> 01:10:16,204
Okay, awesome.

1237
01:10:16,234 --> 01:10:16,594
Love it.

1238
01:10:16,834 --> 01:10:17,284
All right.

1239
01:10:17,284 --> 01:10:18,344
Thank you so much, Adam.

1240
01:10:18,344 --> 01:10:21,754
Great conversation, and like I said, we
might have to have you back on the podcast

1241
01:10:22,287 --> 01:10:22,897
Anytime.

1242
01:10:22,977 --> 01:10:23,737
Thank you for having me

1243
01:10:27,115 --> 01:10:29,095
Thanks for joining Make EdTech 100.

1244
01:10:29,885 --> 01:10:34,455
I know educator time is valuable, and I'm
honored you choose to spend yours with me.

1245
01:10:35,585 --> 01:10:40,205
For more edtech strategies you can use
tomorrow and ways to bring me to your

1246
01:10:40,205 --> 01:10:43,165
school or event, head to lindyhock.com.

1247
01:10:43,975 --> 01:10:47,755
If this episode resonated, hit Subscribe
so you don't miss the next one.

1248
01:10:48,625 --> 01:10:48,965
I'm LindyHoc.

1249
01:10:49,685 --> 01:10:52,079
Go forth and Make EdTech 100