Subscribe
Share
Share
Embed
Technology in education shouldn’t feel overwhelming, confusing, or disconnected from what actually works in classrooms. This show keeps it real. Each week, educator and K–12 EdTech Advisor Lindy Hockenbary shares practical strategies, stories from real teachers, and no-nonsense conversations about how to use tech in ways that feel human, joyful, and grounded in learning.
Whether you’re a teacher, a school leader, or an EdTech product builder, this show helps you Make EdTech 100 — real, relevant, and rooted in what matters most: kids.
Welcome to MakeEdTech100.
I'm LindyHoc, educator, K-12
edtech advisor, and your host.
This is a podcast where we keep it real
about what actually works in classrooms.
No hype, no overwhelm, just practical
strategies, honest stories, and
tools that make a real difference
for teachers and students.
So come along with me on
a journey to MakeEdTech100
Today's guest was already quoted
on the podcast a few episodes ago.
Adam Sparks is the source of use
case not existence when it comes
to technology use in classrooms.
So to continue my series on screen
time in schools, I knew I had to get
Adam on for a great conversation.
Adam is an English and social studies
teacher who recently finished his
master's degree in learning design
and technology, and as part of his
master's coursework, he built a digital
writing tool called Short Answer.
It's really amazing.
We'll talk about that more in the episode.
It's so powerful that he now works
full-time developing and sharing Short
Answer with schools and educators.
Welcome to the podcast, Adam.
Hey thanks for having me I'm very
excited to be here and chat with you Andy
Tell us, what else do listeners
need to know about you
I think you?
got it I taught for seven years I went
to grad school I built what was supposed
to be a master's project We got grant
funding and launched it and here we
are building it three years later So
feel lucky to do the work I do I do it
with my wife Alexa which is special and
unique and fun and I'm really passionate
about e-education and technology in our
current moment and so I feel like I'm
in the right place I'm I'm really on
fire by the work right now which is fun
I named the podcast Make Ed
Tech 100 because that meaning
is open to interpretation
of whatever it means to you.
So I like to ask every guest to tell us
what does Make Ed Tech 100 mean to you?
Oh man Well I think of 100 it's like
keep it 100 Like it's like 100 like
keep Yeah, keep ed tech good I don't
know if that's I I equate it in my
mind to good I don't know if that's
right or or wrong but yeah so keep
educational technology high quality
Yeah
which is fitting for our current moment
W- it is very fitting
for the current moment.
So yeah that's where my
mind goes I don't know
The current conversation is focusing
on quantity and not quality,
Yes
right?
So that is super, super fitting
amongst the screen time debate
you're a former classroom teacher.
I didn't realize you taught social
studies as well, by the way.
I knew you taught English, obviously,
The Writing Connection, and you're
building an education product to
assist with writing instruction.
Build, built and building.
Building education products never ends.
Right
and present and future
tense, all of the above.
So I love that you have
both of those perspectives.
When you hear Utah passed a screen
time bill this legislative season,
something like 15 other states had
legislations considering some form of
screen time blanket ban or limitation.
LAUSD also passed some sort of policy
or something about screen time at
LAUSD in particular, so that's a school
district level, not a state level.
What's your gut reaction to hearing that?
I have a lot First I wanna say I
think it it's all coming from a
good place It's people are really
Agree.
and have been for a while a-about like
cell phones and and social media and the
impact that that has on kids and their
social lives and their overall health and
wellbeing I get that and so I think those
concerns have been around for a while what
I think unfortunately what's happening
is we're taking our fear and frustration
and all the emotion that comes with that
and we're we're just kind of increasing
we're just kind of applying that fear and
frustration to all technology and I think
that is misguided And so I think most of
these laws that are either passed or in
the process of you know being proposed
and debated um are are generally misguided
because I think they're falling for this
kind of moment of moral panic around
technology that it's inherently bad that's
directly contradicted by research showing
positive effect sizes of of technology
use in classrooms for different use
cases in different contexts so I think
we're ignoring research in the name of
emotion and fear and I think a lot of
politicians are all too willing to cash
in on that to score some points with their
constituencies it's pretty low-hanging
fruit and it's pretty like acro it it's
increasingly it's it's cutting across
political divides of like this fear of
technology And again it comes from a good
place We wanna do what's best for kids But
unfortunately I think that fear has has
gotten the best of us and we're starting
to push the pendulum way too far in one
direction And in the process we're we're
increasingly stripping valuable resources
out of the hands of kids and teachers and
that is as a former educator who had to
fight tooth and nail to get devices in
my classroom and when I got those devices
it it completely changed my quality of
instruction in my classroom I find it so
so frustrating And it's hard for me to
talk about it now cause I'm building an
ed tech tool and so people automatically
assume the worst of like you're just y-you
the only reason you care about this is
cause it it's gonna hurt your bottom line
I promise you that's not it I promise
you it's it it's more growing out of this
experience Like the reason I wanted to
build ed tech to begin with was because
of how profound of a positive impact it
had on my classroom I think most of these
laws are misguided I'm hoping that we can
bend them towards positive outcomes for ed
tech and I'm happy to talk more about what
that might look like But generally I think
they're misguided to answer your question
Agreed.
Lumping everything, all technology,
all screen time under one umbrella
is, it's very misguided, and there's a
lot of misinformation, disinformation,
malinformation out there that's
really pushing that narrative.
And I agree with you.
I said, I think in both the first and
second episode I did in this series was
just me sharing my thoughts on this,
and I said, "I think that screen time
should be being evaluated," because
there is a lot of bad use of screen time.
You know, if you're only using screens
as a reward or the students that work
fast and need something to do because
they plowed through your learning
tasks really fast and it's used as
some sort of, like, reward, is-- if
that's the only way you're using tech.
Or there's a huge difference
between digital curriculums and
instructional technology tools.
If you are only having students
working in digital curriculums where
they're watching videos and answering
questions, they're not creating.
They're a very consumption-based tech
that's very, very, very, very different
than in true instructional tech tools.
No I I totally agree with with everything
you just said and I it's it's a part
of A larger point that I think is
really important It's it's like ed ed
tech quote-unquote is not one thing
It's thousands of things It's it's
what you're describing It's like it's
digital curricula it's instructional
tools it's administrative tools it's
A large umbrella
It's it's a It's a huge umbrella and
we're lumping them we're lumping all of
these interventions literally thousands
of interventions under one label and we're
saying it is inherently bad because of
the medium and researchers So the most
influential of the researchers that I know
of that have talked about this is John
Hattie I know that's a name that a lot
of folks in education will recognize just
because his meta meta-analysis work in
visible learning is very influential and
he had a chapter on technology influences
in that book And in his most recent one he
he literally said something to the effect
of like If there is to be a third version
of this book we're not gonna have a
chapter on technology because it's absurd
to refer to technology as an intervention
in and of itself It's a medium through
which thousands of interventions can be
implemented and so we need to instead
measure that Like the use cases and the
context and and the tools themselves
I think that's a healthy perspective
and that's coming from arguably the
most influential education researcher
of the last 20 years we're ignoring
that advice in our current moment and
Mm-hmm
I'm talking about when I talk
about we're ignoring researchers
in our current of moral panic and
I find it very frustrating
It's like saying, this is a good
analogy I have right in front of me.
Two learning tools that I use all
the time when I'm teaching: a pen or
writing utensil and Post-it Notes.
It's like saying all pens are terrible,
and we should get rid of every pen
in every classroom in the world,
and there's no good use of a pen.
The that's literally what
this argument comes down to.
Or Post-it Notes, all
Post-it Notes are bad.
They're terrible for the environment.
They're using trees to create them.
We should never, ever, ever, ever use
a Post-it Note as part of learning
It
ever again.
it's like it Exactly It's it's like
it's like saying like What's the
effect size of a chalkboard Like
well depends on what you're using the
chalkboard for So if you The larger
Yeah.
if you strip if you strip down the
arguments being made here what you get
what it what it reduces down to eventually
is 1950s and 60s media ecology philosophy
like Marshall McLuhan and others where
basically like the medium is the message
The the medium itself is what determines
the message And then in this case they're
saying like the medium of technology Is
inherently distracting you use technology
you will be distracted and that's that's
it Kids can't handle it that philosophical
perspective was written off years ago
it is technologically deterministic
Like humans have free will We can bend
technology to serve our interests and
serve our needs and we do that every day
in classrooms through very simple things
like content filters or through basic
classroom management processes that I'm
happy to get into and talk about and I
mean this literally like I I I think I
was aware of this anti-edtech movement
pretty early in the game because I I
went to a conference in oh my almost
three years ago in Delaware and Jar
Dr Jared Cooney Horvath who is I would
say is the leader of the anti-edtech
movement His book The Digital Delusion
is what is testimony to the US Senate
a couple months ago like is what has
Is he the one that compared it to anthrax?
Was that, was that the
guy that compared... Okay.
All right.
I mentioned that in the first or
second episode that, yeah, there
was a guy, I didn't remember his
name, but that compared tech use
to anthrax, basically saying that
like anthrax, none of it is good.
Yep
Yeah.
uh uh just the fact that someone's
willing to make that comparison I think
gives you a sense of I I don't think
they should be taken seriously if you're
making those types of comparisons in
front of the US Senate Like that's a
silly and dangerous analogy to make
It is.
It is dangerous.
It really is.
It's a dangerous analogy.
is Um and and he So he was the keynote
speaker at a conference I went to in
Delaware like three years ago It was
ResearchEd and was the he was like
the final speaker of the conference he
sp Half of his talk was was literally
referencing like 50s and 60s philosophers
on media ecology It's like interesting
work and it it's certainly something we
can debate amongst you know uh uh higher
ed circles and and folks interested in
like philosophy and theory not research
and it shouldn't be used to guide
education policy And it's insane to me
that that's what we're using basically
it's disappointing on so many levels
so yeah I'm happy to get into any of
the details within there but it it it's
it's it's been distracting for me as
someone like building edtech It's it's
been hard to focus on the work w in this
when when you're living through this
moment where people are almost talking
about the work you do as if you work
in like like the tobacco industry or
Yes.
Yes!
It's crazy
I shared in a couple episodes ago too
that I have had some very interesting
and negative interactions in the past
few months especially when I tell
people what I do, because they read
The Anxious Generation, and they think
that no matter what, all technology
is now, you know, frying the neurons
in our brain and that kind of thing.
I do wanna dig into research
more because multiple things.
One, you and your wife, 'cause your... So
your wife's did y- master's degree too.
You both did your master's
degrees at Stanford, right?
Kind of at the same time.
And your, and your wife's degree
is more research education?
Tell... I can't remember for sure.
No that's a good co so yeah we did So
we both did our master's at the Stanford
Graduate School of Education Mine was
in learning design and technology so
it's more I don't wanna say it's more
pedagogy-focused The the folks in
EDF will be mad about that label But
it it's more focused on like solving
problems in education using the unique
affordances of technology to help
solve those problems Because There,
are unique affordances of technology
that can help solve those problems
There there's a whole degree at...
There wouldn't be a deg- a master's
degree at Stanford University
if there wasn't evidence backing
the use of technology in learning
I y yes thank you Like w we're writing
off an entire field of validated research
I I I met with my program director just
yesterday to talk about this Her name's
Dr Karin Forssell She literally wrote
the book on efficacy and ed tech efficacy
and like how you structure a learning.
tool to actualize best practices in
research so that's LDT She's wonderful
That's Karin Forssell She's the program
director But then my wife studied
education data science so she studied
Mm.
it's a lot more obviously data
science intensive and so it is a bit
more research-oriented as opposed
to like practitioner-oriented
Yep.
And so she did some original research in
Dr Dora Demsky's lab on AI and its ability
to provide feedback on student writing
and of bending AI feedback towards best
practices and and feedback more generally
and so I I think we bring a really
healthy lens to this current moment of
like AI and ed tech in education because
she has a deeply technical understanding
of AI and machine learning and natural
language processing and and I think I
have the pedagogy side of things cause
I taught for seven years and I'm know
I've worked in schools And so I think
we bring a healthy perspective to it And
I I training and education we received
at Stanford like I was talking to Karin
about this yesterday I feel like it
prepared me perfectly for this moment
cause there is a whole body of research
to support technology as a meaningful
technology is a broad label but like
technology can be used meaningfully
to improve learning outcomes for
kids So I'm happy to fight for it
You just gave, I feel like, your
credentials as one of the top people I
know to refer to, to actually say what
does the research say, and make sure
reciting hi-high causal research as well.
You dug into it a little bit but
tell us more of like what does
the research actually say about
technology used as part of the
learning process in K-12 in particular?
And I know, again, I'm lumping a big
umbrella of technology but especially
looking to-towards the, you know, is it
frying the neurons of our brain argument.
Yeah it can if it's used
really poorly If if you're just
parking a
kid in front of YouTube and being like
Hey dude have some have some Chromebook
time Like of course yeah but it can
also be used the exact opposite way
and have extremely positive outcomes
especially when you lean into the unique
affordances of technology So it depends
on how you use it and I that's not an
answer people like They want a blanket
statement of like bad black or white It's
neither It's gray It it depends on how
Yep.
It it technology This is a Sal Khan quote
I use it all the time Technology is an
amplification of human intent If you're
a really great teacher can use technology
to make you a super great teacher
Yep.
Give you superpowers, what I always say.
Yeah
Research supports that If if you're
if you're a burnout teacher you don't
wanna be in the classroom anymore you're
just kinda doing it cause you whatever
five more years and you can get to
retirement you just kinda roll in and and
technology's a great way to just kinda
park the kids on something and keep them
busy you can do that too And and Yeah.
kids can fry their neurons just doing
whatever they want on the internet
it depends on how the tech is used
and I think the research supports
that i-in terms of like talking about
blanket research kind of writ large
the closest thing you can get to that
are meta-analyses cause they look at
like a a huge swath of studies within
different domains so that's why I cite
the John Hattie research cause he's
looked at like for example he looked
at the use of technology and its use to
provide feedback student work and found
an effect size of I think 0.5 standard
deviations which is a large effect
There's lo-- Yes, very large.
Largely positive so that is one simple
use case a positive use case And and
this is this You don't need research to
tell you this if you've used technology
in the classroom It's like well I could
manually collect 120 papers that kids
have handwritten and spend my weekend
reading through it and hopefully I
can get them the feedback back by like
Next Thursday and they handed it in on
Yeah
more than likely because I'm having to
read through maybe like whatever like 20
of like 120 of these and try to provide
comments and things and by the time I
get them the feedback back will they
even remember what we were working on or
I can use a wonderful new AI tool like
Short Answer or pick your favorite cause
there's many tools that will do this
now And the kids submit their let's say
exit ticket they can get feedback on the
quality of that of their response And,
explanation before they leave the
classroom And so they they can then use
that feedback to improve what they're
you know their understanding again before
they leave the classroom that's one
simple use case that when you extrapolate
that out and look at studies across many
different contexts again effect size
of 0.5 like super positive are other
domains where we could look at There was
recently a meta-analysis done by Rebecca
Silverman at Stanford She looked at This
was in the context of early childhood
literacy so teaching kids how to read
and write generally positive effect sizes
It was a very nuanced study and there's
like literally it comes down to like
specific features and apps that do or d
don't support early childhood literacy
outcomes but effect sizes ranging from
like 0.2 to 0.3 which is like generally
positive now this is a whole other
area that we could get into if you want
to around what qualifies as a large
effect size Cause that's another thing
that the anti-ed tech movement is is
doing at the movement sort of positive
research about technology cause I've
And
Those are large effect sizes So there's
a researcher at Brown his name's Matthew
Craft he has pushed back on this narrative
that has grown up in education in the last
15 20 years that came from John Hattie
that a large effect size is is anything
0.4 standard deviations and above If
it's below 0.4 standard deviations and
below it's a small effect size actually
Jared Cooney Horvath used that reference
point in his testimony to the Senate
He he literally subtracted that hinge
point to take away from edu educational
technologies effect size which is
absolutely insane and total pseudoscience
But but people have pushed back on that
0.4 standard deviation specifically
Matthew Craft of like if you look at
just at meta-analyses like the average
effect size is really small for education
interventions So actually like anything
that's I think is is 0.2 and above is
actually quite large in education like in
terms of what a positive effect size would
be And so I try to explain that to folks
and it's just like Well no no no just
no That that that evidence doesn't count
Mm.
that's what makes me think that we're in
a moment of moral panic cause it's like
these are researchers Like these these
are people that like Spend their time
reading peer-reviewed research in journals
Yeah, their life is
literally dedicated to this.
Like, this is what they do
with their life every day.
And they're
to them, like, who do we listen to?
Yeah.
Exactly And man there's a whole other
post Substack post coming soon about
like reform needs to come from all
sides Yes at a policy level Yes at an
ed tech level implementation level It
also needs to come at a research level
because research community in education
is a mess and leads to unclear guidance
to schools on what is best practice
And it's partly why we're in the moment
that we're in and arguing over what a
what qualifies as a large effect size
is a perfect example of that So that's a
really long-winded way of saying research
supports technology in schools when used
well it's just that we're ignoring it
So Matthew Craft, just to make
sure I understand correctly, his
argument is, so like Hattie's
saying 0.5 effect size and above,
Point four
saying actually... Oh, 0.4, sorry, 0.4.
Matthew Craft is saying that
actually in education and the way
education studies work, that we
can go down to a 0.2 effect size.
Yeah and his argument there which I think
is real cause he's replicated it now twice
is basically that Hattie threw everything
in including like correlational studies
which tend to have higher effect sizes He
threw in studies that where the measures
that were being used were created for
the specific test that was being done
as opposed to like broader measures like
standardized tests and things like that
Which tend so you tend to get higher
effect sizes So basically to to put it
into layman's terms like Hattie threw in
all the research It's kind of like a hot
dog It's easy to consume but you and and
tasty but you don't wanna know what's in
it whereas Matthew Kraft only looked at
like high quality like randomized control
trial-level studies And when you look
at that majority like almo I think it's
like 33 it's like in the 30s percent of
studies have no effect on outcomes Cause
it's really hard to have an effect on
hard
cause education's really hard.
There's so
Yeah.
at play in
So many fac- yeah, it's really hard
to say, like, this is the factor that
improved student learning outcomes
or this is the factor that didn't.
So that's another reason why just having
this blanket technology is the reason that
test scores are down, and technology is
the reason that engagement is down, is
really, really flawed thinking, because,
Subscribe
we're, we're not taking into
account testing becoming more
and more a thing in schools.
The pandemic causing
huge amounts of trauma.
I just read a whole book on trauma,
and now I'm on the trauma train,
because I knew this, but this book
was so enlightening to me at how much
trauma affects our brain, and every
brain inputs trauma differently.
Sometimes what is, you know, of like big
T trauma, little T trauma, people say.
Sometimes what one brain interprets
as little T trauma, another
brain interprets as big T trauma.
And the pandemic, I believe, caused
trauma in every single one of us,
adults that have fully developed
brains, and we haven't even begun to
think about the trauma that happened
to the kids from 2020 to 2022.
And I think there's a lot of
brains that had a lot of big T
trauma, and when you have big T
trauma, your brain just shuts down.
But that's not part of the conversation.
Yeah
The correlation does not equal
causation argument, right?
Like, we're saying, "Oh,
I listened to this NPR."
Actually, honestly, I
didn't make it through.
I need to go back to it,
but I got so frustrated.
It was an NPR segment.
I can't remember the exact, And
usually I really-- I love NPR.
My husband's an NPR addict.
He's the one that sent it to me.
I can't remember the name of the
segment, and usually I really like this
journalist and I listen to it, but it
was so one-sided and it was the, this
principal arguing that at the same
time that we put Devices in schools and
that her middle school went one-to-one
Chromebooks was the same time that test
scores and engagement started to go down.
No, that's correlation-- Correlation
does not equal causation.
You cannot say because this
happened, it's because of this.
Yeah Yeah Yeah Yeah and it it's wild
that we're prioritizing correlational
studies over causal studies
Yes.
that that that kinda shows me this is
about moral panic and emotion more than
it is about evidence cause yeah I mean
what else correlates correlation we could
draw is that around that same time kids
started spending seven to eight hours
outside of school on their cell phone or
Yep.
devices
Yep.
of going home to read Harry Potter we're
going home to scroll through TikTok That
may also play into it into this as as you
mentioned as does the trauma of a pandemic
as does I mean a generation of kids
that was raised on reading methodology
that has increasingly been questioned
with the science of reading
movement we could go down a list
of 1,000 different reasons why test
scores are are likely declining cause
it is real that they're declining
Yes.
but drawing this simple correlation
of like we we bought computers
and scores went down It's it's
Yep.
correlation I you can imagine So it's
it's not I don't think it's strong
evidence and I think the people that
would tell you that they support this
like science of insert your favorite part
of education science of movement tell
you we need causal evidence To support
interventions if we're gonna put them
in schools And simultaneously they're
ignoring their own advice in this moment
and being like No actually correlational
evidence is enough it's it's pretty
absurd So it it's frustrating but I
am hopeful and I wrote a Substack post
about this just yesterday I am hopeful
that like it can this current moment
can push for some positive reform cause
I do think edtech does need some reform
Agreed.
We wouldn't be here having
this conversation if it didn't
need reform, is my opinion.
no it does need reform Th-there are
some flaws in the industry that should
be addressed but stripping resources
out of the hands of teachers is worst
possible way to go about doing this but
unfortunately it's just the easiest thing
to legislate so it's what we're doing
Yep.
here we are
I also wanna talk about efficacy,
'cause I know this is something I've
ta- I-- we've had conversations about
this, and ed tech efficacy in general.
If you're not familiar with that
terminology, it basically, efficacy
measures whether a tool, in this case
we're talking digital learning tools,
improve student outcomes or not.
There's a huge conversation in the
ed tech world about how do we measure
ed tech effic-efficacy, and how
do we actually know if an ed tech
tool is improving student outcomes?
So that's what makes this really really
hard and that's why people go to school
for eight years to study this and then
spend their lives trying to figure this
out it's educational re education is a
social science Social science is hard
cause there are so many variables that
you try to control are almost impossible
to control in the way that if you wanna
compare it to medical science um like that
that you can control in a in a human's
body as opposed to like a classroom
Right
speaking to answer your question how do
you measure efficacy The gold standard
of efficacy in technology and a lot of
education in general is a randomized
control trial or if below that it's
like a quasi-experimental study At a
very broad level we're gonna paint with
a broad brush here cause we could get
into nerdy specifics but I don't think
we need to turn this into a research
methods grad school class So like at
a basic level it's you go to a a large
subset of kids ideally Some of them get
the intervention So let's just go with
Short Answer the platform we're building
These kids will use Short Answer during
this period Let's And this subset of kids
that's been randomly selected a a large
number of them they're not going to use
Short Answer They're gonna do business
as usual will these We'll we'll come up
with a different plan et cetera et cetera
We'll do like a pre/post let's say So
we'll look at their writing scores on the
state test the year before and then we'll
look on the writing scores on the state
test the year after and we'll compare the
control group the the kids that didn't
use Short Answer to the kids that did the
experimental group we will subtract them
and then we'll divide it by the standard
deviation within the test scores find
an effect size So when I say effect size
that's what I'm referring to It's like a
0.2 a 0.3 And like what qualifies as big
Well if I would say if you I'm gonna go
with Matthew Kraft if the deviation is
0.2 or above you have a a medium to large
effect size If it's 0.05 and above it's
it's a smaller effect size Sorry If it's
0.2 and above it's large If it's 0.05
to 0.2 it's medium If it's below 0.05
it's it's small and that's a randomized
control trial Now h I need to complicate
that so hooray we got our standard you
know standard deviation and it's wonderful
The problem with that in educational
technology is what you talked about at the
beginning of the podcast which is edtech
moves fast and it changes a lot Like
a product is never built You're always
building it so the problem with that is
okay we we run a randomized control trial
It's great We we 0.5 standard deviation
effect size Amazing You know Look at
this W we improved learning outcomes
Well the problem with that is if in the
process of doing that research you also
added 20 new features to your platform
that completely changed the what you do
All, all the time.
which happens all the
Yeah.
technology And that's a positive thing
And it's a good thing.
Yeah.
a great thing Like if we have a new
whatever new research comes out that like
actually if you do it this way it's better
we can quickly change it right If you
have a textbook good luck Like that thing
ain't getting updated for a decade Like
Yep.
are so slow
Yep.
positive affordance One of the
downsides is that you end up
measuring tools that no longer exist
because like it changed over time
And so the
best advice I've seen on this this comes
from who Molly Zimmerman and Jennifer
Carolan They wrote an article that we read
in grad school at Stanford I think the
healthiest approach to efficacy in edtech
is a portfolio approach where you build
out multiple different studies of varying
sizes and quality over time because that
better enables you to track your product
as it changes over time But just as
importantly it allows you to measure your
impact in different contexts over time
Cause the challenge of edtech that it's
used at scale Like we've got kids in rural
West Texas using Short Answer and we've
got kids in urban Beijing using Short
Answer Those are very different contexts
And so the way that tool works and is used
in those contexts is probably completely
different you need a smattering of
different research into a portfolio over
time to speak to those different contexts
because education is deeply contextual so
like the randomized control trial is the
gold level you know whatever the the high
the highest bar o of efficacy And then
I suppose a meta-analysis of randomized
control trials would then be the only
thing that would be above that but for an
ed tech company because you're constantly
changing I think that portfolio approach
is healthy And that's why if you go to our
website sorry for the shameless plug if
if you go to myshortanswer.com and click
on the Our Pedagogy page there's a little
button that says See our portfolio of
efficacy everything ranging from a logic
model which is basically just like Here's
what our app does Here's the research
that supports every different part of
our app and and why it does what it does
That doesn't mean that our app works
it just means that there's at least a a
theoretical explanation of why it might
work and then you'll see our learner
study that we ran as a part of my master's
capstone project and then you'll see case
studies that we put together based on
surveys and triangulation of interviews
and several other factors And what we're
doing is slowly trying to build up towards
the causational evidence that I just
talked about like a randomized control
trial but we haven't got there yet because
RCTs are expensive and take forever
and for a small
company like us where we're still building
a product and changing things dramatically
from day to day it doesn't make sense So
I think that portfolio approach is healthy
and I think here's a healthy reform idea
require that an ed tech product at least
have a logic model or theory of change
in order to be used in schools It's a
very simple way to try to bend adoption
towards efficacy and towards quality
opposed to what we were talking about
at the beginning of the conversation
where it's like about the medium or just
about X number of minutes per day or
like silly things that aren't aligned
with research that could be a very simple
step forward towards improved focus on
quality that was a long-winded response
but this was literally my master's so
Yeah,
y
that
hit on a
was fantastic.
I like talking about
That was fantastic, and I love
that you noted that it, it's also
so tricky with ed tech research.
It's not like researching a
textbook-based curriculum That is static.
And I, I'll, I tell teachers this,
I tell ed tech companies this: the
small, tiny little things are what
make or break an ed tech product.
Like, the teeny tiniest little details.
So you could make a really small
change, a UI change, a UX change, a
design change, a pedagogical change
in a product, and it can completely
make or break the teacher using it,
how the teacher uses it, and that can
completely change a study that was
done prior or post that change, or both
You're 100 right?
and I've never had more appreciation
for that concept than I do now
as someone building this stuff at
I bet.
if we choose to make this little user
interface element a tab instead of a
toggle that can have a profound impact at
scale So yeah it's it's fascinating And
this is what I love about the work You
go from 10,000-foot level conversations
about what qualifies as an effect size
a positive effect size in meta-analyses
and and then you zoom all the way into
like what should the radius of this
of the corner of this button like how
Should it be a three radius or a five
radius when I design it in Figma So it's
like it's granular and it's it's it's
universal all at the same time and it's
fascinating work cause it all matters
Yeah, and the more you get in and use
and explore ed tech tools, the more you
start to notice those little tiny details.
For me, when I'm evaluating an ed tech
product, I look for those little tiny
details, and that's a big indicator
to me of the quality of the product.
Last question on this topic,
and then we'll move on.
If you are a school leader, a teacher
listening to this, and you're like,
"I wanna make sure that I am using
high-quality ed tech tools in my
classroom," or a school leader that's
like, "I wanna make sure I'm putting
high-quality ed tech tools in the
hands of my teachers and students,"
what would be your advice to them?
What would you tell them?
First I would take matters
into your own hands and what I
mean by that is if a teacher's
requesting to use a tool go to the
Not
look at it Do they have a page or
somewhere where they're explaining
the research supporting their tool
And when I say research I mean
research What what you'll find is
there are a lot of tools that they
a white paper.
No, not a white paper.
a lot of vibey hand-waving that happens
with like our learning principles and
it's just like this sort of like vague
explanation of like constructivism
It's like okay that's nice but
that's not research That that's just
Yeah
That's
That's a philosophy.
I
you're
It's a
aligning your tool to an
educational philosophy, which is
not bad, but it's not research.
Yeah.
It's not bad But it's like what are the
specific things your app is doing and are
those things supported by research And
that's called the it's called theory of
Common
it's called logic model But that
to me that's that's the baseline
of efficacy And under the Every
Student Succeeds Act that was
Sense
by the Obama administration they
Media.
created these tiers of
efficacy and the first tier the
Oh, they did?
I didn't know that
I'm at least looking for that.
baseline Does it have a logic model
Does it have a theory of change Do they
even have somewhere on their site where
they're talking about efficacy Because
if not alarm bells should be ringing It
doesn't necessarily mean that a tool isn't
research-supported or backed or aligned
to best practice if it doesn't have it
but it's a pretty strong indication that
the people building it haven't considered
that which to me is a sign that you
know it's a strong signal I that would
be my simplest advice There are other
places though that schools can go to get
advice on this stuff ISTE has a product
index that's pretty good where they
kind of rate and review and kind of vet
technology EdSurge has one Common Sense
Media used to have one but they gave up
Okay.
I they did so their product index is
I think it's still around maybe but
they're not updating It I don't know
I know Johns Hopkins University
has one It's called like
Okay.
ooh something but the the problem
with all of these is like a a lot
of administrators don't know they
exist and a lot of them also just
like not up to date cause it's hard
to keep up to
date with the edtech market
They change all the time.
Yeah.
all the time right So it's like
I applied last year and I was
approved and got a good score
Mm-hmm.
Mm.
if I added 15 new features I guess I'll
have to apply again And then like even
large organizations like Common Sense
Media can't keep up and that's why I do
think we need some sort of regulatory
body that can like actually be the stamp
of approval just like the way we have the
FDA and like food and drug need we need
the equivalent of that in in education of
like a stamp of approval from a verified
government source that's like This is
good and okay cause that is w going back
to like the flaws of the industry one of
them is there's a lot of garbage efficacy
numbers being thrown around that are based
on not real research It's just kind of
like testimonials and sort of like we're
we put together our little case study and
we're pretending it's like a peer-reviewed
randomized control trial and in reality
it's just like an interview with one
teacher that had a good experience Like
that type of stuff probably should not
be allowed or if it is it needs to come
with a label that's like This statement
has not been approved by the Food and
Drug Administration You know what I mean
so th-there is a lot of fake efficacy
going around That's one of the major
flaws that hopefully can be addressed in
this current moment of angst but now I've
gotten off topic So to go back to your
original question look for a theory of
change look for a portfolio of efficacy
on the website then in lieu of that
maybe go to one of these product indexes
and see what other people are saying
Yeah.
if you're looking at using an ed tech
tool call the district down the street
from you that maybe already is using it
or had teachers and get their perspective
Unfortunately we A lot of times in school
districts we live in silos where like
even we're like right next to each other
but we don't talk to the district over
about what's working and what's not so
I think those conversations could be
healthy too But just even just taking a
more proactive stance of like I'm gonna
look into the efficacy before I approve
a tool that in and of itself would be
a step forward cause oftentimes that's
not happening So maybe this current
moment could be good for that at least
Switching gears, I wanna make sure we
talk more about Short Answer, 'cause
it is a really, really amazing tool.
Pretend I am a skeptical English
teacher, and walk me through Short
Answer and why should I spend my
very limited time to explore Short
Answer and try it with my students?
So if you're a skeptical English teacher
then I'm gonna start with the why I'm not
gonna start with the product Let's look
at our current moment We now live in a
world where AI can write for you like a
machine can can write it's a momentous
it it's a crazy moment in human history
that completely upends I think how we need
to approach we teach and assess writing
It doesn't devalue teaching kids how to
write It is still critically important
to learn how to write But I do think it
changes how we go about teaching writing
the first thing that I would say once
writing leaves the classroom which it
necessarily needs to because oftentimes
it just takes a long time to produce
longer form writing writing leaves the
classroom it's hard to know how much of an
influence AI is having on that student's
process a very simple step forward is
we need to do more writing in class will
probably look like shorter form writing
I also think you know the first thing
I see when I log into Gmail in twenty
twenty-six is a thing that says Hey want
me to write the email for you Our kids are
Wait for me.
and
Yep.
and that button's gonna go everywhere
So that's gonna result in less writing
practice Our kids already struggle with
writing The less you practice the more
your skills atrophy Troubling situation
We can't just say Well the English
teacher needs to do more writing We
don't they don't have the bandwidth It
it's gonna take a team It's gonna take
have to embed more writing across the
curriculum I think that's kind of step
number two where it can't just fall on
the English teacher anymore It needs to
be embedded across the curriculum That's
been advocated for for decades It's been
historically really challenging to pull
off but I think any writing tool needs
to be built with that in mind and and
that's partly why we built Short Answer
And then the third piece that I would say
to address this challenge is it has to
be an engaging experience It has to be an
experience where the kid is gonna fall in
love with writing cause how on earth are
we gonna motivate kids to want to learn
how to do this really really challenging
thing arguably one of the hardest
things you learn how to do in school
Cause writing is so cognitively complex
hard.
How are we gonna motivate a kid to wanna
to wrestle with this and struggle and
fail at it where they can press a button
and it'll do it for them in ten seconds
So the third piece is I struggle to
say engagement cause then we fall into
like well it's just gonna be a game and
silliness But no but it needs to be an
experience where a kid falls in love
with writing and reaches a flow state and
just like loves doing it inherently And
so those three pieces more in class more
across the curriculum and more engaging
I think are critical and that's why we
have built Short Answer So I like to
start with the why especially when talking
about cynical English teachers and cynical
teachers in general about why should I
use your ed tech tool That's our why The
way that we do it with Short Answer is
through short form social writing practice
that's intended to be em-embedded across
the curriculum combining daily formative
assessment of whatever skill you're
teaching with writing instruction So
teacher pulls up code kids sign in kids
see prompt Prompt could be anything from
if you're in English class copy-paste in
your opening paragraph to the research
paper we're writing if you're in science
class explain the process of cell division
as an exit ticket right It it could be
anything It could be assessing content it
could be doing literal writing instruction
Kids do it submit it then teacher clicks
begin feedback Going back to Hattie's
effect size point five effect size of
of using technology to improve feedback
processes in in classrooms kids either
get feedback from their peers and that's
a huge part of what we try to do cause
we think inherently motivating for kids
because it creates an authentic audience
for their work of like Oh my gosh my
classmates are gonna see what I wrote
I should try hard And it's anonymous so
we do it safely but I think that's a key
part of it Or they get feedback from AI
or they get feedback from both And I think
there's some really interesting things
you can do with augmenting AI and and peer
feedback so they get that feedback they
reflect on it And then as a class then the
final step is discussion and reflection
So every Short Answer activity follows
that same kind of core structure of do
some writing get some feedback and reflect
on the feedback and then ideally go
back to the do some more writing because
we're using that feedback to improve our
understanding and using that discussion to
improve our understanding so that's Short
Answer as it currently exists We have a
whole new host of features coming soon
Oh.
are gonna completely augment that
entire process it's gonna lead to some
massive changes to Short Answer this
summer So stay tuned for that just
gonna tease them I don't wanna say
Exciting.
But that's the core idea and it's very
straightforward Our kind of philosophy
going back to your point on design is
like we wanna do a f only a few things
but we wanna do them extremely well as
opposed to trying to be like the we do
a bajillion different things type of
tool so it's very simple and I think
in that way it serves us well And so we
have about a hundred thousand monthly
active teachers and students across all
Wow.
states and several countries at
this point And but that's that's
kind of the core of it both what
it does and the why behind we do
I love it.
It is-- it's so engaging.
Students love it.
Like, they really do.
It really gets them
excited to want to write.
And that anonymous peer feedback element
really motivates them to try because they
don't wanna submit something terrible
That their peers are gonna look at and
tear apart and get horrible feedback on.
So it really does motivate.
And I love that you hit
on the engagement point.
I, I talk about... And I talk to teachers
about this a lot when I'm training
them about our educational philosophy.
I'm not having them write like they're,
you know, I'm sure you had to do it in
your teacher prep program where you had
to write your philosophy on education.
We don't go that far.
But I do like to get them to
really think, like, what is
your philosophy as a teacher?
What is your role?
What is your purpose as a teacher?
And some teachers really think that
they are there to get kids to do
hard stuff, and that kids should
just be able to, that grit, just
plow through and do the hard thing.
I don't need to try to engage them.
I just feel so strongly against that.
And the more that I teach, work with,
talk to Gen Z and Gen Alpha learners,
the more I realize that they are so
different, as in every generation.
Every generation is significantly
different than the previous
generations on a whole, right?
Lumping them all together.
Because we have different experiences
growing up Now technology has just
changed the generational experience
immensely, and I really feel that if
we don't engage Gen Z and Gen Alpha
learners in a different way than we
have previous generations, that we are
just gonna see test scores just plummet
Yeah
and down and down and down.
And Short Answer just is really great
at hitting that, the combination of a
little bit of, like, gamification with
a little bit of peer feedback, with the
AI feedback, with the teacher leading.
The teacher leads the entire activity,
that it is very engaging for Gen Z and
Gen Alpha learners in my experience.
More Gen Z. I've, I've seen it
more with Gen Z than Gen Alpha.
On that note, is there a-- Do you say
with Short Answer middle school, high
school only, high school only, all K12?
Can I use it in higher ed if I'm
a higher ed person listening here?
Good question It's generally
second grade through 12th grade
Okay.
I'll be honest we get some classes it goes
like gangbusters and they're like Heck
yeah this is amazing Our second graders
are writing And then in some classes
it's like This is way too advanced So
it d it
just Depends on the
Depends on where they are.
Yeah.
spec that's specific with second grade
Yep
it's third th third through 12th at the
higher ed level it's a bit we've had some
some folks use it with some success but
it's it tends to be K-12 oriented And
especially when you use the tool you'll
kind of understand why It you can tell
from the design it's oriented towards
K-12 going back you said something that
was really interesting and something I
think about a lot when you're talking
about engagement and what does it
look like and I think that's like what
partly what makes designing ed tech
so interesting is like wanna create an
engaging experience that a kid wants
to do But you don't want it to spill so
far that the kids are just conking out
kind of playing games and not really
Yes.
Yes.
I I do think in the in the larger context
of this conversation around ed tech reform
and why people are skeptical of ed tech
is there are a lot of tools that have
kind of leaned really heavily into sort
There's not
the candy side of things where it's like
yeah the kids beg the teacher to do it but
the kids beg the teacher to do it because
just playing games for 20 minutes Like
a huge amount of learning value.
Yeah.
Yeah.
So it's this really fine line to walk
and I always get nervous when I even
say the word gamification because I
Yes.
kind of
Yes.
go off for people like no no no
it's I mean if you use our tool
you'll get a sense of it It's like
pr the gamification is Very light.
Very light
that we're hoping the core engagement
comes from what you just talked about
which is like Oh my gosh my classmates
are gonna see what I wrote And and
there's something just intrinsically
motivating about like creating something
and sharing it with people I'm hoping
that W-which I think is it's not
extrinsic it's not points and bells
and whistles and badges it's creative
and and sharing and and social in
ways that I think are more healthy and
Which is how Gen Z and Gen Alpha,
that's the world that they live in.
They live in a world of gamification.
I actually have an episode coming up
here in a few weeks, closer to end of
June it'll come out, where we're really
diving into gamification versus game-based
learning and the misconceptions.
There's a huge misconceptions of
not understanding the difference.
Gamification is... Well, actually,
let me start with game-based learning.
Game-based learning is you're
learning through a game.
You are playing a game, and
it's the learning process.
Gamification is we're adding
elements of gaming to learning.
So points, that's
gamification right there.
Badges, leaderboards, XP,
that's s- kinda same as points.
That's gamification.
So huge misunderstandings between
the differences in the two, and
I do think that both have gotten
also a bad rap 'cause of a lot
of misinformation around it.
And there are products out there
that have gone too far with
Yeah
gamification.
think that that hasn't helped the bad,
but The way that Gen Z and Gen Alpha learn
and the way their brains think and the
world that they live in of social media
and interactive internet experiences,
they really do engage to those small
gamification elements in my experiences.
And that's where ed tech can,
if done right, can really help
with that engagement factor.
And it's so ironic because one point
we're saying out of this side of our mouth
that Gen Z and Gen Alpha aren't engaged.
It's really hard to engage them.
They're checked out of learning.
And the other side of our mouth
we're saying, "Oh, gamification
is terrible and never use it."
Right
Super ironic because if you do it
right, gamification is a really
powerful way to engage Gen Z and
Gen Alpha learners in my experience.
And I do think Short
Answer is doing that right.
And I'm not just saying that
'cause Adam is on the podcast here.
Believe me, if anybody knows me,
they know that I speak my mind
and I tell it how it is, even if
people don't want to hear the truth.
So I wouldn't be saying that
if I didn't really think that.
Well thank you and I also think this
generation of kids For that matter
millennials and yeah Gen Z and anyone
that was raised in this like tech
era th-there's a desp desperation
maybe is a strong word but there's a
strong desire for social connection
Yes.
patterns in society And I think
really good ed tech going forward will
be tech that finds ways to sort of
human interaction in the classroom and I'm
I'm hoping that's what Short Answer can be
Yep.
example we have an activity where okay
kids submit writing get feedback Did
the class hit a score goal together
That's kind of the gamification elements
team-based But then there's an option
the teacher can click that's like reflect
together and it pulls up names on the
board and it's like Go sit Adam go sit
with Lindy and you know Jimbo go sit
with Kearney the kids have to go then
sit together then they they together go
through a structured reflection where
they're talking through what they wrote
about why they wrote about it the feedback
they got if they agreed with the feedback
that's finding a way to use technology to
like get kids sitting together talking to
each other socially i-in a structured way
that supports learning through this like
think pair share model So I wanna really
lean into that as we continue to build and
and scale Short Answer and that's gonna
be part of the updates that you'll see
later this summer I keep teasing those
Yay.
I'm excited.
I'm excited to see them.
You know, Short Answer, if you're one
of those that are in the camp of you
think AI is the end of humanity and it's
getting rid of critical thinking and,
you know, all the terrible things, that,
that end of the AI spectrum If you're
at that end, you should go check out
Short Answer because it is developed
in a way that it pulls the very great
powerful parts of the technology.
And not just AI in general, but if you
are on the spectrum of ed tech where you
do think that tech is like, you know,
ruining the neurons in our brain and
there's nothing good about it, as well as
AI is the end of humanity, Sh- my Short
Answer is a great example of how you can
pull the really powerful parts of the
technology to do things like anonymous
peer feedback that would-- That super
powerful engagement element that would be
very hard to do without the technology.
So there you go.
I was just getting th that you've
you've identified an affordance like w
w going back to this larger conversation
about ed tech It does have unique
affordances that that enable it to improve
learning in unique ways One of them is
anonymity when it comes to writing It's
Yeah.
for stuff to be anonymous which can be
obviously also has has some downsides But
in the context of providing peer feedback
on writing that anonymity is much easier
to pull off digitally which is partly
why we've built Short Answer the way it
is So just wanna double-click on that
Two more questions that I know teachers,
educators listening probably have
if they're interested in trying it.
Number one, go to myshortanswer.com.
Is this only for English teachers?
I feel like you already kind
of answered this, but I wanna
specifically hit that point.
Yeah and that's why we called it
Short Answer is cause we we didn't
wanna call it like The Writing Tool
Yes.
Yes.
wh when when a when a science teacher
sees The Writing Tool they're like Ah
it's not for me we were like Ah Short
Answer that's kind of something that
is everybody uses but yeah no it's
not just for English teachers In fact
it's it's built from the ground up to
And
for for English it does work in math
works in math with specific use cases
the best being like error analysis
tasks where it's like teacher uploads
picture of incorrectly worked equation
and using writing kids explain
Oh,
and
okay.
It's not great I'll just be transparent
It's not great for writing if you're
trying to get kids to like work through
a problem because it's made for text and
Yep.
have a digital whiteboard functionality
I could see it being used for
word problems though in math.
Yeah
for concepts and and and for like
definitions and yeah No it it definitely
can be used in math I'm just being
transparent Like of the use cases I
would say generally it goes English
and then social studies science is the
next most common use case and then down
into the electives and that's probably
just a product of there's teachers th
in in the electives than there are in
the the core content area and then math
So um that's kind of the flow and but
we've we've built it from the ground
up to work across the curriculum and
sorry to keep doing this but we have
some features coming this summer that
are gonna help even more to embed it in
your content area So I'm excited to roll
those out and stop teasing them like this
When I was teaching career and tech
ed, I would 100% used Short Answer.
I taught personal finance, and my
students told me that they wrote
more in their personal finance
class than they did in English.
Yeah
'
Cause I--
that's a great that's a great thing
Yeah.
No, they, I literally had students
tell me that multiple times.
Like, "We write..." And of course, they're
like, "We write more in this class than
we do in English class." But that's a
perfect example, like career and tech ed.
Hey, no, 100% would've, would've used
Short Answer a lot actually, and it
works really great if you're like an exit
ticket person, really great for that.
I think you can really use it
as that anticipatory set at
the beginning of the lesson.
Second question that I'm sure people
have is pricing to use Short Answer.
It's interesting that you asked that
question today Just last Friday we changed
some things So straightforward response
beginning June 15th we will no longer
offer individual teacher licenses We
used to offer individual teacher licenses
for $75 per year We just announced last
Friday that in the upcoming month we're
gonna get rid of that option was a
controversial decision and not an easy
one and it was frustrating for a lot
of our teachers if I'm happy to talk
about why we chose to make that decision
if we wanna get into it and in fact I
would kinda like to cause it gives me a
Yeah, go for it
But j-j-just quickly to answer your
question So otherwise it's we do
site-wide licenses or district-wide
licenses We price that per student it.
depends on the size of your district
but right now we tend to charge anywhere
between $2 to $4 per student depending
on the size and level of adoption so
that's general pricing and pricing
structure we're happy to do some pilots
of the premium version of our tool in a
structured way to to try it out before you
buy it I think that's a healthy process in
edtech adoption said we also have a free
plan So if you're just like Dude I don't
have budget but this sounds cool and I
wanna use it fine we still have a really
robust free plan that in-includes many of
the all the el in fact all the elements
that you and I have talked about on this
podcast like the the peer-to-peer feedback
all those things So we do have a free plan
and that will continue to be available
It's just that literally last Friday we
announced that we're removing individual
purchases of licenses I'm happy to talk
about why but that's the pricing structure
We could've done a whole episode
just on, like, research and efficacy
and the screen time debate, and then
we could've done a whole episode on
short answer, and then we could also
have done a whole episode on writing
instruction in the age of AI, because
obviously you are a expert in that area.
I was lucky to get to go to your
workshop at TCA this year, yeah.
to have you
It was cool to see
you there
all right, k- chef's kiss to the
Matthew McConaughey reference there.
It was fantastic, seriously, I
am very critical of my PD time,
especially conference sessions.
A, A, don't have a lot of time to go to
conference sessions, 'cause I'm always
presenting and doing a million other
things, and B, I'm always very careful at
the conference sessions that I do choose
to go to, 'cause I have such little time.
It was very, very good.
I actually have gone back and referenced
your slide deck several times,
Yeah
which I also would say
that I, I rarely do.
It was very helpful for me,
'cause I get tons of questions on
Yeah
The writing in the age of AI debate.
A few things to hit on here.
You said that you think that writing
instruction in the age of advanced AI,
generative AI, agentic AI that we're
in right now, does need to change.
You hit on that a little bit.
What else would you add
to that for teachers?
Yeah no We lead four-hour workshops
on this so I want to preface it
that
know
not
I know,
not we're not
Oh,
justice And
all right,
Oh go
all right.
I
All
I
right
know,
Yeah exactly It was funny at the
beginning of that session to say like
How many people are here just for the
McConaughey pun And like half the people
raised their hands so But I was glad
they were there okay quickly I'll just
rattle them off Don't use AI detection
software It doesn't work that's not
gonna be a solution unfortunately
especially in a world of agentic AI even
revision tracking tools which I would
have suggested even just six months ago
moving forward it's not gonna be reliable
necessarily because you can use AI agents
to autonomously write a paper for you
and kind of simulate a revision history
We're pretty much already in that world
Yep.
So I think that the healthiest forward
addition to more writing in class and
more writing across the curriculum more
layered assessment And I talked about
this in the the TCA presentation This is
from Philip Dawson and his interpretation
of Dr Kiara Rundle's research But it's
basically thinking about any layer of
assessment that you come up with to
measure how well a kid can write or do
anything really is gonna have holes in
it that AI plagiarism can easily get
through cause AI can do just about any
task you throw at a kid where one layer
might have a hole another layer might
not It's called the Swiss cheese model
sometimes you wanna stack the layers of
Swiss cheese you know where one has a hole
another might not you're gonna eventually
block that AI plagiarism for example an
example I talk about in writing is like
okay if it's a five-page research paper
it's big summative for my eighth graders
let's say or freshmen I don't know we'll
do more in-class writing but then we're
also gonna do peer conferencing That's
gonna be a layer where they have to bring
their writing into class and and defend
what they wrote to small groups let's
say then maybe while they do that another
layer is gonna be I'm gonna go around
and have a one-on-one conversation with
students and talk to them while they're
doing the peer conferencing where they
walk me through what they've written And
then maybe we're also gonna use revision
tracking That can be a layer too Like it's
it's flawed but there's some potential
there Like it's still kinda helpful cause
a lot of kids don't have access to agentic
AI quite yet maybe if it's a big summative
big enough summative assessment maybe
we're gonna do an in-class presentation
of your paper where you walk the class
through your process and talk through Why
you structured it the way you did or what
your thesis was or you know defend what
you wrote so you're adding like multiple
different layers to this assessment such
that plagiarism might get through one or
two of them but it's not gonna get through
all of them how you go about structuring
those layers depends on the assignment It
depends on what you're teaching like your
content area I mean what age group you're
working with a lot of different things but
I think that lens is a really healthy one
of assessment layering cause that's just
kind of general good practice in formative
assessment a-anyway so we have this
whole resource and if people reach out
to me my email is adam@myshortanswer.com
I can share it about like what might a
layer look like and like do layers look
like before writing during writing after
writing There's a lot of different things
we can do But just that lens of layering
assessment I would say to keep my response
as succinct as I can that would be my
answer to like structural reform that
needs to happen to continue to accurately
and effectively teach and assess writing
in a world where machines can do it for us
I love that answer from
multiple perspectives.
One, because I 100% agree.
There is no silver bullet answer
Yeah
and I know that's super
frustrating for educators, and
that's not what you wanna hear.
And you want to hear that, "Oh, I can use
revision tracking or AI content detectors,
and that'll just solve this problem."
Silver bullet.
But the evidence is huge
It just does not work.
And exactly what Adam said, the
more advanced, especially the more
we're getting into agentic AI, even
that tracking revision history,
it, it just is not gonna work.
It could be one teeny, teeny
little tool in your layers.
One, one teeny little layer in
your layers maybe, but it can't be
the only thing that you're doing.
I also love that answer because
I loved that you introduced
that Swiss cheese model.
That, I think that's actually what
I've gone back and referenced a few
times in the slide deck because I have
the same analogy, but I use puzzle
pieces instead when I'm talking to
teachers about assessing in the age
of AI, and this idea that if you
only have one puzzle piece, even two
puzzle pieces, that's not a puzzle.
You've gotta have at
least three puzzle pieces.
And ideally, the more puzzle pieces
you have, the harder the puzzle.
A 1,000-piece puzzle is a lot harder
than a 500-piece puzzle to do.
So it's the same idea, like layering
Swiss cheese and covering the
holes, adding more puzzle pieces.
It has to be a multi-layered approach.
And that is not just writing, by the way.
It's essential to writing
because that's what AI large
language models in particular
are best at right now is writing.
It's also, I think, what we have to do
across really all assessment is take
this really multi-layered approach.
I will link to, of course,
myshortanswer.com in the show notes.
We'll link to your Substack.
Uh, I linked to it in a couple episodes
ago, the specific article you wrote.
It was a two-part article
on, you know, edtech is worth
saving or something like that.
But we'll link to your Substack.
I haven't read the one you just put out.
I'll have to go back and read that.
I'll put your email in there as well
so people will have all of that.
Wrapping up here,
Anything else you wanna share
with listeners in terms of
how they can connect with you?
I mean my email's adam@myshortanswer
Anybody can reach out to me I'm
happy to chat I genuinely enjoyed
this conversation and could continue
having it for hours because I'm
Same.
work and I think we're in a really
important moment for education but also
educational technology specifically
And the decisions we make in the next
six to 12 months feel like they're
gonna reverberate for the next five to
10 years So it it feels like a really
important moment and I appreciate you
giving me a chance to share my voice
on this I hope that you will and I
know you do cause you're a rock star
w will continue to lift up the voices
of educators cause I think their
voices are kind of being lost in this
Yes.
There's just like this assumption of
like No one ever wanted technology and we
never we There's no Why did we ever even
adopt it to begin with I'm like Well you
should go talk to the 10,000 teachers that
I see every year at ed tech conferences
Yep.
this stuff because of the positive
impact it's having in their classroom
So please keep doing what you do and
please keep lifting up those voices
cause I think they're really important
Yeah, absolutely.
I might have to have you back on to
dig more into writing instruction,
especially in the age of AI, 'cause
that is such a hot button topic.
And you said it, but I know y- you
guys do, like, four-hour workshops
specifically focused on this.
Do you have that on the
Short Answer website?
Yeah yeah If you go to our at the top
there's like a Getting Started button
and then there'll be an option for like
professional development and workshops
and that'll give you the information
Okay.
Okay.
We will be leading it Oh man what's the
next conference We lead a a lot with
school districts but the next conference
I think we're leading it at is will be
Dakota Teachers of English Conference
Huge conference that I'm sure all your
listeners will be at Just kidding and
Where's that at, by the way?
Is that in Rapid City or Sioux Falls
No it's in Sioux Falls?
It's
Sioux Falls.
Okay, yeah, that's a long ways from me.
that's a long way for you
Yeah.
But we'll be in we'll be in Philadelphia
for the National Council of Teachers
of English Conference and we're
Okay.
the workshop It's it's gonna be the
full four-hour workshop and we're
gonna lead it with one of our schools
Kimberly Sosbe who's an awesome director
of secondary ELA at Grand Prairie ISD
in Texas her and I are gonna lead it
together if you've got English teachers
that are on the East and Seaboard or
who are members of the National Council
of Teachers of English we'll be in
Philadelphia leading it this this fall
any conference we go to we try to lead
a version of it So yeah please reach out
I'm happy to talk about that or anything
Okay.
technology efficacy short
answer writing whatever
I will put a link to the writing
workshops in the show notes as well,
if there's any teachers that are
specifically interested in that.
I wanna try to go-- I got your
50-minute version of your four hours.
I would love to, to get
the full four hours myself.
For real wrapping up, we end every episode
with a Make a Tech100 Moment, which
is like what is your final thoughts of
what you want to leave listeners with?
Oh man
We covered a lot of
ground in this episode.
so much ground If you're an educator
listening to this you probably like
educational technology because you're
you know what you do Lindy and you're
really good at it So I guess my thing
to share would be share your voice and
when I say share your voice I just mean
share your experience share how you use
technology and how it has positively
improved outcomes in your classroom in
this But I think those stories are being
lost in our current echo chamber around
how awful technology is and we're losing
sight of the really positive use cases
and affordances in education And the
most important voices we need to hear
those from are teachers like the people
that might be listening to this podcast
So I'd just say your voice share your
stories on social media on Substack with
your with your principal with your school
board with your parents like any audience
to sort of help com I don't wanna say
combat but just to help put some balm
on the the the current moment that would
be my my I would leave with I guess
100% agree.
The naysayers are loud, so we have
to be loud in response to show
the other end of the spectrum.
By the way, I loved in your
workshop the, I think you
called it radical cen-centrism.
What did you call it?
Radical centrism
so radical centrism this idea of there's
a spectrum on one end of the other.
One spectrum is, you know, it's
great and everything's great.
Like right now, AI.
AI's gonna save humanity, da, da, da, da.
And then the other end of the spectrum
is it's all terrible, it's ruining our
brains, AI's the end of humanity, right?
And your argument was be a
radical centrist in the middle.
I love that for multiple reasons.
One of the reasons is that's exactly
what I tell teachers, especially when
it comes to the AI debate right now.
I always say- You wanna be in the center.
You wanna pull the powerful parts of
the technology and leverage the powerful
parts and mitigate the bad parts.
So
Agreed
I loved... I've been using your radical
centrist terminology ever since then.
Be a radical centrist.
now but yeah Yeah
Is there?
Okay.
Yeah Yeah
Okay, awesome.
Love it.
All right.
Thank you so much, Adam.
Great conversation, and like I said, we
might have to have you back on the podcast
Anytime.
Thank you for having me
Thanks for joining Make EdTech 100.
I know educator time is valuable, and I'm
honored you choose to spend yours with me.
For more edtech strategies you can use
tomorrow and ways to bring me to your
school or event, head to lindyhock.com.
If this episode resonated, hit Subscribe
so you don't miss the next one.
I'm LindyHoc.
Go forth and Make EdTech 100