1
00:00:01,560 --> 00:00:02,310
Hello everybody.

2
00:00:02,310 --> 00:00:04,800
Welcome to another episode of APIs.

3
00:00:04,800 --> 00:00:06,840
You Won't Hate the podcast version.

4
00:00:07,090 --> 00:00:12,160
Today I am joined by Tom Hurst and
we are gonna talk all about mocking

5
00:00:12,520 --> 00:00:16,219
and not taking the piss outta people
nowhere to talk about API mocking which

6
00:00:16,219 --> 00:00:17,599
is a whole different kettle of fish.

7
00:00:18,050 --> 00:00:19,564
So, Tom, hello.

8
00:00:19,700 --> 00:00:20,209
How you doing?

9
00:00:21,015 --> 00:00:21,680
Good, thanks.

10
00:00:21,680 --> 00:00:22,099
Yeah.

11
00:00:22,220 --> 00:00:26,150
I'm really pleased to, to finally,
finally manage to get this organized

12
00:00:26,150 --> 00:00:27,680
and and get to speak to you on the show.

13
00:00:28,144 --> 00:00:29,074
Straight in with the mocking.

14
00:00:29,074 --> 00:00:29,494
Brilliant.

15
00:00:29,774 --> 00:00:33,644
Yeah, we have, we have been trying to
record this podcast for about six months,

16
00:00:33,705 --> 00:00:38,294
and I feel like you know more about
what's going on in my life than many

17
00:00:38,294 --> 00:00:42,854
of my friends, just by the wide variety
of excuses I've used for why we've had

18
00:00:42,854 --> 00:00:44,804
to cancel and reschedule this podcast.

19
00:00:45,201 --> 00:00:46,221
A never dull moment.

20
00:00:46,566 --> 00:00:47,226
Yeah.

21
00:00:47,286 --> 00:00:47,586
Yeah.

22
00:00:47,586 --> 00:00:49,236
I hope, I hope your life's going smoothly.

23
00:00:49,266 --> 00:00:50,256
What have you been up to lately?

24
00:00:50,565 --> 00:00:54,775
I mean, running a startup is a bit of,
a bit of an all consuming endeavor.

25
00:00:55,195 --> 00:00:59,685
So sadly, I I've probably got, you know,
more to say about that than anything else.

26
00:00:59,735 --> 00:01:00,785
There's a, a bit of background.

27
00:01:00,785 --> 00:01:03,845
I'm the CTO and co-founder of why I'm
not the company, which is based on

28
00:01:03,845 --> 00:01:05,075
why I mock the open source project.

29
00:01:05,075 --> 00:01:07,625
It's been around for ish years now.

30
00:01:07,625 --> 00:01:10,171
And when I'm not dealing
with childcare issues.

31
00:01:10,171 --> 00:01:13,051
That's kind of, you know, the
other thing that I'm doing.

32
00:01:13,981 --> 00:01:14,611
Yeah, brilliant.

33
00:01:15,091 --> 00:01:16,561
Running startups is a whole lot of fun.

34
00:01:16,841 --> 00:01:19,481
Have the reason for this taking so
bloody long and the fact that most of our

35
00:01:19,481 --> 00:01:23,171
listeners might be incredibly surprised to
see this podcast popping up in their feed.

36
00:01:23,201 --> 00:01:26,201
'cause we've had quite a few
long gaps between episodes is

37
00:01:26,201 --> 00:01:28,091
just trying to, trying to run.

38
00:01:28,931 --> 00:01:32,321
Trying to do, trying to work on APAs You
Won't Hate, and Green Turtle, which is

39
00:01:32,321 --> 00:01:37,201
like a software consultancy and protect
Earth, which is like Reforesting, the uk.

40
00:01:37,581 --> 00:01:40,971
It is a bit much, and all
three of them are going well.

41
00:01:40,971 --> 00:01:44,031
It's nice, but oh my God,
can I not handle that?

42
00:01:44,156 --> 00:01:45,196
startups rather than one.

43
00:01:45,501 --> 00:01:46,521
Basically, yeah.

44
00:01:46,571 --> 00:01:49,121
And, and protect Earth is nuts.

45
00:01:49,121 --> 00:01:52,511
But we have just hired a new project
manager, which I'm really excited about.

46
00:01:52,541 --> 00:01:55,961
'cause he basically is taking
over the, the day-to-day of

47
00:01:55,961 --> 00:01:58,091
like finding, you know, finding.

48
00:01:58,661 --> 00:02:01,781
Farmers to plant with and get in the
right trees in the right place and

49
00:02:02,111 --> 00:02:05,831
taking over the day-to-day operations
of that more project based stuff.

50
00:02:05,951 --> 00:02:09,281
And it leaves me to run around being
like, Hey, we should buy that chunk of

51
00:02:09,281 --> 00:02:13,211
land over there and hey, that you come
partner with us, give us some money,

52
00:02:13,331 --> 00:02:17,771
and, and all that sort of other important
business work without the whole, we need

53
00:02:17,771 --> 00:02:20,111
2000 of these trees over there by Tuesday.

54
00:02:20,141 --> 00:02:20,441
It's

55
00:02:21,221 --> 00:02:23,996
So you can be the CEO delegating
all the grunt work to someone.

56
00:02:24,041 --> 00:02:24,971
Yeah, exactly.

57
00:02:25,091 --> 00:02:25,181
It?

58
00:02:25,721 --> 00:02:30,491
Yeah, although we, we all just spend my,
my brain is slightly wrecked from spending

59
00:02:30,761 --> 00:02:34,691
Tuesday, two days ago we were doing a, a
first aid course with a forestry module,

60
00:02:34,691 --> 00:02:38,861
and it's just all of the creative ways
you can like maim and destroy yourself

61
00:02:38,861 --> 00:02:42,446
using a wood chipper or getting ticks
and Lyme disease, which I bloody out.

62
00:02:42,446 --> 00:02:44,561
That was the reason for the last delay.

63
00:02:44,561 --> 00:02:46,721
I'm like, I'm sorry I can't do a podcast.

64
00:02:46,721 --> 00:02:48,851
I've just found eight ticks
in me and I'm ill now.

65
00:02:49,391 --> 00:02:50,321
Life is madness.

66
00:02:50,321 --> 00:02:52,871
But a talking about computers is fun.

67
00:02:53,561 --> 00:02:58,391
Let's, I, I am, I am going mad,
but I do love to hear what's

68
00:02:58,391 --> 00:02:59,801
going on in the API community.

69
00:02:59,851 --> 00:03:00,481
And so

70
00:03:00,566 --> 00:03:00,786
So.

71
00:03:00,871 --> 00:03:02,731
glad to have you on
talking about wire mock.

72
00:03:02,791 --> 00:03:03,631
Wire mock is cool.

73
00:03:04,081 --> 00:03:08,221
So in the world of mocking, mocking is,
is, is a really important topic, I think.

74
00:03:08,221 --> 00:03:13,141
But I think it's one of those words that's
become so completely overloaded that no

75
00:03:13,141 --> 00:03:15,061
one's ever talking about the same thing.

76
00:03:15,331 --> 00:03:16,021
And so.

77
00:03:16,741 --> 00:03:19,441
Just wondering if you can give
us a bit of an intro to kind of

78
00:03:19,541 --> 00:03:21,371
yeah, where, where mocking fits in.

79
00:03:21,401 --> 00:03:25,541
Like mocking appears in testing, but
it also appears in like API design.

80
00:03:25,871 --> 00:03:29,111
And then some people talk about like
sandboxing and, and what's that,

81
00:03:29,111 --> 00:03:30,671
or prototyping and what's that.

82
00:03:30,881 --> 00:03:35,201
So that's quite a, a big topic to just
dive straight into, but could you give

83
00:03:35,201 --> 00:03:37,331
us a bit of a lay of the land on that?

84
00:03:37,963 --> 00:03:38,433
Yeah, sure.

85
00:03:38,483 --> 00:03:43,403
The terminology is pretty tricky and
I think I've probably played my part

86
00:03:43,403 --> 00:03:47,218
in in muddying the waters further
over the years, I'll have to admit.

87
00:03:47,218 --> 00:03:50,158
So with apologies to kind of Martin
Fowler in the, the XP community

88
00:03:50,158 --> 00:03:53,608
generally, who originally came up with
a very specific definition of mocking,

89
00:03:53,608 --> 00:03:57,478
which was a, you know, essentially a
particular type of activity within.

90
00:03:57,918 --> 00:04:00,078
Sort of the creation of
test doubles more broadly.

91
00:04:00,108 --> 00:04:02,358
I guess test doubles were the
overall category, and then mocking

92
00:04:02,358 --> 00:04:05,658
was a, a technique for doing, you
know, behavioral verification.

93
00:04:06,828 --> 00:04:09,558
I think over time the word mocking
has sort of broadened and become

94
00:04:09,558 --> 00:04:13,838
this, this this sort of umbrella
term in itself describe test

95
00:04:13,838 --> 00:04:15,798
doubles further muddying the waters.

96
00:04:15,798 --> 00:04:18,738
Originally it was applying to kind
of mocking in code, so substituting

97
00:04:18,738 --> 00:04:19,908
interfaces in code, whereas.

98
00:04:21,208 --> 00:04:24,178
in the context I'm talking about, and
in the context of wire markets, it's

99
00:04:24,178 --> 00:04:28,748
about APIs over the wire, you know, so
so, so networks APIs rather than kind

100
00:04:28,748 --> 00:04:30,428
of in, you know, in code or in memory.

101
00:04:31,148 --> 00:04:35,378
And I, I think there's a, I think
there's further, further sort of water

102
00:04:35,558 --> 00:04:38,798
muddying aspects to this as well, and
that there's an older generation of tools

103
00:04:38,798 --> 00:04:40,418
referred to as service virtualization.

104
00:04:40,818 --> 00:04:44,028
Which is obviously incredibly confusing
because when, but when you say

105
00:04:44,028 --> 00:04:47,178
virtualization, you tend to think of kind
of hypervisors and that kind of stuff.

106
00:04:48,258 --> 00:04:50,118
so there's an older generation
of tools that do that.

107
00:04:50,118 --> 00:04:52,578
And then, and that now also there's
this term that sort of entered the

108
00:04:52,578 --> 00:04:57,538
mix of API simulation, which sort
of is used by, by newer tools.

109
00:04:57,598 --> 00:05:00,598
And, you know, we use to
describe our product as well.

110
00:05:01,678 --> 00:05:02,428
I, I suppose the.

111
00:05:03,708 --> 00:05:06,798
What I've kind of settled on as my,
the, the way I describe this is a,

112
00:05:06,798 --> 00:05:11,538
is kind of a continuum where mocking
tends to be the, the closest to the

113
00:05:11,538 --> 00:05:12,798
kind of marketing failure definition.

114
00:05:12,798 --> 00:05:14,928
The stuff you do, you're
probably driving it in code.

115
00:05:15,258 --> 00:05:18,888
You are, you're using mocks as essentially
as, as sort of close test doubles in.

116
00:05:19,708 --> 00:05:21,568
Unit or unit like tests.

117
00:05:21,608 --> 00:05:27,408
And you know that it's like code driven
and bundled together, whereas, and also

118
00:05:27,408 --> 00:05:28,878
they, it is probably worth pointing out.

119
00:05:29,148 --> 00:05:30,408
They, they're, they're quite simple.

120
00:05:30,408 --> 00:05:33,298
They tend, they, they're kind
of, know, canned responses,

121
00:05:33,298 --> 00:05:35,038
canned stubs, you know, to, to,

122
00:05:35,123 --> 00:05:35,343
Hmm.

123
00:05:35,698 --> 00:05:36,868
to sort of match requests.

124
00:05:36,868 --> 00:05:39,358
Whereas at the, sort of, at the
other end of this continuum, you have

125
00:05:40,378 --> 00:05:44,518
simulation, which is probably just a,
you know, warming over of, of service

126
00:05:44,518 --> 00:05:48,218
virtualization as a concept where the
thing you've built is the simulation is

127
00:05:48,218 --> 00:05:49,868
much more sophisticated and realistic.

128
00:05:49,868 --> 00:05:50,558
So you have.

129
00:05:50,968 --> 00:05:53,918
You know you know, complex
kind of behavioral variations.

130
00:05:53,918 --> 00:05:56,828
You maybe have the ability to generate
responses dynamically rather than

131
00:05:57,128 --> 00:05:58,748
just sending static responses.

132
00:05:58,748 --> 00:06:01,818
You have the ability to store and
retrieve states pull things from

133
00:06:01,818 --> 00:06:03,108
data sources, that kind of thing.

134
00:06:03,808 --> 00:06:07,858
So, so I, I find that sort of a
use, useful, useful continuum to,

135
00:06:07,888 --> 00:06:09,028
you know, to think of these things.

136
00:06:09,028 --> 00:06:12,868
And I think there are where you sort
of choose to be along with continuum.

137
00:06:12,868 --> 00:06:15,598
Kind of depends on the, you
know, what you're trying to

138
00:06:15,598 --> 00:06:16,693
do, what your, your use case.

139
00:06:18,238 --> 00:06:19,138
Yeah, that makes sense.

140
00:06:19,138 --> 00:06:22,848
I mean, yeah, the the way, the
way that I tried to talk about it.

141
00:06:23,298 --> 00:06:27,918
When it was working on Prism at stoplight,
what, which is another mocking tool that's

142
00:06:28,008 --> 00:06:32,088
no longer really being actively developed,
I don't think the, the, the terminology

143
00:06:32,088 --> 00:06:36,258
we'd used was like, yeah, mocking is
kind of a simple, you make a request and

144
00:06:36,258 --> 00:06:37,818
you're gonna get a response of something.

145
00:06:37,818 --> 00:06:41,208
It may or may not be useful, and there
are ways to like increase the usefulness

146
00:06:41,238 --> 00:06:43,068
and relevance of the generated thing.

147
00:06:43,398 --> 00:06:46,968
I think we, we had like static and dynamic
responses, so static would just be.

148
00:06:47,778 --> 00:06:51,498
This mock is based off the open API
it said there's gonna be a name,

149
00:06:51,498 --> 00:06:54,648
which is a string, and there's gonna
be an email, which is format email.

150
00:06:54,648 --> 00:06:56,838
And looking at that, it
can generate a random name.

151
00:06:56,868 --> 00:06:57,198
Oh no.

152
00:06:57,198 --> 00:07:00,528
The static would just be like,
whatever the example was in the schema.

153
00:07:00,778 --> 00:07:04,828
And the dynamic would be, alright, let's
try and generate a random name using faker

154
00:07:04,828 --> 00:07:06,688
and generate a random email using faker.

155
00:07:06,688 --> 00:07:09,388
So that was at least different,
not the same thing every time.

156
00:07:09,728 --> 00:07:11,678
But there was no like logic there.

157
00:07:12,053 --> 00:07:13,943
And, and, and no persistence there.

158
00:07:13,943 --> 00:07:17,843
And so I think that for me, the
definition of mocking was kind of a

159
00:07:17,843 --> 00:07:20,153
bit dim without mean to offend anyone.

160
00:07:20,153 --> 00:07:21,473
That was the way I, I classed it.

161
00:07:21,563 --> 00:07:26,243
And then, and then sandboxing was like,
oh, and we can, we can put a database,

162
00:07:26,243 --> 00:07:29,483
spin that up in the background so when
you save stuff, we'll put it out and

163
00:07:29,483 --> 00:07:33,323
we'll try and match up the fields if
possible and, and return that back to you.

164
00:07:33,503 --> 00:07:36,313
But that was really blooming hard
when people would do things like, they

165
00:07:36,313 --> 00:07:40,753
would, you know, send data as a raw
JSON object with name and email, and

166
00:07:40,753 --> 00:07:44,173
then they'd return it wrapped in like
a data envelope and then it wouldn't

167
00:07:44,173 --> 00:07:47,953
know, oh, do you have to like do a lot
of weird stuff to try and figure out.

168
00:07:48,253 --> 00:07:51,163
Matching request to responses, even
with a database in the background.

169
00:07:51,163 --> 00:07:52,903
So that was always a bit weird.

170
00:07:53,203 --> 00:07:57,793
And then kind of prototyping was
another term that was like, you

171
00:07:57,793 --> 00:08:01,123
are just building an application
before you build your application.

172
00:08:01,123 --> 00:08:05,203
And you might build a, a, a rubbishly
little application, just fire out

173
00:08:05,203 --> 00:08:09,193
in, in no JS or something with the,
the grand scheme of it being done

174
00:08:09,193 --> 00:08:11,113
in in Java or something later on.

175
00:08:11,113 --> 00:08:14,143
You might just work on something really,
really quickly and then make it later.

176
00:08:14,143 --> 00:08:17,023
But that was always a big
faf 'cause people would just.

177
00:08:17,288 --> 00:08:20,648
Start writing enough code that they'd get
attached and then they just wanna put that

178
00:08:20,648 --> 00:08:24,188
to production, even though they skipped
infinite, there's like massive gaping

179
00:08:24,188 --> 00:08:28,038
holes everywhere and you can just bend
the roof, the future, like, you know, just

180
00:08:28,043 --> 00:08:32,393
was it mass assignment issues all over
the place and all that sort of rubbish.

181
00:08:32,393 --> 00:08:36,713
So it's always been really interesting
trying to make any tools in this

182
00:08:36,713 --> 00:08:40,943
space because whatever you make,
somebody will want the other stuff.

183
00:08:41,333 --> 00:08:43,643
And there are so many different
verticals that this mocking.

184
00:08:44,578 --> 00:08:47,218
This mocking functionality could be
used in so many different parts of

185
00:08:47,218 --> 00:08:49,288
the lifecycle, like with wire mock.

186
00:08:49,348 --> 00:08:52,498
Where, where do you feel like your
tooling, where, where does your

187
00:08:52,498 --> 00:08:58,378
tooling aim to be in terms of like
stage of the lifecycle and, and what

188
00:08:58,378 --> 00:08:59,758
sort of verticals are you going after?

189
00:08:59,758 --> 00:08:59,788
I.

190
00:09:01,273 --> 00:09:05,653
So, so the, the genesis of the
tool is definitely in the of.

191
00:09:06,913 --> 00:09:09,913
You know, the, the mocking for in
the, in the developer's toolkit.

192
00:09:09,913 --> 00:09:14,323
So, so, so the mocking to support,
kind of test driven development,

193
00:09:14,323 --> 00:09:16,963
you know, acceptance testing, test
automation, that kind of thing.

194
00:09:17,458 --> 00:09:17,678
Hmm.

195
00:09:18,523 --> 00:09:23,783
and, you know, so it, it fits in
that kind of you developer in a loop.

196
00:09:24,168 --> 00:09:26,598
You know, they, they, I mean, the open
source tool is very much that sort

197
00:09:26,598 --> 00:09:29,568
of developer in a loop where you are,
you are building functionality and

198
00:09:30,228 --> 00:09:33,058
you are, you are, you know, you are
supporting the construction of your

199
00:09:33,058 --> 00:09:35,008
code with, you know, with mock APIs.

200
00:09:37,003 --> 00:09:40,693
The, you know, our commercial
products is, is kind of deliberately

201
00:09:40,773 --> 00:09:42,273
about broadening that base.

202
00:09:42,273 --> 00:09:46,953
And so it's in, one sense, it's
broadening it in, in terms of the,

203
00:09:47,103 --> 00:09:49,803
you know, an individual organization
software development lifecycle.

204
00:09:49,803 --> 00:09:52,653
So where you want to start
deploying mocks into.

205
00:09:53,398 --> 00:09:56,098
These kind of mixed, integrated
environments, you know, where you want to

206
00:09:56,098 --> 00:10:00,628
be able to do of isolate sets of related
services and test them, or where you

207
00:10:00,628 --> 00:10:04,258
want to be able to kind of deploy, say
a microservice into a real environment.

208
00:10:04,258 --> 00:10:07,528
But you need to, to mock most or all
of its dependencies to, you know, to

209
00:10:07,528 --> 00:10:09,118
be able to, to isolate it and test it

210
00:10:09,473 --> 00:10:09,593
Hmm.

211
00:10:10,038 --> 00:10:13,698
When you, you know, particularly, you
know, I mean, microservice environments

212
00:10:13,698 --> 00:10:14,868
are probably the best example of this.

213
00:10:14,868 --> 00:10:15,618
You know, places where.

214
00:10:15,998 --> 00:10:19,688
You have lots of different teams who are
all building APIs and they're all talking

215
00:10:19,688 --> 00:10:23,548
to each other's APIs and everything's
changing you know, rapidly and somewhat

216
00:10:23,548 --> 00:10:28,978
unpredictably and, and you know, we're
sort of about kind of trying to help

217
00:10:28,978 --> 00:10:30,928
tame some of that, that kind of chaos.

218
00:10:31,518 --> 00:10:34,218
But we do, but we do what we
are also sort of playing in the,

219
00:10:34,423 --> 00:10:35,658
I, I suppose at the top level.

220
00:10:35,658 --> 00:10:38,028
I kind of think of that being,
you know, sort of two, two

221
00:10:38,028 --> 00:10:40,218
kind of top level, I guess.

222
00:10:40,518 --> 00:10:44,598
Types of use case with sort of maybe
even different sort of buyers or

223
00:10:44,598 --> 00:10:47,778
different sets of personas, different
user personas associated with them.

224
00:10:47,778 --> 00:10:49,818
And I think one of them is
the one I just described.

225
00:10:49,818 --> 00:10:52,288
So developers and, and test engineers.

226
00:10:52,738 --> 00:10:56,718
And then the other one is, is kind
of I suppose API people probably

227
00:10:56,718 --> 00:10:59,878
more, you know, the, the maybe
main listeners of this podcast.

228
00:10:59,968 --> 00:11:04,168
So people who are building APIs
for consumption by the public, by

229
00:11:04,228 --> 00:11:09,178
customers, you know, who are you know,
thinking about how to, to make APIs

230
00:11:09,208 --> 00:11:11,068
you know, accessible, well designed.

231
00:11:11,408 --> 00:11:14,198
You know, where their APIs a kind of
competitive asset for their company.

232
00:11:14,573 --> 00:11:14,793
Mm.

233
00:11:14,848 --> 00:11:16,648
And I, and I think that, you
know, mocking gets used in

234
00:11:16,648 --> 00:11:18,088
both of those, those contexts.

235
00:11:18,088 --> 00:11:22,158
But what I've observed so far
actually is that the, the, the tools

236
00:11:22,158 --> 00:11:23,388
get built for one or the other.

237
00:11:23,508 --> 00:11:24,168
And I think,

238
00:11:24,168 --> 00:11:24,618
Right.

239
00:11:24,708 --> 00:11:25,038
Prism.

240
00:11:25,038 --> 00:11:29,248
Prism earlier, I think is an example of a
tool that was optimized for the, the sort

241
00:11:29,248 --> 00:11:32,068
of API or you know, API building audience.

242
00:11:32,158 --> 00:11:32,723
Yeah, for sure.

243
00:11:33,268 --> 00:11:35,248
and I, I think Postman
probably falls into that.

244
00:11:35,308 --> 00:11:38,008
Yeah, postman's mocking functionality
falls into that category as well,

245
00:11:38,153 --> 00:11:38,443
Yeah.

246
00:11:38,578 --> 00:11:42,628
it's about supporting the design
process and supporting the supporting

247
00:11:42,748 --> 00:11:46,948
sort of production of onboarding
materials and documentation for,

248
00:11:46,948 --> 00:11:49,888
for end users and making it easier
for them to, to discover an API.

249
00:11:50,878 --> 00:11:53,578
But like I say, then there's this
whole other sort of category of users

250
00:11:53,578 --> 00:11:57,238
who are doing, who are trying to do
serious kind of qa, you know, dev

251
00:11:57,238 --> 00:11:59,068
and QA activities with, with mocks.

252
00:12:00,113 --> 00:12:02,978
And, and I think, yeah, this, these
two sort of parallel kind of families

253
00:12:02,978 --> 00:12:06,188
of products I think have, have grown
up around those, those two, you know,

254
00:12:06,218 --> 00:12:07,838
quite different sets of use cases.

255
00:12:08,168 --> 00:12:11,498
And I think we're, you know, we're,
we, we're, we're trying to of, you

256
00:12:11,498 --> 00:12:14,868
know, plain in, in both of these,
you know, we're trying to ensure

257
00:12:14,868 --> 00:12:18,298
that they, the, the product gives
you the, the affordances to, yeah.

258
00:12:18,303 --> 00:12:21,868
To, to, to be able to use mocking
to support API documentation to

259
00:12:21,868 --> 00:12:27,018
support the design process, but
also be then deployable into a,

260
00:12:27,018 --> 00:12:28,728
a really demanding QA process.

261
00:12:29,688 --> 00:12:32,088
That's, it's really hard to,
to walk that line, isn't it?

262
00:12:32,118 --> 00:12:35,868
'cause yeah, certainly when working on
Prism I, I was around for the rewrite

263
00:12:35,868 --> 00:12:42,438
from it, turning from a, a go library
into kind of a a, a. Type script based

264
00:12:42,438 --> 00:12:44,538
CLI, and it just, it did everything.

265
00:12:44,538 --> 00:12:46,758
It did it tried to do every single
verse call you could imagine.

266
00:12:46,758 --> 00:12:48,138
And a lot of mock tools get into this.

267
00:12:48,138 --> 00:12:52,758
They, they're like, because they
understand the schema of what's

268
00:12:52,758 --> 00:12:56,388
allowable and they are trying
to cover a few like instances.

269
00:12:56,438 --> 00:12:58,508
And they also have like
traffic coming through it.

270
00:12:58,688 --> 00:13:00,968
You get this like mocking brilliant.

271
00:13:00,968 --> 00:13:01,448
Let's do that.

272
00:13:01,448 --> 00:13:03,428
We've got the, we've
got the open API, let's.

273
00:13:03,448 --> 00:13:08,148
Fire out some some potential endpoints and
requests and responses, but then actual

274
00:13:08,148 --> 00:13:11,028
traffic starts to come through and you
can get into like validation of that.

275
00:13:11,028 --> 00:13:13,848
Like, Hey, are the requests coming
through to this matching the open API?

276
00:13:14,148 --> 00:13:17,148
And so then you, then you actually
get like validation responses,

277
00:13:17,148 --> 00:13:18,408
but then you go, Hey, what?

278
00:13:18,408 --> 00:13:20,838
We can have a proxy where you
pass real traffic through.

279
00:13:21,078 --> 00:13:24,053
And then we could start to say,
well do those, match the, the

280
00:13:24,058 --> 00:13:25,998
requests and responses in open API.

281
00:13:26,248 --> 00:13:26,908
And then.

282
00:13:27,003 --> 00:13:27,163
then.

283
00:13:27,318 --> 00:13:30,588
You can use that for QA and you can
use that for maybe in production,

284
00:13:30,588 --> 00:13:34,618
if you don't mind having the, you
know, the actual critical path.

285
00:13:34,618 --> 00:13:37,888
Go through some random library that may
or may not be particularly performant.

286
00:13:38,248 --> 00:13:41,368
And then you start going into like
learning of well actually there's not,

287
00:13:41,668 --> 00:13:45,058
there's not an implementation here that
we're proxying to, but people are sending

288
00:13:45,058 --> 00:13:48,998
lots of traffic through this and so
let's, let's make an API out of that.

289
00:13:48,998 --> 00:13:52,388
Or maybe there is an implementation that
doesn't have any open API, so let's.

290
00:13:53,123 --> 00:13:53,843
Make funny.

291
00:13:53,843 --> 00:13:57,563
You open the traffic and it starts to
become so confusing that whenever you

292
00:13:57,563 --> 00:14:01,913
have a tool that can do anything, no
one knows what the hell to do with it.

293
00:14:01,973 --> 00:14:04,343
Like, you've got that read me
and you look at it and go, blah,

294
00:14:06,048 --> 00:14:07,433
what, what is all this Then?

295
00:14:07,733 --> 00:14:10,343
And so that was something I did with
Prism was just like, delete a lot of

296
00:14:10,343 --> 00:14:14,013
that functionality from the roadmap with
the intention of bringing it back later.

297
00:14:14,228 --> 00:14:14,858
Simpler.

298
00:14:15,138 --> 00:14:18,618
But it just becomes really complicated
to try and deal with that many different

299
00:14:18,618 --> 00:14:20,508
personas that many different use cases.

300
00:14:20,688 --> 00:14:25,488
Well, and so really open-ended
questions here, but just like,

301
00:14:25,608 --> 00:14:26,658
how do you approach that?

302
00:14:26,688 --> 00:14:30,408
'cause you've got, you've got open
A-P-I-G-R-P-C, all this stuff, and

303
00:14:30,408 --> 00:14:33,288
you've got like testing and, and,
and using it in the design process.

304
00:14:33,288 --> 00:14:36,978
Like, how the heck do you manage
any sort of roadmap and balance of

305
00:14:36,978 --> 00:14:38,298
personas and functionality there?

306
00:14:39,138 --> 00:14:40,458
Yeah, it's, it's difficult.

307
00:14:40,458 --> 00:14:42,258
I mean, we, we experience
all of those problems.

308
00:14:42,258 --> 00:14:44,358
I mean, I, I've, I've been
working on y mark open source

309
00:14:44,358 --> 00:14:46,068
for 14 years now, more or less.

310
00:14:46,428 --> 00:14:49,528
And you know, so there's, there's
a lot in there and, and there's

311
00:14:49,528 --> 00:14:51,898
a lot in there for sort of, lots
of, lots of different people.

312
00:14:51,898 --> 00:14:56,728
And it, and I, I think I probably only
recently have start really seriously

313
00:14:56,728 --> 00:15:00,838
started thinking about, kind of
segmenting the audience up into, you

314
00:15:00,838 --> 00:15:05,158
know, in, into you know, different
user needs and and, and trying to

315
00:15:05,158 --> 00:15:06,718
work out how to focus on those.

316
00:15:06,788 --> 00:15:08,528
But it's, it's a tough balance.

317
00:15:08,528 --> 00:15:11,708
I mean, when, you know, when you're
selling a product in a way that's, that

318
00:15:11,708 --> 00:15:14,658
can be a good thing because it gives
you these, these signals about you know,

319
00:15:14,658 --> 00:15:16,848
people will tell you what they want to
buy and what they don't want to buy.

320
00:15:16,848 --> 00:15:21,408
So you can kind of use, use that as
a, as a sort of as an easy filtering

321
00:15:21,673 --> 00:15:22,723
Just turn that off.

322
00:15:22,723 --> 00:15:23,428
No one's using that.

323
00:15:23,438 --> 00:15:24,148
Don't worry about that.

324
00:15:25,228 --> 00:15:26,368
Yeah, yeah, yeah.

325
00:15:26,708 --> 00:15:29,648
Or likewise, you, you know, there'll
be one thing where you do some, you

326
00:15:29,648 --> 00:15:33,938
know, almost you, you chuck in some sort
of sort of token wave towards it, and

327
00:15:33,938 --> 00:15:35,048
then everyone goes, oh, that's great.

328
00:15:35,048 --> 00:15:35,798
We want more of that.

329
00:15:35,798 --> 00:15:38,708
And suddenly you find that you've pivoted
all your engineering effort to kind of

330
00:15:38,708 --> 00:15:42,678
making this thing really you know, the,
the, the kind of a central feature.

331
00:15:42,708 --> 00:15:47,088
So yeah, having, having customers that
are willing to talk to you is kind of a,

332
00:15:47,088 --> 00:15:48,798
is a really, really useful thing there.

333
00:15:49,138 --> 00:15:52,198
The funny thing about doing open
source is that people although you

334
00:15:52,198 --> 00:15:55,048
get, you know, if you have a healthy
project, you have contributors

335
00:15:55,098 --> 00:15:56,448
coming along, contributing things.

336
00:15:56,448 --> 00:16:00,888
It's sort of, very hard to, you
know, there's a lot of noise in the,

337
00:16:00,888 --> 00:16:03,738
in the, the signal in terms of what
people really need, you know, because

338
00:16:03,738 --> 00:16:05,208
people will show up often with really.

339
00:16:06,333 --> 00:16:07,563
obscure things.

340
00:16:07,623 --> 00:16:10,443
And, but push really hard
to, to get it into the code

341
00:16:10,503 --> 00:16:10,533
Mm,

342
00:16:11,343 --> 00:16:14,733
and you don't really know whether,
you know, is this a, is this a strong

343
00:16:14,733 --> 00:16:17,553
signal that this is something people
really need or is this just kind

344
00:16:17,553 --> 00:16:20,583
of one person kind of really making
a, making a big deal out of it?

345
00:16:21,723 --> 00:16:22,503
I can't believe.

346
00:16:22,533 --> 00:16:25,353
you get people kind of giving thumbs
up on, on GitHub and so on like that.

347
00:16:25,353 --> 00:16:26,973
So I guess it's, you know,
there's, there's some degree

348
00:16:26,973 --> 00:16:27,933
of signal in those things.

349
00:16:27,933 --> 00:16:30,813
But I, I, I certainly, I, I
found running the commercial

350
00:16:30,813 --> 00:16:31,983
products has been a lot more.

351
00:16:32,313 --> 00:16:35,103
Informative in terms of getting
you know, understanding what people

352
00:16:35,103 --> 00:16:38,433
really need it for than, than
running the open source community

353
00:16:38,433 --> 00:16:39,843
for years and years was, I guess,

354
00:16:40,368 --> 00:16:41,358
Yeah, that's interesting.

355
00:16:41,358 --> 00:16:41,568
Yeah.

356
00:16:41,568 --> 00:16:44,178
'cause I mean, yeah, the number of
times I've heard like, oh, I can't

357
00:16:44,178 --> 00:16:46,758
believe you haven't got random feature
you've never heard of in there.

358
00:16:46,758 --> 00:16:49,218
And me and these other three thumbs
agree that it should all work.

359
00:16:49,218 --> 00:16:51,648
But we all work at the same company
and we're the only people that care.

360
00:16:52,068 --> 00:16:53,718
It can be quite hard
to dedicate your time.

361
00:16:53,808 --> 00:16:54,368
risk, isn't it?

362
00:16:54,513 --> 00:16:57,183
Is it they they'll go and get all their
mates to go and stick a tick on it and you

363
00:16:57,183 --> 00:16:58,623
go, oh, this is really popular, isn't it?

364
00:16:58,623 --> 00:16:59,103
And then.

365
00:16:59,558 --> 00:17:01,448
Yeah, it's, it's it's hard to know really.

366
00:17:01,448 --> 00:17:03,878
It's the, you know, the, the,
the perils of small numbers.

367
00:17:04,208 --> 00:17:05,378
Yeah, that's funny.

368
00:17:05,648 --> 00:17:08,248
Long pause, which I'll
hopefully remember to edit out.

369
00:17:08,298 --> 00:17:10,848
'Cause I had a really good question
and a really great segue and then I

370
00:17:11,058 --> 00:17:12,918
focused on doing a stupid bit instead.

371
00:17:13,698 --> 00:17:14,598
Are we talking about here?

372
00:17:14,838 --> 00:17:20,328
So with open API with open
API it's like that's the main

373
00:17:20,328 --> 00:17:21,618
focus for me a lot of the time.

374
00:17:21,618 --> 00:17:24,348
And so I'm super excited
about how you handle that.

375
00:17:24,348 --> 00:17:25,098
'cause I feel like.

376
00:17:25,173 --> 00:17:25,293
I.

377
00:17:25,413 --> 00:17:27,873
Talking about those different personas
where you've got kind of the API

378
00:17:27,873 --> 00:17:32,863
design part and the testing part that
really affects the expectations of

379
00:17:32,863 --> 00:17:34,873
what open API is and how it's used.

380
00:17:35,193 --> 00:17:36,093
And so.

381
00:17:36,783 --> 00:17:40,503
When you are trying to do basic mocking,
and we talked about the ability to

382
00:17:40,503 --> 00:17:44,673
get kind of validation and, and other
behavior and stuff into there, I've

383
00:17:44,673 --> 00:17:49,023
seen quite a lot of people requesting
the functionality or suggesting the

384
00:17:49,023 --> 00:17:52,443
functionality or working on building
the functionality into their tools.

385
00:17:52,723 --> 00:17:56,413
Where the open API document
starts to get all of these.

386
00:17:56,633 --> 00:18:02,263
X extensions where they kind of litter the
open API with starting off with littering

387
00:18:02,263 --> 00:18:07,413
it with like Xen faker to let it know
that this should be a randomly generated

388
00:18:07,413 --> 00:18:09,873
IP address that seems pretty harmless.

389
00:18:09,923 --> 00:18:13,343
But then it starts getting into when
these three fields come in, they

390
00:18:13,343 --> 00:18:16,763
should be multiplied together and
a tax rate should be put on top.

391
00:18:16,763 --> 00:18:19,698
Depending on the country and like, and
all this really complicated behavior

392
00:18:19,698 --> 00:18:23,418
that you can't, you can't really infer
just from looking at a standard open

393
00:18:23,418 --> 00:18:24,883
API, it means someone's gotta start.

394
00:18:25,303 --> 00:18:26,383
Building that somewhere.

395
00:18:26,473 --> 00:18:31,393
And either you start wedging all of
that into the open APIs extensions,

396
00:18:31,393 --> 00:18:37,813
or you start littering the GitHub
repo with like random JavaScript

397
00:18:38,113 --> 00:18:39,883
files that, that handle some of it.

398
00:18:39,943 --> 00:18:41,293
And you kind of reference those somehow.

399
00:18:41,593 --> 00:18:43,483
And whatever it is you end up
doing, there's the assumption

400
00:18:43,483 --> 00:18:45,073
that you can, and, and I'll.

401
00:18:45,283 --> 00:18:48,763
Interested in changing the open
API and obviously you support

402
00:18:48,793 --> 00:18:49,993
GRPC and everything else.

403
00:18:49,993 --> 00:18:51,433
My experience is more open api.

404
00:18:51,433 --> 00:18:57,673
So I ask like how, how do you
handle mocking behavior in wire mock

405
00:18:57,673 --> 00:19:00,073
without just literal an open api?

406
00:19:00,073 --> 00:19:00,518
Or is that what you do?

407
00:19:02,053 --> 00:19:05,013
So so yeah, so we do support a
little bit of the, the littering

408
00:19:05,013 --> 00:19:07,833
through vendor extensions, but it's,
it's not something we've kind of

409
00:19:07,893 --> 00:19:09,543
leaned into particularly heavily.

410
00:19:09,753 --> 00:19:12,273
I mean, yeah, this is a, this
is genuinely a hard problem.

411
00:19:12,273 --> 00:19:14,733
This is something I, I, I end
up talking about quite a lot.

412
00:19:14,833 --> 00:19:18,403
The, you know, open API, you know,
in its, you know, unextended form

413
00:19:18,403 --> 00:19:23,193
anyway, is, is describing you know,
a, a certain aspects of an API.

414
00:19:23,193 --> 00:19:26,913
It's not a complete description of
an API's you know, constraints or

415
00:19:26,943 --> 00:19:28,473
business rules or, you know what I mean?

416
00:19:28,473 --> 00:19:32,663
There are all sorts dimensions about a,
a way an a p will will operate that that

417
00:19:32,663 --> 00:19:38,863
you, you can't capture with, with open
a necessarily when you, when you take.

418
00:19:41,353 --> 00:19:44,443
The fidelity of the mock is gonna
be limited by the you know, what

419
00:19:44,443 --> 00:19:47,233
can be expressed in, in open
API in the, in the first place.

420
00:19:47,503 --> 00:19:47,723
Hmm.

421
00:19:47,773 --> 00:19:50,023
actually this kind of relates back
to this use case, this thing in the

422
00:19:50,023 --> 00:19:53,053
first place because I think there,
there are a few tools out there that,

423
00:19:53,083 --> 00:19:57,763
that make it very easy to get from
open API to sort of something re, you

424
00:19:57,763 --> 00:19:59,653
know, a credible mock of some kind.

425
00:20:00,153 --> 00:20:02,863
But for the and like I say, if you're,
if you're doing this sort of bringing

426
00:20:02,863 --> 00:20:05,833
documentation to life thing, then,
then that's probably all you need.

427
00:20:05,833 --> 00:20:06,673
And that's great, you know?

428
00:20:06,688 --> 00:20:08,608
You just wanna have a
little for, for that?

429
00:20:08,608 --> 00:20:09,748
Bringing documentation to life.

430
00:20:09,748 --> 00:20:13,978
I feel like the use case is just, we've
got documentation, which is a list of

431
00:20:13,978 --> 00:20:17,938
endpoints with some sample requests,
some sample responses, and if you click

432
00:20:17,938 --> 00:20:22,048
the try it button, then you can see the
sample response in a different format.

433
00:20:22,288 --> 00:20:26,578
And you could even potentially copy
this URL or copy this curl command and

434
00:20:26,578 --> 00:20:29,368
do it in your terminal so that instead
of looking at it through the docs,

435
00:20:29,368 --> 00:20:30,628
you're looking at it in your favorite.

436
00:20:30,828 --> 00:20:33,408
Editor, but that's
about as far as it goes.

437
00:20:33,408 --> 00:20:33,558
Right?

438
00:20:33,558 --> 00:20:37,158
That's just a like, poke, poke
this and get something back.

439
00:20:37,158 --> 00:20:38,533
And that's, that's it.

440
00:20:38,623 --> 00:20:39,043
exactly.

441
00:20:39,043 --> 00:20:39,493
Exactly.

442
00:20:39,493 --> 00:20:43,063
And the something you get back is,
is, is, is is very limited, you know?

443
00:20:43,063 --> 00:20:45,793
So yeah, know there are all sorts of
fa you know, fairly trivial things that

444
00:20:45,793 --> 00:20:47,173
you can't really express very well.

445
00:20:47,173 --> 00:20:48,793
It's like, yeah.

446
00:20:48,793 --> 00:20:51,463
Lots of APIs will say, you know,
if you give one, one combination of

447
00:20:51,463 --> 00:20:54,703
request parameters, then you'll get
a, a response of one shake back.

448
00:20:54,703 --> 00:20:56,983
Whereas if you, if you give a different
combination, you'll, you'll get a

449
00:20:56,983 --> 00:20:58,033
sort of slightly different response

450
00:20:58,158 --> 00:20:58,218
the.

451
00:20:58,333 --> 00:21:00,763
and you, you can't describe
the relationship between that

452
00:21:00,763 --> 00:21:03,433
pattern of parameters and,
and that type of response in.

453
00:21:03,773 --> 00:21:06,623
In open API for instance, you
know, you can't express, you

454
00:21:06,623 --> 00:21:08,093
know, even trivial business rules.

455
00:21:08,093 --> 00:21:11,568
I mean, the, the, the, the demo I give
actually a lot is, is the the, the train

456
00:21:11,568 --> 00:21:14,053
travel API that the, the open API that you

457
00:21:14,278 --> 00:21:14,568
Yeah.

458
00:21:14,743 --> 00:21:15,043
I think.

459
00:21:16,043 --> 00:21:19,213
There in the so you can, you
can load that into wire mock

460
00:21:19,363 --> 00:21:21,163
and it will generate a mock API.

461
00:21:21,163 --> 00:21:23,203
And then we've got a little
demo app that's like a, yeah.

462
00:21:23,233 --> 00:21:26,773
Simple train booking app that uses
it and it, and it kind of works,

463
00:21:26,773 --> 00:21:29,503
but you can see very quickly that,
for instance, you can put in.

464
00:21:29,863 --> 00:21:32,323
A search date that's in the
past and it will happily give

465
00:21:32,323 --> 00:21:33,433
you back results, whereas

466
00:21:33,958 --> 00:21:34,438
Right.

467
00:21:34,543 --> 00:21:35,623
do that in a real system.

468
00:21:35,623 --> 00:21:37,783
So you, you want to be able to
kind of express this rule that this

469
00:21:37,783 --> 00:21:39,073
date has got to be in the future,

470
00:21:39,423 --> 00:21:39,543
Hmm.

471
00:21:39,733 --> 00:21:41,353
but that's not something
you can say in open API.

472
00:21:42,613 --> 00:21:46,973
so, the, you know, the approach that
we take and it's, it's a, a, it's an

473
00:21:46,973 --> 00:21:50,283
imperfect one, but it's one that you
know, one that works better than, than

474
00:21:50,283 --> 00:21:55,193
just doing the sort of vanilla, you
know, one way translation is to, to kind

475
00:21:55,193 --> 00:21:59,063
of say that you, the Open API and the
mock are, are both partial descriptions

476
00:21:59,063 --> 00:22:02,033
of the API and they both describe
sort of slightly different aspects.

477
00:22:02,033 --> 00:22:05,043
And so between them, you know,
you get more than either one of

478
00:22:05,043 --> 00:22:06,033
them can provide on their own.

479
00:22:06,033 --> 00:22:06,633
So we.

480
00:22:06,923 --> 00:22:08,573
We maintain both side by side.

481
00:22:08,913 --> 00:22:10,503
You can generate in both directions.

482
00:22:10,553 --> 00:22:13,643
So you can as I've already said,
you can, you can upload an, an open

483
00:22:13,643 --> 00:22:17,033
API or you can, you can, you know,
incrementally edit one and we'll, we'll

484
00:22:17,033 --> 00:22:19,103
generate and amend your stubs for you.

485
00:22:19,523 --> 00:22:22,343
You can also go the other way so you can,
I mean, and this is probably another,

486
00:22:22,553 --> 00:22:26,693
a whole other conversation, but you can
prototype by kind of usings as examples

487
00:22:26,693 --> 00:22:28,388
and generating the open API from those.

488
00:22:28,823 --> 00:22:29,223
Oh, cool.

489
00:22:29,268 --> 00:22:29,718
Yeah.

490
00:22:30,318 --> 00:22:33,023
But the, you know, the important thing
is a common pattern, and this is you,

491
00:22:33,023 --> 00:22:36,458
this is the one I demonstrate sometimes
with the train travel app, is to

492
00:22:36,608 --> 00:22:39,638
think of the, the o the thing you've,
you've built from an open API point

493
00:22:39,668 --> 00:22:42,138
as import as a, as a starting point.

494
00:22:42,528 --> 00:22:45,858
And then you're kind of layering on,
you know, the, the sort of behavioral

495
00:22:45,958 --> 00:22:49,168
richness, you know, to the, to the
extent that you need for your use case.

496
00:22:49,228 --> 00:22:52,678
And you know, it might be that you
just want to, you know, just for your.

497
00:22:53,323 --> 00:22:56,323
Your API portal users, you want to be
able to give them several variations

498
00:22:56,323 --> 00:22:57,343
of requests that they can make.

499
00:22:57,343 --> 00:23:00,533
You know, you want the you know, the
paginated, non paginated versions,

500
00:23:00,533 --> 00:23:02,163
you know, maybe you want the yeah.

501
00:23:02,163 --> 00:23:04,893
To be able to retrieve an invoice
in several different states, you

502
00:23:04,893 --> 00:23:07,143
know, with, with you know, different
combinations of data rather than

503
00:23:07,143 --> 00:23:09,723
just the, the default one, you
know, all of these kind of things.

504
00:23:09,723 --> 00:23:13,683
So, so you might wanna just do that and
layer on a little bit of richness in a way

505
00:23:13,683 --> 00:23:15,873
that's tricky to do, purely from open API.

506
00:23:16,718 --> 00:23:20,848
Or you might want to go quite a lot
further and, and model you know unhappy

507
00:23:20,848 --> 00:23:25,318
cases, false stateful, all this kind of
thing in the way that you need, if you're

508
00:23:25,318 --> 00:23:29,178
doing sort of serious QA activities, and
particularly if you're using it in a like

509
00:23:29,178 --> 00:23:33,258
I say, the, the, the, what I, what I've
tended to find is that the, the sort of

510
00:23:33,258 --> 00:23:34,908
broader the test that you're running.

511
00:23:35,208 --> 00:23:38,568
So the more the, the more of the
system that it, it encompasses.

512
00:23:39,793 --> 00:23:42,588
and thus, you know, the sort
of , the broader the scope of a, of

513
00:23:42,588 --> 00:23:45,828
an individual test case, the more
realistic the mocks need to be.

514
00:23:46,173 --> 00:23:47,088
Yeah, I gotcha.

515
00:23:47,118 --> 00:23:47,988
you end up being down.

516
00:23:48,018 --> 00:23:48,258
Yeah.

517
00:23:48,258 --> 00:23:51,198
If you've got a, if you've got say,
you know, collection five microservices

518
00:23:51,198 --> 00:23:54,858
that are collaborating, and you kind
of built an outer shell of mocks around

519
00:23:54,858 --> 00:23:57,858
that so that you are, you know, you, you
don't have to spin up your, your other

520
00:23:57,858 --> 00:23:59,928
500 microservices just to make it work.

521
00:24:00,063 --> 00:24:00,353
Yeah.

522
00:24:00,358 --> 00:24:00,738
Yeah.

523
00:24:00,858 --> 00:24:03,393
Gotta wait for Kubernetes to load
just so I can test this one endpoint.

524
00:24:03,893 --> 00:24:06,023
And you, I mean, it's amazing
the extent to which you see

525
00:24:06,023 --> 00:24:07,493
organizations trying to do that.

526
00:24:07,628 --> 00:24:08,018
yeah.

527
00:24:08,108 --> 00:24:08,708
Being

528
00:24:08,753 --> 00:24:09,593
do you know, coming up with

529
00:24:09,638 --> 00:24:09,728
there,

530
00:24:10,193 --> 00:24:13,803
really like co complex kind of routing
systems that, that let you combine.

531
00:24:13,893 --> 00:24:13,923
I

532
00:24:14,033 --> 00:24:14,153
I.

533
00:24:14,193 --> 00:24:17,493
these are sort of remote systems
that have, you know, combined local

534
00:24:17,493 --> 00:24:19,663
and and remote services for testing.

535
00:24:19,693 --> 00:24:23,643
And I'm sure some people make it work
very well, but it strikes me as being you

536
00:24:23,643 --> 00:24:25,233
know, being a sort of maddening endeavor.

537
00:24:25,383 --> 00:24:27,633
But anyway, , the point I was
trying to make is that if you're,

538
00:24:27,633 --> 00:24:28,593
if you're doing that, then.

539
00:24:29,168 --> 00:24:32,948
You, you know, you might sort of be
testing a single operation from sort

540
00:24:32,948 --> 00:24:35,888
of an end user's point of view or
a caller of one API that actually

541
00:24:36,048 --> 00:24:36,128
Hmm.

542
00:24:36,278 --> 00:24:40,598
out into, you know, this sort of
multi-step kind of stateful interaction.

543
00:24:41,018 --> 00:24:46,278
And generally, if you want to, if you want
to be able to do that kind of test then.

544
00:24:46,893 --> 00:24:50,123
You know, the mocks need to be you
know, they often need to be stateful,

545
00:24:50,123 --> 00:24:53,243
or at least they need to be sort of
pseudo stateful and, and model that

546
00:24:53,243 --> 00:24:54,743
behavior a bit more realistically,

547
00:24:55,073 --> 00:24:57,563
um, than the, the sort of sort
of stateless, canned response

548
00:24:57,563 --> 00:24:59,603
you can get away with if you're
doing like a narrow unit test.

549
00:24:59,988 --> 00:25:03,388
Talking about that kind of behavioral
functionality for the different

550
00:25:03,418 --> 00:25:06,688
personas, I think might kind
of break it down for folks that

551
00:25:06,688 --> 00:25:07,678
haven't thought about this too much.

552
00:25:07,758 --> 00:25:13,368
So if you are working on that
kind of API design or kind of

553
00:25:13,368 --> 00:25:15,368
documentation documentation like Dev X.

554
00:25:15,798 --> 00:25:20,568
Side of things, then it might be really
helpful for Yeah, it, it's helpful for you

555
00:25:20,568 --> 00:25:25,428
to have some behavior in the mock where
basically if you think of the, like stripe

556
00:25:25,428 --> 00:25:30,988
example where they used to say, if you
post post a payment with a credit card of

557
00:25:31,018 --> 00:25:33,448
like 4, 4, 4, 4, 4, then it will succeed.

558
00:25:33,448 --> 00:25:36,598
If it's like 4, 2, 4, 2, 4, 2,
then it'll fail, whatever that was.

559
00:25:36,778 --> 00:25:38,968
You can, you can get different
responses, come back.

560
00:25:39,283 --> 00:25:40,933
From, from the request.

561
00:25:40,933 --> 00:25:44,113
And that can be just a little bit
of looking at that and, and doing

562
00:25:44,113 --> 00:25:45,973
a rule to give you a response.

563
00:25:46,123 --> 00:25:50,143
Now, it doesn't necessarily have
to kind of persist that data to a

564
00:25:50,143 --> 00:25:55,363
point where you are making some other
requests to a list of previous payment

565
00:25:55,363 --> 00:25:57,103
attempts, and that one will show up.

566
00:25:57,293 --> 00:26:00,743
That would also be useful,
but in the kind of API design.

567
00:26:01,463 --> 00:26:05,363
Use case that's just literally, we've
got, we, we are trying to build the API

568
00:26:05,433 --> 00:26:08,703
client and the API itself in tandem.

569
00:26:08,973 --> 00:26:12,663
And we would like the kind of
iOS developer, web developer to

570
00:26:12,873 --> 00:26:14,643
verify that this will work at all.

571
00:26:14,643 --> 00:26:17,283
They can make requests in the right
sort of datas in the right place.

572
00:26:17,283 --> 00:26:19,668
It might be the wrong person's bloody
name when they get the response,

573
00:26:19,668 --> 00:26:23,373
but like, fundamentally, does it at
all kick in the tires help at all?

574
00:26:23,643 --> 00:26:26,493
And once they say yes, you can then
quickly rush off and write some of that.

575
00:26:26,498 --> 00:26:29,298
Code and then kind of build the
prototype and they can get that

576
00:26:29,298 --> 00:26:30,768
back and go, oh, that works lovely.

577
00:26:30,768 --> 00:26:31,518
Thank you very much.

578
00:26:31,518 --> 00:26:35,058
And you can go and tidy it up and
turn it up and use the open API as

579
00:26:35,148 --> 00:26:37,788
the contract test to verify that
you haven't changed it since then.

580
00:26:37,788 --> 00:26:40,758
So that's, that's really helpful
for mocking in the kind of API

581
00:26:40,758 --> 00:26:45,048
design world, but it absolutely
useless for most sort of qa.

582
00:26:45,788 --> 00:26:49,338
Functionality where you're like
you know, do, does a webhook

583
00:26:49,338 --> 00:26:51,708
fire off if I send a failed?

584
00:26:51,828 --> 00:26:55,848
You know, does, does the failed webhook
send off if I send a this credit card or

585
00:26:55,848 --> 00:26:58,548
if something else go off, if I do that
one where you need it to be more realistic

586
00:26:58,548 --> 00:27:02,028
and have more behavior in there, and
maybe talking about webhooks as a step

587
00:27:02,028 --> 00:27:03,678
too far, but like, you know what I mean?

588
00:27:03,678 --> 00:27:03,858
Yeah.

589
00:27:03,858 --> 00:27:04,968
No, we do WebBook as well.

590
00:27:04,973 --> 00:27:06,113
They're very popular feature.

591
00:27:06,263 --> 00:27:09,653
Yeah, I think more people use them
than, than admit to it, it seems.

592
00:27:10,598 --> 00:27:14,588
But could you, could you mock a open
API and have it sending off like

593
00:27:14,588 --> 00:27:17,258
mock webhooks in certain situations?

594
00:27:17,546 --> 00:27:20,786
so actually we don, we, the converter
doesn't convert to our web hooks

595
00:27:20,786 --> 00:27:21,386
automatically.

596
00:27:21,386 --> 00:27:22,476
I'm ashamed to say at the moment.

597
00:27:22,476 --> 00:27:25,476
I mean, we, it is one of those things
that's on, been on the roadmap for a

598
00:27:25,476 --> 00:27:28,436
while, but it's, you know, it's just
been less important than other things.

599
00:27:28,436 --> 00:27:29,006
So we haven't done it.

600
00:27:29,006 --> 00:27:29,246
There's no

601
00:27:29,246 --> 00:27:29,666
Mm-hmm.

602
00:27:29,966 --> 00:27:32,481
I mean, , the web hook definitions
you can define in WireMock are

603
00:27:33,231 --> 00:27:34,691
not identical syntactically,

604
00:27:34,741 --> 00:27:38,201
they're capable of accommodating
what you can describe in in open API.

605
00:27:38,201 --> 00:27:38,591
So

606
00:27:38,601 --> 00:27:39,021
That's cool.

607
00:27:39,051 --> 00:27:39,411
No, what

608
00:27:39,461 --> 00:27:41,711
whenever anyone mentions anything,
I'm like, oh, what about this?

609
00:27:41,741 --> 00:27:44,891
Like you, I, I will always have an idea
on top of what it is you currently do.

610
00:27:44,891 --> 00:27:46,451
I'll be like, but that's great.

611
00:27:46,451 --> 00:27:47,051
But what about that?

612
00:27:47,141 --> 00:27:47,591
Oh, sorry.

613
00:27:47,591 --> 00:27:48,161
We haven't done that yet.

614
00:27:48,701 --> 00:27:49,901
No worries.

615
00:27:50,201 --> 00:27:50,651
I think.

616
00:27:50,981 --> 00:27:52,651
Just to, pull on a thread.

617
00:27:52,661 --> 00:27:54,281
You started a moment ago.

618
00:27:54,355 --> 00:27:57,205
The prototyping these really is, is
really interesting and, and actually

619
00:27:57,475 --> 00:28:00,295
may, it's may be a slightly sort of
controversial type, but I dunno, maybe

620
00:28:00,295 --> 00:28:01,525
one you'd be sympathetic towards.

621
00:28:01,525 --> 00:28:05,955
I so we, we, we sort of promote the
idea of using mox as prototypes.

622
00:28:06,080 --> 00:28:06,200
Hmm.

623
00:28:06,345 --> 00:28:09,045
I, I think, again, depending on which
of these two personas you're in, you

624
00:28:09,045 --> 00:28:12,385
know, that it, it's a kind of similar
activity, but sort of with different.

625
00:28:12,680 --> 00:28:14,690
Different terminology
and , different emphasis.

626
00:28:14,690 --> 00:28:18,250
So some of our first customers actually,
the reason they, they, they they bought

627
00:28:18,250 --> 00:28:21,580
the product was because they had, you
know, backend teams building APIs and

628
00:28:21,580 --> 00:28:23,620
then they had a bunch of different
front end teams like, you know,

629
00:28:23,650 --> 00:28:25,090
mobile and web and so on like that.

630
00:28:25,450 --> 00:28:28,240
And they were frustrated at
having to, you know, work in

631
00:28:28,240 --> 00:28:30,040
this very serialized way because,

632
00:28:30,070 --> 00:28:30,150
Hmm.

633
00:28:30,190 --> 00:28:30,850
know, they would.

634
00:28:31,715 --> 00:28:35,695
Come up with, yeah, a new feature would
need a new, new bit of API and they would

635
00:28:35,755 --> 00:28:38,945
sit and whiteboard it and figure out
what they, they thought it should do.

636
00:28:39,060 --> 00:28:40,005
Six months later.

637
00:28:40,115 --> 00:28:42,065
would go off and build something and

638
00:28:42,115 --> 00:28:42,405
Yeah.

639
00:28:42,575 --> 00:28:44,315
a test environment a few weeks later.

640
00:28:44,645 --> 00:28:48,155
And only then could the, the, the teams
calling it really kind of get going.

641
00:28:48,765 --> 00:28:52,305
Building stuff, and then there'd be this
sort of slightly long, painful feedback

642
00:28:52,305 --> 00:28:55,635
loop where, oh, actually we forgot all
of this stuff in the design session.

643
00:28:55,635 --> 00:28:58,425
And you know, now the backend
team have to go and remediate it.

644
00:28:58,845 --> 00:29:03,155
And the, know, use, using Mark
kind of solves two problems there.

645
00:29:03,155 --> 00:29:06,305
You know, one, one, it means that
you can, you can very quickly

646
00:29:06,305 --> 00:29:09,185
have something that the front end
teams can start building against.

647
00:29:09,185 --> 00:29:15,410
So they're not waiting for, for the
backend teams to, To ship something, all

648
00:29:15,440 --> 00:29:19,730
but by, by doing that, it gives, it gives
the opportunity for, for early feedback

649
00:29:19,970 --> 00:29:20,260
Yeah.

650
00:29:20,430 --> 00:29:22,200
It means that they, you know,
by Yeah, because this is the

651
00:29:22,200 --> 00:29:23,820
thing you, you don't really know.

652
00:29:24,265 --> 00:29:26,845
Until you've actually built something
against an API, whether it, whether,

653
00:29:26,905 --> 00:29:30,475
I mean, I know, I know there's lots of
other, you know, there, there's governance

654
00:29:30,475 --> 00:29:33,266
tooling and there are lots of sort of
heuristics you can apply to APIs to,

655
00:29:33,271 --> 00:29:34,615
to, to say whether they're good or not.

656
00:29:34,975 --> 00:29:37,935
But it's like, you know, sort
of a, you know, like a, an ounce

657
00:29:37,935 --> 00:29:40,665
of practice is worth a ton of
theory in, in, in this case.

658
00:29:40,665 --> 00:29:42,615
You know, you try and if you try
and build something against an

659
00:29:42,615 --> 00:29:44,025
API, then, then you'll really know.

660
00:29:44,805 --> 00:29:46,755
it's fit for purpose, if
this design is any good.

661
00:29:46,965 --> 00:29:50,355
And so the earlier you can do that,
and, and, and also, and, and this is

662
00:29:50,355 --> 00:29:51,405
the crucial point I was gonna make.

663
00:29:51,405 --> 00:29:55,735
The, the, the, the more sort of
breadth of behavior of that API

664
00:29:55,735 --> 00:29:57,355
you can cover in that exercise.

665
00:29:57,755 --> 00:30:01,025
The, the quicker you are gonna
converge on an API design, which

666
00:30:01,085 --> 00:30:03,235
is which is really suitable and,

667
00:30:03,235 --> 00:30:03,355
Hmm.

668
00:30:04,405 --> 00:30:07,735
You know, the, the, I I think what,
you know, one, one of the, the, the

669
00:30:07,735 --> 00:30:11,625
downsides of using a the tools that
sort of do, do you like a one shot open

670
00:30:11,625 --> 00:30:15,795
API to mock generation is that you,
you lose the ability to go and exercise

671
00:30:15,795 --> 00:30:16,995
all those kind of secondary parts.

672
00:30:16,995 --> 00:30:20,245
So I'd say particularly if you have a
a very kinda state heavy API, you know,

673
00:30:20,245 --> 00:30:23,925
one where the response you get back,
you know, the, the, the structure of the

674
00:30:23,925 --> 00:30:26,925
payload for instance, varies quite a lot
depending on the state of the resource.

675
00:30:27,145 --> 00:30:29,075
That you've retrieved then yeah.

676
00:30:29,075 --> 00:30:32,165
That's the sort of thing that, that
doesn't, you know, it, it doesn't really,

677
00:30:32,225 --> 00:30:33,485
it kind of gets lost in all of that.

678
00:30:33,485 --> 00:30:36,425
You know, you get the one
representative response back, which

679
00:30:36,425 --> 00:30:39,335
is probably just gonna be like
one state outta the set, you know?

680
00:30:39,335 --> 00:30:42,125
And, and if you're using one of those
tools and if you can actually, yeah.

681
00:30:42,125 --> 00:30:44,585
If you have a, a tooling that
lets you, you model that in a

682
00:30:44,585 --> 00:30:46,145
slightly more sophisticated way.

683
00:30:46,450 --> 00:30:50,620
So that you can get through all of that
and you can cr you know, you can raise the

684
00:30:50,620 --> 00:30:54,100
level of realism, of the experience that
the consumer gets, then you're, you're,

685
00:30:54,190 --> 00:30:59,720
you're much more likely to, to produce
a a better design more quickly that you

686
00:30:59,720 --> 00:31:03,140
know, that you can then ship something
with, rather than having to fi Yeah.

687
00:31:03,140 --> 00:31:06,140
Again, getting into that kind of, you
know, shipping stuff into a production

688
00:31:06,140 --> 00:31:09,650
like environment and testing it,
and only then realizing that, that

689
00:31:09,650 --> 00:31:12,110
actually, you know, when, when we,
when we get into the real staple

690
00:31:12,110 --> 00:31:13,580
version of this a p, it's not what we.

691
00:31:14,900 --> 00:31:15,710
Yeah, absolutely.

692
00:31:15,710 --> 00:31:18,710
I feel like the, the, the key thing
is to make sure that it's not.

693
00:31:20,210 --> 00:31:24,860
Somebody asked for functionality, they get
told to go and look at their shoes for six

694
00:31:24,860 --> 00:31:26,750
months until we finished building it all.

695
00:31:26,750 --> 00:31:29,060
We had to rush because people
were complaining at us.

696
00:31:29,060 --> 00:31:31,340
So we built some rubbish and
we told ourselves it would be a

697
00:31:31,340 --> 00:31:34,070
prototype, but actually we've ended
up just shipping that live and it

698
00:31:34,070 --> 00:31:35,090
doesn't really do what they said.

699
00:31:35,120 --> 00:31:36,890
'cause the whole six month
wait with no feedback.

700
00:31:36,890 --> 00:31:38,150
And that's just a mess like that.

701
00:31:38,390 --> 00:31:40,370
That's basically like default in API.

702
00:31:40,755 --> 00:31:41,625
That's what everyone does.

703
00:31:41,835 --> 00:31:42,495
That's just what they do.

704
00:31:42,675 --> 00:31:45,795
And it's why I've spent so much
time being like API design first is

705
00:31:45,975 --> 00:31:47,985
good, actually you should try this.

706
00:31:48,235 --> 00:31:51,415
'Cause it sounds like just being
the nerd that was like, you have

707
00:31:51,415 --> 00:31:55,475
the decade that everyone was running
around, like the religion of TDD or

708
00:31:55,475 --> 00:31:56,915
the religion of anything else, right.

709
00:31:57,125 --> 00:32:00,575
Of just, Hey, we've got this wacky
idea that doesn't really make a lot of

710
00:32:00,575 --> 00:32:02,315
sense and it massively slows you down.

711
00:32:02,315 --> 00:32:06,515
Or like yeah, like domain driven design
for everything all the time and just.

712
00:32:07,840 --> 00:32:07,990
It.

713
00:32:07,990 --> 00:32:12,310
It kind of sounds like the sort of
thing that similar, those similar types

714
00:32:12,310 --> 00:32:16,330
of people would say, but it genuinely
saves everyone a whole crap load of

715
00:32:16,330 --> 00:32:20,770
time if you just actually sit down,
talk about what you think you might

716
00:32:20,770 --> 00:32:22,540
want, and then make a mock for it.

717
00:32:22,720 --> 00:32:24,670
And that's step one is good for everyone.

718
00:32:24,670 --> 00:32:27,880
Whatever mocking tool you use, probably
you can do that step one where you're

719
00:32:27,880 --> 00:32:29,650
like, Hey, here's, here's a thing.

720
00:32:29,955 --> 00:32:33,575
The Prism classic example was you using
the pet dog 'cause it was pre-trained

721
00:32:33,575 --> 00:32:35,255
travel API, and it's burned into my brain.

722
00:32:35,615 --> 00:32:39,395
But the example of you post a cat and
you get a dog as a response, right?

723
00:32:39,555 --> 00:32:41,745
Tech technically fine, technically valid.

724
00:32:41,745 --> 00:32:45,045
Like you, you send it a request
and you get something back

725
00:32:45,045 --> 00:32:46,605
and you can do the most basic.

726
00:32:46,935 --> 00:32:50,565
I have built an interface that when
I send this off, I will then get a

727
00:32:50,565 --> 00:32:52,335
response and I can build something.

728
00:32:52,545 --> 00:32:55,795
But if you want to get a a,
a cat back instead of a dog.

729
00:32:55,975 --> 00:32:58,255
You have to start adding in
these weird little switches.

730
00:32:58,255 --> 00:33:03,035
And I think we did something like using
the prefer header where you kind of

731
00:33:03,085 --> 00:33:08,725
add a HTP you start messing with what's
real and what's the, what's the thing?

732
00:33:08,810 --> 00:33:11,600
that's, that's always been one of my
kind of cardinal design principles

733
00:33:11,600 --> 00:33:12,860
actually is ne never do that.

734
00:33:12,860 --> 00:33:13,310
You know, the,

735
00:33:13,315 --> 00:33:13,676
Yeah, yeah.

736
00:33:13,730 --> 00:33:17,050
caller should never have to change
its behavior in order to to interact

737
00:33:17,050 --> 00:33:18,460
with a mock versus a real thing.

738
00:33:18,825 --> 00:33:19,185
Yeah.

739
00:33:19,540 --> 00:33:23,140
you know, the, you, you should always
try and try and find ways of using the.

740
00:33:23,485 --> 00:33:26,365
Yeah, the signals that are sort of
intrinsic to the, the call that's being

741
00:33:26,365 --> 00:33:28,905
made to you know, to, to affect behavior.

742
00:33:29,305 --> 00:33:30,990
Even if that's kind of harder to do in,

743
00:33:31,075 --> 00:33:31,645
For sure.

744
00:33:31,705 --> 00:33:31,945
Yeah.

745
00:33:31,975 --> 00:33:36,075
'cause I think all, all of our like all
of prism's validation responses, if you

746
00:33:36,075 --> 00:33:40,605
send a request and it's invalid, you
would get like RFC 7, 8, 0 7 or whatever.

747
00:33:40,605 --> 00:33:41,415
There's a new one now.

748
00:33:41,415 --> 00:33:44,335
But you would get that kind
of standard error response.

749
00:33:44,485 --> 00:33:47,965
And then under the hood, I think that
was, I, I did suggest that to all,

750
00:33:47,965 --> 00:33:52,165
and I was thinking, well, with my mole
on of, of course, everyone's using

751
00:33:52,165 --> 00:33:53,755
the standard for error responses or.

752
00:33:54,780 --> 00:33:57,990
And, and e even if I didn't truly
believe that, I'm like, but they should.

753
00:33:57,990 --> 00:33:59,670
So, you know, fuck 'em, it's a problem.

754
00:34:00,130 --> 00:34:03,400
But I had, we had to do something
of just like, there has been

755
00:34:03,400 --> 00:34:06,430
an issue, the response, the
validation response wasn't defined.

756
00:34:06,430 --> 00:34:07,720
Let's take a guess at this.

757
00:34:07,870 --> 00:34:10,840
And then they actually go ahead
and implement the error format.

758
00:34:10,900 --> 00:34:11,710
And it's just like.

759
00:34:11,830 --> 00:34:14,860
A simple string or a text
or HDML or something.

760
00:34:15,260 --> 00:34:17,900
And so again, you, you kind of
end up blurring the lines and all

761
00:34:17,900 --> 00:34:22,970
of the line blurring just always
felt weird, but it's, it's hard to

762
00:34:22,970 --> 00:34:25,010
get functionality out of a mock.

763
00:34:25,070 --> 00:34:30,320
And so I think we, we skipped from basic
dumb mocking with that little bit of

764
00:34:30,320 --> 00:34:33,620
faker to come up with some, some, well,
whatever's an email address shut up.

765
00:34:34,080 --> 00:34:36,570
We skipped straight from
mocking through the proxy.

766
00:34:36,570 --> 00:34:38,850
We had functionality that was.

767
00:34:39,995 --> 00:34:44,105
You could proxy to an actual
implementation, which would

768
00:34:44,265 --> 00:34:48,255
if you returned a 5 0 1 not
implemented, would return the mock.

769
00:34:48,675 --> 00:34:51,225
Whereas if you returned anything
else, it would return the real one.

770
00:34:51,345 --> 00:34:54,315
And so that kind of blurred
the line between mock and

771
00:34:54,315 --> 00:34:56,205
prototype and beta really.

772
00:34:56,205 --> 00:34:57,645
'cause what we were basically saying was.

773
00:34:58,320 --> 00:35:01,620
Instead of launching into a prototype
where you're trying to get that fast

774
00:35:01,620 --> 00:35:05,340
feedback, but actually it will still
take a while to make this prototype.

775
00:35:05,340 --> 00:35:07,530
And actually you'll probably end up
getting attached to it and trying

776
00:35:07,530 --> 00:35:11,160
to put it into production instead
of that awkward jack of all trades.

777
00:35:11,160 --> 00:35:11,910
Not very helpful.

778
00:35:11,910 --> 00:35:16,140
Middle of the road prototype first
approach, we were saying step one,

779
00:35:16,140 --> 00:35:20,370
kick the tires with this kind of
shy mock and then you can use that

780
00:35:20,370 --> 00:35:23,790
feedback to quickly go and implement
endpoint at a time over here.

781
00:35:24,090 --> 00:35:26,460
And it can be a slightly better kind of.

782
00:35:26,865 --> 00:35:31,185
Build, which you can start to put
live piece at a time as and, and so

783
00:35:31,185 --> 00:35:36,765
instead of the entire API as waterfall,
bits of it were slightly more agile.

784
00:35:37,155 --> 00:35:39,105
And they could, they could use
the mock for something else.

785
00:35:39,295 --> 00:35:39,505
Thing.

786
00:35:39,505 --> 00:35:42,505
I mean, it's slightly different approach
to it, but you know, with wok you can,

787
00:35:42,535 --> 00:35:46,500
you can set up a proxy stub that says,
when these criteria are matched, you

788
00:35:46,500 --> 00:35:48,300
go and proxy off to this other address

789
00:35:48,345 --> 00:35:48,465
I

790
00:35:48,600 --> 00:35:48,930
than

791
00:35:49,040 --> 00:35:49,330
Nice.

792
00:35:49,645 --> 00:35:49,705
aub.

793
00:35:50,005 --> 00:35:52,465
So typically what you do is you, you
have one of those that's very loose,

794
00:35:52,465 --> 00:35:56,485
matching as a, as at a low priority, and
then you stack a load of specific stubs

795
00:35:56,485 --> 00:35:58,315
on top of that for the things that you.

796
00:35:58,555 --> 00:36:00,745
And essentially there's two use
cases, either like a load of

797
00:36:00,745 --> 00:36:03,265
things you haven't built yet,
so they only exist in stub form.

798
00:36:03,575 --> 00:36:06,215
Or there's because the other,
the other funding you do with

799
00:36:06,215 --> 00:36:07,715
wok is really low level faults.

800
00:36:07,715 --> 00:36:10,295
You know, you can like send reset
packets and drop the connection

801
00:36:10,295 --> 00:36:11,015
and all that kind of thing.

802
00:36:11,365 --> 00:36:11,815
Oh, cool.

803
00:36:12,155 --> 00:36:14,405
and, and you know, based
on matching criteria.

804
00:36:14,405 --> 00:36:17,675
So, so if you want to use it as a
kind of fault injection engine or two.

805
00:36:18,305 --> 00:36:23,645
To inject unusual responses that you can't
make the real test environment make, then

806
00:36:23,645 --> 00:36:25,205
you can just kind of layer those on top.

807
00:36:25,205 --> 00:36:28,565
So, so what WMA will do is that, you know,
request comes in, it'll run down the list

808
00:36:28,565 --> 00:36:30,755
and say, does this request match anything?

809
00:36:31,055 --> 00:36:34,265
If yes, oh, okay, this is the go
and send a reset packet response.

810
00:36:34,265 --> 00:36:34,985
So I'll do that.

811
00:36:35,355 --> 00:36:37,395
If not, it runs all the way down
the list, gets to the proxy and

812
00:36:37,395 --> 00:36:40,135
says, okay, I'll go to the test
environment and serve a real test,

813
00:36:40,420 --> 00:36:40,870
That's good.

814
00:36:40,870 --> 00:36:44,440
So you can kind of skip needing to
set up proxy, proxy or similar as

815
00:36:44,440 --> 00:36:48,970
a, as a separate tool and just say
like, if, if this request comes from

816
00:36:48,970 --> 00:36:52,330
Dave, give him a really bad day and
it'll be a really slow response.

817
00:36:52,330 --> 00:36:56,670
Or, or like anyone, or you could literally
say like, if this if, if this is coming

818
00:36:56,670 --> 00:37:00,850
from you could pretend like certain
country is getting a very slow response.

819
00:37:00,850 --> 00:37:02,110
Does it, does it work?

820
00:37:02,485 --> 00:37:05,875
Realistically well with the client if
the responses are that slow or do we need

821
00:37:05,875 --> 00:37:08,005
to go deploy another server over there?

822
00:37:08,305 --> 00:37:09,145
That's really helpful.

823
00:37:09,805 --> 00:37:10,195
Yeah.

824
00:37:10,195 --> 00:37:11,015
No, it's very.

825
00:37:11,340 --> 00:37:12,450
Powerful way of using it.

826
00:37:12,450 --> 00:37:15,450
And as I say, whe whether another kind of
sub use case of the QA thing, you know,

827
00:37:15,450 --> 00:37:18,680
is the kind of non-functional you know,
the, the possibility for non-functional

828
00:37:18,680 --> 00:37:19,910
testing that mocking opens up.

829
00:37:19,940 --> 00:37:22,830
You know, you can you
can go and you can run

830
00:37:22,920 --> 00:37:23,040
I.

831
00:37:23,070 --> 00:37:25,920
tests and soap tests and you know, things
that are, that are about performance

832
00:37:25,920 --> 00:37:29,580
and stability, but you can create a
set of environmental conditions that

833
00:37:29,580 --> 00:37:33,360
you are, are really hard to kind of
make real systems create on demand.

834
00:37:33,640 --> 00:37:33,800
Yeah.

835
00:37:33,800 --> 00:37:34,080
Yeah.

836
00:37:34,245 --> 00:37:37,865
Quite a few years ago I was working at a,
a big retailer in, in the uk and, and they

837
00:37:37,865 --> 00:37:41,105
had the sort of the, you know, the same
kind of Black Friday problems that every,

838
00:37:41,120 --> 00:37:44,995
all you all retailers did, you know, where
all of their customers would show up on

839
00:37:44,995 --> 00:37:48,275
one day and they would, you know, hammer
the hell out of their site and all the

840
00:37:48,275 --> 00:37:51,875
payment gateways would be running, you
know, taking four times as long to respond

841
00:37:52,115 --> 00:37:52,565
Mm,

842
00:37:52,655 --> 00:37:53,945
they're all getting hammered as well.

843
00:37:54,365 --> 00:37:54,965
And

844
00:37:55,075 --> 00:37:56,305
Immensely stressful for

845
00:37:56,455 --> 00:37:56,515
Yeah.

846
00:37:56,725 --> 00:37:59,145
involved if you're a techie,
but the, the nice thing about

847
00:37:59,145 --> 00:38:00,045
obviously Yeah, the, yeah.

848
00:38:00,045 --> 00:38:02,625
The problem you have when you are
testing in the run up to Black Friday

849
00:38:02,625 --> 00:38:04,155
is, it's not currently Black Friday, you

850
00:38:04,245 --> 00:38:04,695
Yeah.

851
00:38:04,755 --> 00:38:08,055
your, your systems are, are
not under load in the same way.

852
00:38:08,490 --> 00:38:11,799
And you know, the, the third
party system you depend on

853
00:38:11,980 --> 00:38:13,150
aren't under load and they're not

854
00:38:13,300 --> 00:38:13,520
Mm.

855
00:38:13,569 --> 00:38:14,830
weirdly and all that kind of thing.

856
00:38:15,259 --> 00:38:16,770
You can very easily deceive yourself.

857
00:38:16,770 --> 00:38:19,410
And I, I mean, I've lost count at the
number of times I've seen this happen.

858
00:38:19,410 --> 00:38:22,439
You know, where organizations have
done what they think is really

859
00:38:22,439 --> 00:38:25,830
extensive performance testing in
ideal environmental conditions.

860
00:38:26,520 --> 00:38:31,410
And then, and then the big day comes up
when, when everything's going to hell and.

861
00:38:31,549 --> 00:38:34,640
Something that they didn't test for
or predict happens because, you know,

862
00:38:34,640 --> 00:38:35,900
because some crucial aspects of the

863
00:38:36,155 --> 00:38:36,445
Yeah.

864
00:38:36,720 --> 00:38:40,029
The great thing about, about all
this about using mocking tools that

865
00:38:40,029 --> 00:38:43,199
have these kind of non-functional
features is that you can deliberately

866
00:38:43,199 --> 00:38:46,709
degrade aspects in a very controlled
way of the, the environment and

867
00:38:46,874 --> 00:38:47,284
Mm.

868
00:38:47,339 --> 00:38:48,990
out ahead of time what's gonna happen.

869
00:38:49,474 --> 00:38:50,044
That's brilliant.

870
00:38:50,044 --> 00:38:53,244
Yeah, I mean, that was for my,
my two years at WeWork of just

871
00:38:53,244 --> 00:38:57,354
dealing with the most insane
problems of systems architecture.

872
00:38:57,354 --> 00:39:01,074
Where there was a favorite example
of all is that one where there was

873
00:39:01,074 --> 00:39:04,374
a third party, API, that everyone
forgot about that handled like.

874
00:39:04,734 --> 00:39:08,004
Adding new key cards for new
members of the coworking space.

875
00:39:08,214 --> 00:39:12,164
And that was handled by some random
company who had a SLA promising it

876
00:39:12,164 --> 00:39:15,204
would be always less than a second,
never more to get a response.

877
00:39:15,204 --> 00:39:20,115
And we had, I think there was
like, I. West coast, east coast and

878
00:39:20,115 --> 00:39:23,115
other, which was fine when other
was like Australia and London.

879
00:39:23,115 --> 00:39:26,084
And then eventually it was like China
and Russia and Brazil and every single

880
00:39:26,084 --> 00:39:29,394
other major country you can think of
all on this one TV little like EC2

881
00:39:29,394 --> 00:39:30,474
instance floating about somewhere.

882
00:39:30,834 --> 00:39:34,824
And yeah, every time Australia had a
busy day on the first of the month when

883
00:39:34,824 --> 00:39:38,664
all their customers turned up to, to
check in for their new contract, no one

884
00:39:38,664 --> 00:39:41,874
in Los Angeles could go to the bathroom
because the key cards wouldn't work.

885
00:39:43,669 --> 00:39:46,489
And things like that of just like,
well, we've tested all of our APIs.

886
00:39:46,489 --> 00:39:49,699
It's like, right, but you did forget to
test that other dependency that you had.

887
00:39:49,799 --> 00:39:52,709
And so yeah, you could just like save
a couple of requests and responses.

888
00:39:52,709 --> 00:39:55,199
You don't actually care what
the actual thing coming back is.

889
00:39:55,559 --> 00:39:56,159
Exactly.

890
00:39:56,159 --> 00:39:59,069
You just want it to cut, you know, when
it does come back, it's a valid response.

891
00:39:59,069 --> 00:39:59,549
That's fine.

892
00:39:59,834 --> 00:40:02,924
But you want to test what happens if
that takes two minutes to respond.

893
00:40:02,924 --> 00:40:04,884
Like it was it was like two minutes.

894
00:40:04,974 --> 00:40:05,724
That's mad.

895
00:40:05,914 --> 00:40:08,884
That's so slow that we've actually
kind of rejected it from our logging

896
00:40:08,884 --> 00:40:11,434
system because they, they said
it would be less than a second.

897
00:40:11,434 --> 00:40:13,444
So two minutes must be the
logging system's broken.

898
00:40:13,834 --> 00:40:16,474
So you need to be able to, you need
to be able to test things like that.

899
00:40:16,564 --> 00:40:18,754
I've spent a lot of time doing
this kind of performance and

900
00:40:18,754 --> 00:40:21,794
resilience engineering, those runtime
characteristics , the systems you're

901
00:40:21,794 --> 00:40:26,044
building and, and in particular with
respect to APIs is is really important.

902
00:40:26,074 --> 00:40:28,894
You know, the difference between
something taking, you know, half a

903
00:40:28,894 --> 00:40:32,854
second to respond versus two seconds
in the context of a busy system,

904
00:40:33,079 --> 00:40:33,319
Yeah,

905
00:40:33,904 --> 00:40:35,674
is, is absolutely enormous.

906
00:40:35,674 --> 00:40:37,984
And and you know, it can mean
the difference between stability

907
00:40:37,984 --> 00:40:39,364
and instability fundamentally.

908
00:40:39,764 --> 00:40:40,339
for sure.

909
00:40:40,709 --> 00:40:43,749
There are techniques like sort of
chaos engineering that you can sort

910
00:40:43,749 --> 00:40:46,979
of bring, bring to bear in order to
try and make sense of these things.

911
00:40:46,979 --> 00:40:50,879
But I, I think, suppose with that in
particular, no, no criticism of it at all.

912
00:40:50,879 --> 00:40:53,429
I think there are probably you know,
if you're trying to build really highly

913
00:40:53,429 --> 00:40:55,109
scale, highly available systems, then.

914
00:40:55,344 --> 00:40:58,434
Then it's an essential technique, but
the, the problem is, it is a, there's

915
00:40:58,434 --> 00:41:03,914
quite a big barrier to even, you know I,
I think starting doing that kind of thing.

916
00:41:04,904 --> 00:41:08,264
so if you can deploy these little tactics
where, I mean, I, I often wonder how many,

917
00:41:08,294 --> 00:41:09,614
how many organizations have run kale?

918
00:41:09,769 --> 00:41:10,429
Experiments,

919
00:41:10,904 --> 00:41:11,264
Yeah.

920
00:41:11,359 --> 00:41:14,719
know, these big expensive things where
you have whole teams of people involved

921
00:41:14,719 --> 00:41:18,199
and you have to have done all of this kind
of DevOps maturity work to even get to

922
00:41:18,199 --> 00:41:19,999
the point where it's safe to, to do it.

923
00:41:20,359 --> 00:41:24,409
And then all you find is like someone's
set a time out too high on, on some API

924
00:41:24,464 --> 00:41:25,244
Yeah.

925
00:41:25,244 --> 00:41:25,304
Yeah.

926
00:41:25,309 --> 00:41:28,789
kind of think, you know, fine, you know,
it's good that you found it, but maybe you

927
00:41:28,789 --> 00:41:32,699
could have found that in a. Much earlier
in the process and much more cheekly

928
00:41:32,699 --> 00:41:36,659
and, you know, with much, much less kind
of labor involved than, than this way.

929
00:41:36,719 --> 00:41:37,199
And,

930
00:41:37,224 --> 00:41:37,514
Yeah.

931
00:41:37,559 --> 00:41:39,779
so I, I kind of like to think
about how, you know, how you can,

932
00:41:40,049 --> 00:41:42,929
how you can kind of surface those
things as, as early as possible.

933
00:41:42,929 --> 00:41:45,059
You know, it's kind of, it's,
it's like with the prototyping

934
00:41:45,059 --> 00:41:46,019
thing really, I think you,

935
00:41:48,029 --> 00:41:52,069
the, a lot of this is about, about sort
of, I mean, I, you know, I say I, I,

936
00:41:52,069 --> 00:41:55,249
I admit, I, I was a, a sort of fairly
fanatical XP adopter back in the day.

937
00:41:55,259 --> 00:41:58,349
I do still have this fairly, sort
of strong belief and bias towards

938
00:41:58,349 --> 00:42:03,679
the idea that if you can make
feedback easy and early, you'll

939
00:42:03,679 --> 00:42:06,679
end up with a, a better quality
product being produced more quickly.

940
00:42:06,729 --> 00:42:10,269
Both of these examples, the prototyping
example and the, the fault injection.

941
00:42:10,269 --> 00:42:13,719
If you make something that's sort
of trivially easy for someone to, to

942
00:42:13,719 --> 00:42:16,629
discover the sort of correctness and
properties of, and so like that, then

943
00:42:16,629 --> 00:42:18,909
they'll do it and they'll, you know,
it'll get built into people's routines.

944
00:42:19,209 --> 00:42:22,929
Whereas if you make it happen late
and, and it's expensive and requires

945
00:42:22,929 --> 00:42:26,199
lots of inve investment and upskilling,
then it, it probably won't happen.

946
00:42:26,249 --> 00:42:29,784
There's a sort of , an example I heard
from a friend about this recently

947
00:42:29,784 --> 00:42:33,644
where, i, you know, prototyping
API prototyping wasn't being used.

948
00:42:33,644 --> 00:42:37,214
So he works in one of the large banks, you
know, and they have obviously, you know,

949
00:42:37,214 --> 00:42:41,444
huge sprawling kind of API integrated
systems of, of various generations.

950
00:42:41,864 --> 00:42:45,014
And they, they were, there was a,
a team he was involved with that

951
00:42:45,014 --> 00:42:48,434
were working on an API that sort of
fronted, you know, this sort of layer

952
00:42:48,434 --> 00:42:51,524
upon layer of, of legacy technology,
you know, all the way back to kind

953
00:42:51,524 --> 00:42:52,664
of bits of mainframe, I think.

954
00:42:53,234 --> 00:42:54,314
And they.

955
00:42:54,884 --> 00:42:55,814
They, you know, they've done this.

956
00:42:55,874 --> 00:42:56,084
Yeah.

957
00:42:56,084 --> 00:42:58,664
The, what I described earlier,
this kind of whiteboard exercise

958
00:42:58,664 --> 00:43:01,774
of sketching out what this API
needed to do in order to build this

959
00:43:01,774 --> 00:43:03,154
new product they wanted to launch.

960
00:43:03,554 --> 00:43:06,374
And then, you know, they did all of
that months and months of work happened.

961
00:43:06,374 --> 00:43:08,834
You know, a first version of this, a
p was shipped into a test environment

962
00:43:08,834 --> 00:43:11,764
and then someone immediately went we're
missing that ID field that we need

963
00:43:11,764 --> 00:43:13,414
in order to be able to do any of this

964
00:43:13,594 --> 00:43:13,884
Yeah.

965
00:43:13,889 --> 00:43:14,299
Yeah.

966
00:43:14,524 --> 00:43:17,164
and, and, and this resulted
in a three month project

967
00:43:17,224 --> 00:43:17,554
What.

968
00:43:17,719 --> 00:43:19,759
five teams or something
like that in order to.

969
00:43:20,094 --> 00:43:22,914
You know, to be able to go and wire
everything up to, to surface this,

970
00:43:22,914 --> 00:43:24,684
this one additional I Id field.

971
00:43:25,299 --> 00:43:25,479
Oof.

972
00:43:26,184 --> 00:43:29,274
and I, you know, people, people
doing like greenfield microservices

973
00:43:29,274 --> 00:43:32,034
kind of go, oh, it's easy to
change things, it's no problem.

974
00:43:32,034 --> 00:43:32,334
But,

975
00:43:32,374 --> 00:43:32,664
Yeah.

976
00:43:32,764 --> 00:43:35,234
Yeah, I suppose the other thing I
always like to point out is that a

977
00:43:35,234 --> 00:43:36,554
lot of us are not in that situation.

978
00:43:36,634 --> 00:43:40,294
A lot of it's build APIs as
modernizing facades over very

979
00:43:40,294 --> 00:43:41,854
hard to change technology.

980
00:43:41,884 --> 00:43:44,614
And so the, the, the cost of
getting it wrong is enormous.

981
00:43:45,649 --> 00:43:46,579
Yeah, that's rough.

982
00:43:47,029 --> 00:43:50,479
Good job from pulling us back from the
brink from me, about to launch into

983
00:43:50,479 --> 00:43:55,149
a tirade about how at this point in
late stage capitalism with the finance

984
00:43:55,149 --> 00:44:00,479
bros taking over the world of tech,
we are currently actively firing.

985
00:44:00,734 --> 00:44:04,844
Every single engineer that can't
be replaced with copilot instead of

986
00:44:04,844 --> 00:44:08,864
investing in things like a group of
10 people whose whole job it is, is

987
00:44:08,864 --> 00:44:10,094
to sit around and fuck things up.

988
00:44:10,094 --> 00:44:11,954
On the off chance there's
an issue they can discover.

989
00:44:12,284 --> 00:44:14,954
Well, that's probably another
topic for another day.

990
00:44:15,194 --> 00:44:18,824
But on the topic of ai, the last thing
I wanted to, the only thing I really

991
00:44:18,824 --> 00:44:22,424
wanna mention about that is that
I've seen a few companies starting to

992
00:44:22,424 --> 00:44:24,884
do like AI mocking and I feel like.

993
00:44:26,399 --> 00:44:27,389
How do you feel about that?

994
00:44:27,389 --> 00:44:28,769
Is that something you
have on your roadmap?

995
00:44:28,769 --> 00:44:32,029
I know you have MCP, that's a topic
for another podcast and it makes

996
00:44:32,029 --> 00:44:34,429
sense for you to support all the
protocols that you possibly can.

997
00:44:34,799 --> 00:44:39,539
But for AI responses we've talked
about, like static and dynamic,

998
00:44:39,539 --> 00:44:42,599
we've talked about faker, we've
talked about kind of persisting

999
00:44:42,604 --> 00:44:44,099
data and trying to wire it back up.

1000
00:44:44,099 --> 00:44:48,149
We've talked about trying to glue together
kind of behavioral stuff in a, in a

1001
00:44:48,149 --> 00:44:49,889
different mock, in a different format.

1002
00:44:50,789 --> 00:44:55,169
How do you feel like AI can help that, or
do you feel like it's a silly distraction,

1003
00:44:55,409 --> 00:44:57,089
hype, bandwagon, nonsense thing?

1004
00:44:58,139 --> 00:45:00,929
So, so I, I think it
can actually be helpful.

1005
00:45:01,009 --> 00:45:05,149
I agree that the, you know, with ai, the,
the, the hype often runs, you know, a

1006
00:45:05,149 --> 00:45:07,399
bit ahead of, of you know, the real, real

1007
00:45:07,464 --> 00:45:07,584
I.

1008
00:45:07,969 --> 00:45:08,269
of it.

1009
00:45:08,269 --> 00:45:12,419
But but nonetheless, actually, I
think this is a, a domain where AI can

1010
00:45:12,489 --> 00:45:13,839
you know, can be genuinely helpful.

1011
00:45:14,319 --> 00:45:17,779
So we one, one of the things we see a
lot, you know, organizations that, that,

1012
00:45:17,844 --> 00:45:19,854
that use mocks kind of really extensively.

1013
00:45:21,414 --> 00:45:25,044
they, you know, traditionally, you know,
you know, particularly kind of, obviously

1014
00:45:25,884 --> 00:45:28,704
the last three years trying to build
a, a product, which, which makes lot,

1015
00:45:28,704 --> 00:45:31,974
lots of aspects of this easier, but sort
of prior to that where people, we, we

1016
00:45:31,974 --> 00:45:35,444
were kind of cobbling together mocking
solutions out of open source tools.

1017
00:45:36,164 --> 00:45:37,424
just a lot of toil involved.

1018
00:45:37,454 --> 00:45:39,944
You know, that if you're, you know,
if you're in a, a microservices

1019
00:45:39,944 --> 00:45:43,214
environment and you've got 500 or a
thousand services in your organization

1020
00:45:43,454 --> 00:45:46,634
and they're constantly changing
and everybody, everybody who calls

1021
00:45:46,634 --> 00:45:49,004
'em has built their own set of
mocks for everyone else's service.

1022
00:45:49,004 --> 00:45:49,574
You know, you've got

1023
00:45:49,789 --> 00:45:50,009
Hmm.

1024
00:45:50,154 --> 00:45:54,644
Sort of combinatorial explosion of mocks
being created and maintained, and and

1025
00:45:54,644 --> 00:45:58,154
it's just a lot of toil, you know, to, to
maintain them, to make sure that you are,

1026
00:45:58,154 --> 00:45:59,684
you are testing against mocks that are.

1027
00:45:59,954 --> 00:46:02,014
Are true to, the real thing
that you're gonna call.

1028
00:46:02,444 --> 00:46:04,684
PE people, , quite rightly
don't really wanna do it.

1029
00:46:04,684 --> 00:46:07,564
They wanna focus on their actual job
and they, they want the, the amount

1030
00:46:07,564 --> 00:46:09,964
of effort they're devoting to that
once they've got over the initial

1031
00:46:09,964 --> 00:46:12,544
sort of novelty of like, Hey, this
is this fun, new thing to play with.

1032
00:46:12,864 --> 00:46:12,944
Hmm.

1033
00:46:12,994 --> 00:46:15,784
And it just becomes, becomes a, a, you
know, another maintenance headache.

1034
00:46:15,814 --> 00:46:19,804
So I, I think it can really genuinely
reduce you know, reduce that toil.

1035
00:46:20,204 --> 00:46:22,454
So I'll give you a few specific
examples of things that we're,

1036
00:46:22,454 --> 00:46:24,734
we're working on and that we're
experimenting with at the moment.

1037
00:46:24,734 --> 00:46:27,644
So, so one of them is so like I
say this, this, this big problem

1038
00:46:27,644 --> 00:46:31,814
of drift, you know, where real APIs
change mocks don't get updated.

1039
00:46:32,119 --> 00:46:34,739
Tests are being run against
you know, out date mocks and,

1040
00:46:34,739 --> 00:46:35,999
and they become untrustworthy.

1041
00:46:35,999 --> 00:46:38,789
You get false positives and then
failures happening further down

1042
00:46:38,789 --> 00:46:39,809
the line, which people don't like.

1043
00:46:40,199 --> 00:46:43,949
So ideally you want to be able to, to
automate the, you know, the sort of

1044
00:46:43,949 --> 00:46:47,762
detection and, and remediation of those
mocks so that you know, those, you know,

1045
00:46:47,767 --> 00:46:51,539
everything stays aligned and, and you can,
you know, the, the, your, your level of

1046
00:46:51,539 --> 00:46:53,819
trust is maintained in your, your tests.

1047
00:46:55,404 --> 00:46:59,624
So the, you know, one thing a,
a AI is good at is, is is is

1048
00:46:59,624 --> 00:47:02,594
kind of, you know, poking around
and experimenting with things.

1049
00:47:02,624 --> 00:47:06,964
And and I I, I find it particularly
good when you pair it with, sort of

1050
00:47:06,964 --> 00:47:10,534
non-AI techniques for, for, you know,
validating things, for instance.

1051
00:47:10,639 --> 00:47:11,059
Okay.

1052
00:47:11,464 --> 00:47:15,514
AI plus Open API actually is a, is a, a
really, a really powerful combination.

1053
00:47:16,204 --> 00:47:19,174
So we, we have a, we also have a,
a validator built into the product.

1054
00:47:19,594 --> 00:47:25,164
So what you can do is, can, you can via
our MCP tool, you can say, say to an ai,

1055
00:47:25,164 --> 00:47:29,564
go and pull the latest open API for this,
you know, this API that we're mocking.

1056
00:47:30,064 --> 00:47:30,184
Hmm.

1057
00:47:30,464 --> 00:47:33,614
and now I want you to, you know,
so, so import it into yox so that

1058
00:47:33,614 --> 00:47:37,184
we, we've got the latest, the latest
open API now far off a bunch of test

1059
00:47:37,184 --> 00:47:39,134
requests to, to everything in there.

1060
00:47:39,494 --> 00:47:42,794
And if there are any validation errors,
go and update the stubs in a way that

1061
00:47:42,794 --> 00:47:45,314
that corrects for them, and then give
me a report on what you've changed.

1062
00:47:45,749 --> 00:47:46,169
Okay.

1063
00:47:46,169 --> 00:47:50,399
Yeah, so kinda using it for fuzz
testing on somebody else's API based

1064
00:47:50,399 --> 00:47:54,204
on their, their open API definition
You fuzz, test the heck out of that

1065
00:47:54,204 --> 00:47:55,524
and then see what's, what's broke.

1066
00:47:56,114 --> 00:47:56,404
Yeah.

1067
00:47:56,594 --> 00:47:58,664
well, it's, it's kind of, it's
kind of this closed feedback loop.

1068
00:47:58,664 --> 00:48:01,424
I mean, obviously the, you know, AI's
hallucinate a lot and if you just

1069
00:48:01,424 --> 00:48:04,844
say to them, here's a, here's a new
open a p, I go, go and fix my stuff.

1070
00:48:05,144 --> 00:48:07,814
Sometimes it'll get it right and
sometimes it'll get it massively wrong.

1071
00:48:07,814 --> 00:48:07,904
So

1072
00:48:08,039 --> 00:48:08,519
Yeah.

1073
00:48:08,864 --> 00:48:10,994
losing a lot of the,
the productivity benefit

1074
00:48:11,044 --> 00:48:13,924
you are asking it to hallucinate though,
in that instance you're saying like,

1075
00:48:13,924 --> 00:48:16,774
Hey, can you go and hallucinate a
bunch of requests that may or may not

1076
00:48:16,774 --> 00:48:19,294
match at the API fire those all off.

1077
00:48:19,504 --> 00:48:20,854
That's literally what QA is.

1078
00:48:20,854 --> 00:48:24,614
That's like creatively coming up with
sometimes right, sometimes wrong,

1079
00:48:24,614 --> 00:48:27,194
sometimes absolute bullshit requests.

1080
00:48:27,344 --> 00:48:31,244
And if you're getting the AI to
be creatively stupid, that is is a

1081
00:48:31,244 --> 00:48:32,954
perfect use case for it, in my opinion.

1082
00:48:33,269 --> 00:48:33,779
Yeah.

1083
00:48:33,929 --> 00:48:35,879
Well this, I mean, the other, the
other thing I'm playing around with

1084
00:48:35,879 --> 00:48:37,469
is this idea of an API crawler.

1085
00:48:37,469 --> 00:48:41,099
I mean it, yeah, it pretty much exactly
what you described at the moment.

1086
00:48:41,099 --> 00:48:44,279
When, I mean maybe it's QA people
or devs or anybody that who

1087
00:48:44,279 --> 00:48:45,389
needs to interact with an API.

1088
00:48:45,989 --> 00:48:48,599
There's this process they often
end up following, which is sort of.

1089
00:48:48,829 --> 00:48:50,659
They'll get a little bit
of information about it.

1090
00:48:50,659 --> 00:48:53,779
You know, it might be, you know, just
some kind of, some folk knowledge from

1091
00:48:53,779 --> 00:48:55,039
somebody else in the organization.

1092
00:48:55,039 --> 00:48:58,719
It might be you know, a really
sketchy out of date swagger document.

1093
00:48:58,719 --> 00:49:01,649
It might be you know, sort of a
half-baked postman collection or a

1094
00:49:01,764 --> 00:49:01,984
Mm.

1095
00:49:02,069 --> 00:49:03,749
doc or whatever, you know,
do you know what I mean?

1096
00:49:03,749 --> 00:49:04,349
You've got some,

1097
00:49:04,479 --> 00:49:04,989
Yeah.

1098
00:49:05,609 --> 00:49:08,549
so something to start with, but, but it
doesn't really tell you the full story.

1099
00:49:08,549 --> 00:49:10,829
So you fire up an HDDP client and you.

1100
00:49:11,199 --> 00:49:13,479
You get yourself an API key and you
go in there and you start kind of

1101
00:49:13,479 --> 00:49:15,339
poking around and making requests and

1102
00:49:15,749 --> 00:49:16,029
Nice.

1103
00:49:16,389 --> 00:49:19,879
you know, via trial and error, you
build up much more full picture of

1104
00:49:19,879 --> 00:49:23,419
how that API works until you can do
whatever it is you need to do with it.

1105
00:49:23,419 --> 00:49:25,609
You know, whether it's testing it or
whether it's an integrating something

1106
00:49:25,609 --> 00:49:29,149
with it as a developer, and yeah,
that, that seems to me to be a very,

1107
00:49:29,179 --> 00:49:31,099
very automatable process as well.

1108
00:49:31,309 --> 00:49:31,399
You

1109
00:49:31,459 --> 00:49:31,579
Hmm.

1110
00:49:31,939 --> 00:49:33,829
and, and, and this is what I'm
playing around with saying, Hey,

1111
00:49:33,829 --> 00:49:34,819
here's a, here's a doc with it.

1112
00:49:34,819 --> 00:49:37,489
You know, it's got a bunch of like
URLs in it that you can go and call

1113
00:49:37,699 --> 00:49:39,229
and I'll give you an an API key.

1114
00:49:39,244 --> 00:49:40,384
Try and work your way through this.

1115
00:49:40,384 --> 00:49:44,704
API follow any links you find, you know,
try posting payloads that sort of look

1116
00:49:44,704 --> 00:49:46,234
similar to the ones you've read back.

1117
00:49:46,334 --> 00:49:46,624
Yeah.

1118
00:49:46,629 --> 00:49:47,139
Yeah.

1119
00:49:47,194 --> 00:49:49,174
picture of what's, you know,
what operations are available,

1120
00:49:49,174 --> 00:49:50,374
how you post them, and so on.

1121
00:49:50,794 --> 00:49:52,684
And I. We've sort of rigged
that up, you know, essentially

1122
00:49:52,684 --> 00:49:54,214
uses our prototyping mechanism.

1123
00:49:54,214 --> 00:49:59,144
So, so you can, as it's doing it, it will
feed things into a recording that then

1124
00:49:59,204 --> 00:50:03,404
become a, a, a, a mock from, you know,
that's generated from that exploratory

1125
00:50:03,404 --> 00:50:05,564
session that you've asked the AI to do.

1126
00:50:05,834 --> 00:50:08,234
And then that in turn
produces an open API, so

1127
00:50:08,564 --> 00:50:08,954
Okay.

1128
00:50:09,014 --> 00:50:09,704
I like that.

1129
00:50:10,034 --> 00:50:11,834
you know, do you know what I mean?

1130
00:50:11,834 --> 00:50:15,804
It's like, like sort of that, that again,
this, this kind of grunt work of of

1131
00:50:15,804 --> 00:50:17,484
just going around and poking everything.

1132
00:50:17,829 --> 00:50:18,999
I'm trying to make sense of it

1133
00:50:19,134 --> 00:50:19,944
Yeah, for sure.

1134
00:50:20,104 --> 00:50:23,074
A useful live relevant piece
of documentation and knowledge

1135
00:50:23,074 --> 00:50:25,614
about this API, you know, you're,
you're kind of asking the AI to do

1136
00:50:25,614 --> 00:50:28,584
that, and, but I say because it's
ultimately making requests that are,

1137
00:50:29,244 --> 00:50:30,474
they will either succeed or fail.

1138
00:50:30,504 --> 00:50:34,714
You know, there's no, it's not like
the the AI hallucinating is going to

1139
00:50:34,744 --> 00:50:38,254
degrade the quality of the output in
that instance because the, you know,

1140
00:50:38,254 --> 00:50:40,984
you're talking to a real API that
will, you know, will either accept your

1141
00:50:40,984 --> 00:50:44,434
request and return you something real,
or it will just say, no, that's wrong.

1142
00:50:44,434 --> 00:50:45,244
I'm not accepting that.

1143
00:50:45,404 --> 00:50:45,694
Yeah.

1144
00:50:45,694 --> 00:50:45,974
Okay.

1145
00:50:45,994 --> 00:50:46,264
you know,

1146
00:50:46,534 --> 00:50:46,654
I.

1147
00:50:46,834 --> 00:50:49,864
of those are sort of you know,
like VA validating information

1148
00:50:49,864 --> 00:50:53,344
that, that that sort of avoids,
avoids things being hallucinated.

1149
00:50:53,584 --> 00:50:54,064
So I think, I

1150
00:50:54,229 --> 00:50:54,979
That's really smart.

1151
00:50:54,994 --> 00:50:55,894
sort of thing's really useful.

1152
00:50:56,434 --> 00:50:58,414
It's a lot more useful than
like having a shared mock that

1153
00:50:58,414 --> 00:50:59,674
you give to all of your users.

1154
00:50:59,674 --> 00:51:03,454
And then like the main user profile
that comes back is like D'S nuts or like

1155
00:51:03,454 --> 00:51:06,124
Hitler, none of which is very helpful.

1156
00:51:06,364 --> 00:51:10,024
Anyway, we have gone long and that
is my fault for bringing up the topic

1157
00:51:10,024 --> 00:51:12,124
of AI at minute 47 of the recording.

1158
00:51:12,124 --> 00:51:16,174
But I would just like to thank you
for your time and for having a really

1159
00:51:16,204 --> 00:51:17,854
interesting chat with me about mocking.

1160
00:51:18,154 --> 00:51:22,054
Before you go, how can people find you and
your awesome stuff that you're working on?

1161
00:51:23,204 --> 00:51:24,794
So I'm on LinkedIn, Tom Hurst.

1162
00:51:24,844 --> 00:51:29,834
The company is yoc and you also, there's
yoc.org is the, the open source website.

1163
00:51:29,884 --> 00:51:32,524
And there's like links to Slack
community and everything else on there.

1164
00:51:32,879 --> 00:51:33,289
Lovely.

1165
00:51:33,319 --> 00:51:35,179
We'll sprinkle all those
in on the footnotes.

1166
00:51:35,509 --> 00:51:38,234
Once again, thank you very much and
cheers for listening in everybody.