Subscribe
Share
Share
Embed
In this episode, Michael shares details from a major internal platform shift at work, including the decision to completely remove an underused public JSON API and rebuild integrations around real customer needs instead of hypothetical use cases. The conversation dives deep into Laravel Passport, Sanctum, OAuth flows, request authorisation, and some tricky edge cases around testing authenticated APIs.
Jake then broadens the conversation into AI infrastructure, local model hosting, security implications of autonomous AI systems, NVIDIA hardware demand, and the future potential of photonic processors as a solution to the growing power and cooling bottlenecks facing AI workloads.
Show links
Jake then broadens the conversation into AI infrastructure, local model hosting, security implications of autonomous AI systems, NVIDIA hardware demand, and the future potential of photonic processors as a solution to the growing power and cooling bottlenecks facing AI workloads.
Show links
Creators and Guests
What is North Meets South Web Podcast?
Jake Bennett and Michael Dyrynda conquer a 14.5 hour time difference to talk about life as web developers
Hey, I'm Michael Dyrynda.
And I'm Jake Bennett
And welcome to episode 192 of the North
Meet South web podcast.
All right, Michael, last time we were on
the air, I spent most of the time talking,
and so it is now your turn to bring to the
table
all the amazing goodness that you are
working on or all the problems that you
have. And if we have time
at the end, maybe I could throw something
in there.
But today is your show, my friend.
Well, actually, you know what? It wasn't
actually me. Well, I suppose it was me
sort of talking a lot of the time. It was
David Hemphill was on, right?
-Right.
-Time before that, though, it was almost
all me talking. So anyway.
Yeah, how are things going for you, dude?
What, what's, what's going on with work
at, at your place?
It's good. Um, work's, work's been busy
doing, doing lots of, uh, different
things, um,
just
building new APIs 'cause we, we, we
purged, we, we destroyed, we deleted-
-Ah
-... off the face of the earth, the old
-Jason API a few months ago.
-Really?
-And now-
-Okay. What's the reason for that?
-Yeah, well-
-What, what was... Is this-
-I mean, the main-
-... the internal API that all of you guys
use?
-Yes. Yeah.
-Okay.
I mean, when, when it was built, the idea
was that everything would be exposed over
a public API that we would be the primary
consumer of with our front end, with our,
um,
-Vue SPA on the front end. But then-
-Right
... we would have people that would
consume that API for-
-Ah
-... their own purposes as well.
-Okay. Did you-
-But because of the-
... advertise this? Was it made-- Was--
Did you have, like, public documentation,
or was it just people figured it out and
were like-
Yeah, there was public documentation for
it. Yeah, yeah, we had docs for it, but
the, the kinds of customers that we had
were not really of a size or scale
bec-because they're, they're brokers,
right? Asset finance brokers or mortgage
brokers. They're not really
companies that have
-in-house dev teams and people that are-
-Not gonna be technical
-... gonna be building these integrations-
-Yeah
... and things like that. And so
there was opportunity to use things like
Zapier to, to push from, like, their CRMs
via Z-Zapier into our APIs and things like
that, but it just never really
eventuated. And as we started to rebuild
our platform,
it became more and more clear that the
benefit of using our platform was not
in the API that
-was never used anyway. It was more from-
-Yeah
... the logic and the business rules and
everything that tied it all together. You
know, you could create a person resource
or a business resource or a, you know,
whatever,
but it was the rules around what that
meant and how they came together and, and
the, you know, the roles that you would
assign to people and their relation to the
application that are attached to and
things like that. And that,
that core business logic lived in our
application. It lived in the front end. It
was, you know, it was not...
And so
over time, you know, I said three years
ago, four years ago, "We don't need this
API. No. Like, who, who is the, the
audience for this?" And it was, you know-
-Yeah
-... it was always a case of, like, someone
build it, and they will come, and, and
that never eventuated. But-
Yeah
... we, we got rid of that a few months
ago now, and it's, and it's gotten to the
point where, okay, now we can look at what
we do actually need to expose. And as
we've grown over the last few years, and,
and we're, we're working with,
um, larger businesses, and they've got
more capability on their side to
integrate, what, what do we actually need
to provide them, and where do we draw the
line around, you know,
they're just sending us lead data, and
then they're coming in, and they're using
our platform to fill that in. They're
building out,
um, you know,
org chart is probably the most simplistic
way of doing it, but it's like, uh,
company structures. So, you know, a
company that has, uh, a director, that has
shareholders, whatever, you know, all
this stuff that we can kind of map out.
And so, you know, use the platform to, to
do that kind of stuff rather than,
um,
APIs. So we've kind of now with the
experience of knowing who those consumers
are and what they actually want to do and
be able to do, we can now build that kind
of functionality out. So that's, that's
where we are now in terms of,
um, you know, building, building all that
functionality out.
So that's, that's what I've been working
on, um, you know, using Passport and, and
doing some, like,
some interesting things, I think, that are
kind of just the way that
our
users, um, need to be able to,
to interact with the platform. So we don't
really have the notion of a,
a request user. Like, a user is not making
a request. It is a-
-Okay
-... it is a Passport client that is making
a request, and that client has access to
one or more groups, and then it has access
to one or more users in each of those
groups. So it's kind of like the client is
then granted
access to one or more groups,
-and groups like just group-
-Do you have a model for client?
...
like businesses. We've got a-- Yeah, it's
just the Passport model. And then so we've
-introduced another one-
-What-
-... that is like a pivot-
-Yeah
-... table that says, like, this client-
-Wait, wait. Sorry
-... has access to-
-So, like, client, client is the Passport
model, or client is just a model that
represents, like, what, who is the client
-in your domain? Like, g- in the plan-
-Uh, so yeah, Passport-
-... who's the client?
-It's a Passport client. Yeah.
Oh, it's just like Passport client is the
name of it?
The term-- Yeah. So we're talking about,
like, we go-
-Gotcha. So you just-
-... we go and create a Passport client.
Yeah. And then that Passport client just
has, like, um,
tokens?
-Like it has-
-Yep
... personal access tokens? Because
Passport-
-Yeah, so it's got a client ID-
-That's kind of how it works
... and secret. Yeah. So Passport manages-
-Yep
-... all of that side of things.
Um, we don't,
we don't allow our customers to create
their own clients. We will create them for
them, and then we'll give them the ID and
secret, you know, the credentials. And
then we will link up, okay, this Passport
client belongs to this group of users. And
so you have access to one or more groups,
and then you will have access to all of
the users and all of the resources within
-those groups kind of thing.And so-
-Can you, um... I, I don't wanna interrupt
with that, but I, I am curious if we could
loop back to it at some point. Passport
-versus Sanctum. Like, if there was an in-
-Mm-hmm
... if the, you guys had discussions
around that, or
any interesting, like, caveats or
trade-offs between those two solutions.
-Um-
-Um,
mostly just because Passport was already
in the platform, like, we were already
using it for things.
Um,
I think we are using Sanctum for our first
party functionality.
Ah. Yeah, that would make sense.
Yeah. So all of our first party SPA is
built,
um, built on top of Sanctum.
Uh, but all of, all of the API stuff is
Passport, which I think is,
I think that is the delineation. That
Passport is-
-Okay
-... for more external stuff and Sanctum is
more for your first party applications. I
mean-
-Interesting
-... I could be wrong
-I wouldn't-
-But that's-
I didn't know that that was the
delineation. Yeah, I mean, it's, I think
there is, like, a Passport or Sanctum.
"Before we get started, we wish to
determine if you would be better served by
Passport or Sanctum.
If your application absolutely needs to
support OAuth 2, you should use Laravel
Passport. However, if you're attempting to
authenticate a single page app, mobile
application, or issue API tokens, you
should use Laravel Sanctum."
-Hmm.
-Laravel Sanctum-
-And I think we've got like this-
-... does support OAuth 2, but it's a much
simpler API authentication development
experience.
-Mm-hmm.
-Which has been my experience. Yeah, like
OAuth 2 is just,
I don't know, there's a lot of stuff that
integrates it with it now, like, you know,
to the point you're making, like,
you know, Zapier, all, all those things.
They, they have full OAuth 2 support, so
it's not, it's not like a big lift on
their side, you know what I mean?
-Yeah.
-It's probably very familiar to some of
those things. Whereas, like Sanctum, it's
just like you have to go and you have to
issue an API token and all that good
stuff. So-
-Mm-hmm
-... um, yeah, I hate it, I hate OAuth 2. I
-mean, I, that's a little bit-
-Yeah
... of a strong language, right, sort of
thing. I just, it's, it's always been a
pain in the neck for me when I have to set
up a new application that has to
-authenticate with OAuth 2 versus-
-Mm-hmm
... uh, using Sanctum. 'Cause Sanctum, you
just grab, you, you grab an, a single API
token,
and it's good. For forever, right? For as
long as you need it.
-Right. Yep, yep.
-Whereas with OAuth 2, it's like it
expires, you have to issue a refresh
token, and then the refresh token-
-Right
-... has to be used to fetch a new token,
and then you get a new re- refresh token,
all that stuff. Um-
-Mm-hmm
-... yeah, so, eh, I don't know. I don't
know.
Yeah.
So, yeah, because, because the client will
have access to resources
based on, on that pivot table,
um,
it means that
we don't have, like, the authenticated
user is not being as attached to these
-things. The authenticated user has-
-Yeah, that makes sense
-... access to groups and users.
-Mm-hmm.
And so we, we require that you pass these
via headers. So you pass the group ID as a
header, you would pass the user ID as a
header, and then as part of the, the
request authorization, we go and check,
"Hey, do you have these fields? Does this
request require both a group and a user,
or, or is it a group only thing?" Um,
and then based on the request, which we,
which we set on the request, the form
request object, which we use for all of
our validation authorization stuff,
it will then figure out, "Do I need one or
both?"
It will then go and look up that user, and
it will set that as the, as the user on
the, on the API guard,
-so that-
-Mm-hmm, mm-hmm
... you know, we can use request user
throughout the process-
-Yeah
-... and things like that.
Um,
interesting side bit to that, we, we have
this macro
defined on the request class called
authenticated user,
and that mostly exists to satisfy Stan,
like static analysis.
-Mm-hmm, mm-hmm.
-Because
the,
um,
the request user will, doesn't actually
return a user, it returns an,
an implementation of authenticatable, but
it can also return-
-Yeah
-... null.
And so you, you find yourself, you know,
depending on what level of LaraStan,
PHPStan you're using, you've gotta go and,
you know, assign a variable, and then
attach a doc block.
-You know, a comment to it that says-
-Which is lots of operators everywhere
-downstream
-... that, like, this is a user. Yeah.
-Yeah.
-Yep. So we've just created a, a request
macro that basically the request macro
says authenticated, the, the method
authenticated user will always
return a user.
-'Cause we know that by-
-Yeah
... the time we get to that point, we've
already gone through the request, the, the
auth middleware that says y- there's a
user here, so we know that it's going to
exist. We don't have to do those null
checks everywhere and things like that.
-Nice.
-And whatever else. So,
um, that's been very handy. I did, did run
into an issue today, actually, where
we're using that authenticated user. And
I, I've actually... Let's,
don't worry about all of this work stuff.
This is boring, um, in the context of, uh,
Laravel, and, and for our listeners, I
think it's probably more useful to talk
about, like, some of the interesting
things that I've run into very recently,
actually,
is the way that Passport handles
authentication and how it kind of fakes
all of this stuff in CI-
-Hmm
-... or in your test environment.
-Okay.
-So when you use, like, Passport acting as,
or Passport acting as client, it does
some munging of the-
-Which-
-... request life cycle, which can bite
you.
Yeah, can I, can I quick inter- interrupt?
So, like, when you're talking about that,
you're saying, like, in your testing, in
your CI,
where you do Passport acting as, which is
basically just a nice little helper
function to say, like, "Okay, in the case
that you authenticated using Passport as
this user," like, assume that's the case,
now start this request. And you can do the
same with Sanctum, but there's just
little helpers that are available for
Passport, whatever. And so you're saying
that's all doing fake stuff behind the
scenes.
Right. Yeah. And it, and it, and it's not
just, like, setting a user. It's going
and, and scaffolding a whole bunch of
stuff out. So if you say,
um, that this route is being accessed by
this client, so, like, this OAuth client,
it'll go and set some things up there. If
you say it's using this user, it'll go and
set a user on the guard. Um, if you say
that, that this route requires specific
scopes, like if you need a user's colon
read-
-Yeah, right
-... or users.read or whatever, it will go-
-Mm-hmm
-... and it will set the, the, um, the
Passport resource server to have those
things already in there. So we've run into
issues in the past where I have
forgottenTo,
to register the passport token. So there's
in a service provider-
-Mm-hmm
-... you put like passport.tokens can, and
then you pass it array, an array of
tokens.
Now, I've forgotten to do that in the
past,
saying that, you know, "Here's a user's
.read token." So all of your tests are
passing, you, you put tests in there to
make sure that like, "Hey, if I don't pass
this scope that this will fail. If I do
pass the scope, it will let me through,"
-and all of that.
-Right. Sure.
All the tests are there and it's working.
You get to production because you haven't
actually registered the token.
Um, the, uh,
best I can tell, and I'm not saying this
with 100% certainty because I haven't,
haven't looked into it too closely as yet,
but the best I can tell, the only place
that it checks that you've got a valid
scope is on the, the token endpoint where
you are actually requesting a token for a
scope.
So if I was to,
and this is where we end up running into
it, is when the consumer of the API then
goes to, you know,
myapi.com/oauth/token,
says, "I want a, a client credentials
grant type. Here's my client ID, here's my
secret, and I want this user's .read
scope."
If I have not put that user's .read scope
into the tokens can,
um,
method
-in the service provider somewhere-
-Okay. Yeah
... it will come back with an error saying
that this is an invalid scope, you know,
and you can't do it.
-Mm.
-Um, but n- none of that happens in, in
your test because you're not, you're not
doing that process. Um,
-so we've now gone and dynamically-
-Interesting
... registered all of these scopes, right?
-Yeah.
-So we've put all of the scopes inside of
an enum.
-And so-
-So, yes, exactly. Because-
Now we just do like, in inside that method
we just do like dot, dot, dot
scopes, colon, colon, cases,
and then we just loop through it. You
know? For each of these, we'll map it and
we'll return all of those values. So
here's the, the value, and here's the
description from that enum, and that we,
we don't have to worry about it anymore,
-a- about forgetting-
-Right
... to do that. 'Cause,
you know, I went and wrote a test, and as
I was writing this test that then returned
all of the enums to make sure that we
registered, I was like, "Well, actually, I
don't need to write a test for this
behavior.
I can just use this same loop that I've
written that,
you know, is testing that the enums have
been registered to just-
-And just register them
-... register them."
-Yeah.
-Yeah.
-Yeah, exactly.
-And, and so like the only, like they all
need to be registered. If they're in that
scope enum, it's because we need them for,
you know, functionality, and so we put
that in there. So
just a little, little gotcha that I've run
into a couple of times now and I thought,
"Well, we're gonna go and fix this in
such a way that I don't run into it," and
then, you know, you don't for... 'Cause
how often are you adding scopes? You know,
you come back in six months and you go,
"Okay, I added it to the docs and I added
-it here-
-Yep. Yeah
... and I wrote the test for it, and we're
all good and we're in production," and
then someone's like, "Well, I, it's not
giving me the scope." And it's like, "Oh,
yeah. I see." So.
Yep. Yep. We had a similar situation with
our permissions. So,
um,
yeah, we started using enums and using
cases, and it is just the best because-
-Mm
-... you had it in one spot and it just
works everywhere.
And so, yeah, that is a clutch move. I'm
loving, loving those enums. That's, that's
pretty great.
-Pretty great.
-So just-
-That is interesting though
-... yeah, there's like
That all your tests would pass and
everything, and then nope, production
-fails. That's, that sucks.
-Think you're doing the right thing. But
-yeah.
-Yeah.
Acting as, like acting as will create a
new act, uh, access token, and it will-
-Mm-hmm
-... set the OAuth client ID, it'll set the
user ID, it'll set the scopes. It'll
attach the access token to the user,
um, and then it will go and set the user
on the auth guard. It will set, um, and
then it runs like auth should use on the
guard-
-Mm-hmm
-... and it kind of just like munges all of
that together.
-Yep, yep.
-So it's like
it's circumventing a bunch of stuff that
you would actually do in, in production
where you would, where you would go and
get a token and you would go through this
registration process and things like that.
Um, acting as client will mock a resource
server, and it will go and, you know, set
the scopes and the user ID and the client
ID and all of this kind of stuff. So,
um, yeah. We, and we ran into some like
other weird issues where if you're using a
client,
OAuth client that is, that doesn't have a
user attached to it, there's some like
stuff happening behind the scenes in
Passport that will take the client ID and
attach it to the user ID,
and then it will go and try and find that
user using the like database provider
from the database.
Um,
and, and in our staging environment, we've
still got some like old data in there
where we haven't, we haven't changed all
of the IDs over to Snowflakes. So when
we've got like client ID of five in the
OAuth clients table, it then goes, "Oh,
there's no user attached to this, so we'll
set that to the user ID as well."
-Oh, yes.
-Then it goes and tries to find the user.
-Ooh, boy.
-And it does, you know, selects out from
users where ID equals five, and then it
returns a completely different user, and
things start to go wonky because that user
doesn't have access to the... I'm like,
-"Well-
-Yeah, that's interesting
-... this seems a bit..."
-Yeah.
Yeah, so we went, we went to fix that up
yesterday, but it's just like...
And I,
I had a, a rough idea of like how to fix
it, but I sent Claude after it, and it was
-just watching-
-Yes
... Claude well actually itself going in
loops, going, "I could do this, and the
easiest fix would be this." And then he
goes, "But hang on," something, and it's
just like, okay, well, we- we'll-
Any time that you have to dive into
Passport, uh, it gets a bit... Like
Passport abstracts a lot of stuff away,
but any time you have to go diving through
there to figure something out, it gets a
bit grim. It gets even worse when you have
to start diving into like the League
OAuth package, and then you can write the
-whole day off at that point.
-Yeah. I need to look into League OAuth,
um, as like, uh, not as the server side of
it, but as like the consumer side of it.
'Cause like I said, when we have to
integrate with an OAuth endpoint, uh, it's
sort of painful, and I know there's
solutions out there. I just, it was like-
-Mm
-... one of those things where it's like,
"Oh, I can do this in like a half, like an
hour and a half or whatever. No big deal.
I can just kind of like do this myself,"
and you do it, and then,
um, you know, it freaks out.
Uh, but yeah. Anyway, I need to look at
that, uh, that League OAuth 2 stuff
because I think there's probably a better
way to do that than the way that I'm doing
it, which is like some home-rolled sort
of, sort of deal.
So anyway.Hey, um, you wanna talk about,
uh, adversary in the middle attacks real
quick?
Yeah. I saw you put that in there. Tell
us, tell us about it.
Have you ever heard of this?
-No.
-Okay.
There was a breach by a company, or of a
company called Striker,
-um, maybe a month ago.
-Mm-hmm. Mm-hmm.
And it was one of the largest
cyber attacks, like, in recent history.
They wiped 200,000 devices in,
like, 79 countries in four hours.
And so the way that that worked is they
stole credentials from an administrator,
and they went into Microsoft Intune, which
is like a, uh, MDM sort of endpoint
manager,
and they just issued a wipe all command to
every single enrolled device.
-Phones, computers-
-Wow.
-Everything.
-Really?
Just wiped them all. And it wasn't malware
or anything. It was like, it was designed
to do that. They just had the credentials
to do it. And they had no sort of, like,
um, it's called MAA, multi-administrator
approval, similar to like a pull requests,
-right? How you can't-
-Yeah
... merge a pull request without another
person taking a look at it.
-Right.
-They didn't have that turned on,
and so we have that turned on everywhere
now, multi-administrator approval. But the
way that the leading theory as to how
they stole the token is called adversary
in the middle attack. So this is really
interesting. Um,
I feel like up until this point, we've
always touted MFA as like, well, it
doesn't matter if somebody gets a link in
an email and gets phished because they
have MFA, and they can't defeat their MFA.
No big deal, right?
-Mm-hmm. Mm-hmm.
-Well,
-this can.
-Now it's actually can.
And so,
yeah, this can. And so the way that it
works is they send you to a link that is a
fake link,
but they act as a reverse proxy,
and they will send you to your real
tenant's
single sign-on login page.
And so,
um,
it's proxied though, right? But so what
it'll do is it'll ask, "Hey, what's your
username?" You put it in. They proxy it
forward. "What's your password?" They
proxy it forward. "What's your 2FA?" They
proxy it through, or you get a
notification on your phone, and then you
type it in. And they steal your token
that you're getting back, and then they
forward it back through to you so that you
never even know that anything happened,
right? Everything looks completely legit.
It looks totally fine,
but they now have the token.
And so
the only way to defeat this is using
strong MFA, like a hardware token or
passkeys,
which until recently I've been like,
"Screw passkeys. I'm never using a
passkey. They're so annoying." They're
actually not that annoying, actually, if
you have,
uh, like 1Password. They're pretty
convenient.
-Mm-hmm. Yeah, yeah.
-Pretty convenient.
They integrate with 1Password.
Super nice. Super nice. But that's the
only way to kind of defeat these things is
to use a passkey.
Otherwise, 2FA, it just kind of goes right
around it, no big deal. So, you know, if
you're... You know, obviously you have to
be compromised in the first place by
clicking on a link that is not a correct
link.
-Mm-hmm.
-But after that, everything looks
completely legit. And so it's kind of
scary because we're now having to say
like, "Okay,
MFA alone is not sufficient, especially
for our people who are at the top level of
our organization as far as the technical
side of things is concerned." And so we've
had to sort of enforce passkeys
everywhere, which has been a bit of a
thing. Um, but it feels like, man, it
feels like these attacks are coming
quicker and quicker, more and more
frequently. Seems like every time-
-Yeah
-... we're talking, we're talking about
another one. I don't know if you knew
Canonical, like Landscape-
-Oh, yeah, um-
-... Ubuntu, Landscape, Canonical.
-Ubuntu, yeah.
-Canonical, they've been under a D- a DDoS
-attack since like-
-Yeah
-... uh, like s- Friday, last Friday.
-Yeah. I, I thought it was very nice-
-Um-
-... of those attackers to say that like,
"This is gonna continue for four hours."
-Like-
-Yeah.
-... "Oh, okay. I guess-
-Yeah
... in four hours we'll be okay. Sorry,
guys." You know?
Yeah. Right. Right. And then they're like,
"Oh, no, just kidding. Four days later,
it's still kind of going on."
-Mm-hmm.
-Um, but yeah, it's kind of crazy, man. It
is just a Wild West out there right now,
it feels like. Um,
and then there's that, that CVE that came
out, that copy run something something
whatever. I don't know if you got that
notification in Forge
-that was like-
-Oh, uh, yeah, yeah, yeah
-... "Hey, by the way-
-Recent service
... there's like this CVE thing that you
need to take a look at." Yeah. So man, it
just feels like... I, I feel like half of
my time in the last month has been spent
chasing down things that are like, "Oh,
don't,
don't miss this thing. Make sure you're
taking care of this. Make sure you don't
miss out on-"
-Yeah
-... you know, this whatever. And so, um,
anyway, I don't know. It's, it's
interesting. I feel like
maybe this will be the thing that'll get
us to adopt passkeys, uh, by like-
-Mm-hmm
-... you know, largely. Um,
and so anyway, it's, it's been good to use
passkeys and kind of get familiar with
it.
Um, but n- now it's making me wonder too,
like, okay, which of my apps should I
implement passkeys in, and
is that a thing I really need to do? And
so I don't know. It'll be, it'll be
interesting. It's gonna be a whole new
requirement now, like on all these
security things that we get from our
clients. Like, you do 2FA, but do you do
passkeys? You know?
And so that might be something we need to
start looking into. I feel like Matt
Stauffer did like a deep dive on passkeys
not too long ago, didn't he?
Uh, yeah. I remember seeing it around
somewhere. Um, I mean, it's... 'Cause
it's, it's built into the, the Laravel
starter kits now as well as a, as an auth
option, so.
-Nice.
-Must be part of, uh,
-Fortify.
-Fortify?
-Mm.
-Yeah.
So anyway, I gotta look into that a little
bit. Now, thankfully for us, like our
authentication stuff is all built in
through SSO, uh, with our tenant, with our
Microsoft tenant client. So,
um,
we don't really have to worry about that.
We can just enable it in there, and it's
-all set to go. But-
-Mm-hmm
... for things that we do that are
public-facing, those, that'll be something
we'll have to take a look at, so.
It's a crazy world out there, man. It is a
crazy world.
-It certainly-
-Yep
... I mean, it's getting there more and
more now simply because,
like y- you would have to say that it's
because ofThe availability and the
prevalence of AI now.
-Agreed.
-That you can just-
-Yeah, agreed
-... send the AI off to, to do your bidding
now, where,
you know, before there was obviously
people that were very skilled
and, you know, state-backed organizations
and whatever else that are in positions to
go around hacking things. But, but now,
you know, you just set Claude off, and off
it goes and figures out, "Okay, I'm gonna
go and bust into this thing or that thing
or, or whatever else," and-
-Exactly
-... you just let it spin until it finds
something.
And, and it's just gonna happen more and
more often
and, and
faster and faster just because it's, it's
possible now, so.
Yeah. Like, I don't know if you saw
that... And, and people can use like, you
know, their own local models and stuff. I,
I sent a chat this morning, um, that we
have a couple local models, whatever, and
the token throughput was 140 million
tokens.
-Oh, this is on your-
-And-
-... on your G-
-Yeah
-... GTX, RTX thing?
-Yeah. DGX. Yep. And we've got, like, four
of them now.
Um, so these things, we're creating a
little farm over here. It's been really
cool. Um, and so we should have Andy on to
talk about this. Maybe he'll talk about
it in his podcast a little bit, but it's
been really interesting to ke- see some of
the stuff that they're working with and
working on. They're using the Gemma 4
-model now.
-Mm-hmm.
-And it's been, uh-
-Yeah
... pretty, pretty awesome. Yeah, it's
been really, really good. And so they've
got that on all the DGXs, and they're sort
of running a load balancer in front of
-them, and it's, it's been really-
-Mm
... really interesting to watch and follow
along with what he's doing. But yeah-
-Yeah
-... we should have him on some time to
-talk about all the-
-I am-
-... all the cool stuff he's working on.
-I, I did, I did play around with the Gemma
4 model on my work machine, 'cause it's
like a 64 gig M4 Max, and-
-Mm-hmm
-... it... like, it worked well enough. I
don't...
Like, obviously it's not as quick as the
frontier models using-
-Of course, yeah
-... Claude or GPT or whatever, um,
on a, on a consumer-grade device. But,
like, we're getting there, and we're
getting there pretty quickly,
so.
Um,
just... 'Cause I, I need to run, um,
shortly, but I just... I saw this thing
this morning,
uh, from Kyle Daigle, who's the COO at
GitHub.
Um, you know, there's, there's been a lot
of
downtime recently. They're struggling with
keeping-
-Yeah
-... GitHub up. It's all... They had, like,
this weird,
um, merge queue bug the other day where it
just, like-
-Oh, boy
-... went and deleted a whole bunch of
commits from people's repositories and
things like that. I think they, they said
it was something like 2,800 commits
across, you know, millions of users, which
-may or may not be accurate. But-
-Oh, gosh
... I saw this,
uh, tweet this morning come across my
timeline from Kyle saying,
uh, that, "Yep, platform activity is
surging.
There were one billion commits in 2025,
and now
it's
-275 million per week."
-Oh my gosh.
"On pace for 14 billion this year if
grwoth, growth remains linear," and in
brackets, "Spoiler, it won't."
-"GitHub Actions-
-Yeah
... has grown from 500 million minutes a
week in 2023 to one billion minutes a week
-in 2025."
-Oh my word.
"And now 2.1 billion minutes so far
this week."
-Oh my gosh.
-Uh, you know.
-And, and, you know, it's Monday, right?
-Yeah.
"So we're pushing incredibly hard on more
CPU, scaling services and strengthening
GitHub's core features. And as a fine
purveyor of handcrafted shit code for many
years, I'm not gonna weigh in on that."
So-
... we- you think, you know,
two years ago it was just us.
-Yeah.
-Humans could only write so much code,
-could only test-
-That's right
... so much code, could only open so many
PRs. Now,
even, even though... Like, y- you would
think in our tech sphere, in our tech
bubble, that everyone's using AI and it's
the end of the world and, and whatever
else. It's roughly
the same number of people that are now
using 5, 10 agents at once to open, you
know, to work on multiple features in
parallel, to, um, ship multiple PRs in
parallel to, you know, may or may not
review them. But every time you fire
a PR up out there,
and like for our CI,
it's
eight processors or five shards, you know,
running at four minutes each, plus
CI, uh, plus, um, Stan and Pint and
whatever else are running. You know, we're
talking 10 jobs per PR. If everyone's
opening up 10 PRs per day all of a sudden-
Yeah
... you can see how this gets out of
control. And so-
Oh, yeah.
Sure. Yes. Of course. Um, GitHub is
struggling, and, and, you know, there's
people out there that are like, "GitHub
was never built for agents."
-They'd be like, "We're leaving."
-It, it-
-Yeah.
-Of course, it wasn't. 'Cause when GitHub
was built 20 years ago, the notion of
agentic development was not even the, the
-twinkle in anyone's eye. Um-
-Mm-hmm
... and so
-the fact that it's still up 89.8%-
-Right
...
of the time is ridiculous. And people
talking about, you know, who's building
the next GitHub, I'm like, "How do you
solve this problem?"
-Who, who has got-
-Oh, no. It's-
-Like, we're talking Microsoft here
-... who has the resources, the developers-
Right
... the funding? Who has all of that
stuff-
-We're talking Microsoft
-... in order to be able to build it?
You know? Um-
-Yeah.
-What, what do we market... What is the
market cap? You know, how, how big a
company? They, they're a $3 trillion
-company.
-Yeah.
-Right? Now, obviously GitHub is-
-Yeah
... is probably only a tiny fraction of,
of Microsoft's portfolio,
but we're talking about a $3 trillion
company. Um,
what...
Who, who's coming in? Who's bigger than
Microsoft that's gonna tackle this
-problem?
-Yeah, who's disrupting that?
-Well, and, and that-
-Totally
... that's not to say that, like, more
money is gonna solve the problem.
You know, it, it's gonna be innovative
thinking, it's gonna be, like, looking at
this in a different way. It's gonna be
building something differently to support
this. But-$3 trillion worth of resources.
Who is going to solve
the problem that agentic development and,
like, the sheer volume... And, and it's,
you know, I say it's largely the same
number of people, but you've obviously got
the outliers, the people that, you know,
the, the people that were around the
peripheral, CEOs, COOs, CTOs, product
people, marketers that are now building
stuff that they couldn't build before.
Okay, but we're still not talking a
huge... Like, we're not,
it's not 100% growth or a 200% growth.
It's still a small percentage on top of
what was there before. And, and the volume
is still growing so much more. So it's,
um, it's the... And
the thing is compute. Everyone's fighting
for the same compute resources. Anthropic
wants all the, all the GPUs and, and the
RAM, and OpenAI wants it, and xAI wants
it, and Microsoft wants stuff, and
AWS wants stuff. We've got local providers
here that cannot get hardware to put in
their data centers. They're just out of
capacity, and they can't...
You know, normal growth, sure, they've
reached capacity, but they cannot buy more
hardware to put into their data centers
now to bring on more customers because
they just can't get access to the
hardware. So
even if you had the capital
to, to solve the pro- You, you physically
cannot solve the problem. We can't build
data centers fast enough. We can't power
them. We can't
cool them. We can't, you know, all this
stuff.
Um,
it's just an interesting, interesting
challenge that we've, you know, we've
created it for ourselves, um, but it's not
gonna be solved
quickly or easily or cheaply, for sure.
Yeah. Okay. How much time do we have left,
Michael?
I mean, we're at 33 minutes now, and I've,
I've gotta-
-Okay
-... go and look at some paint, so very
-exciting stuff.
-Okay.
We should talk about, next time we should
talk about, oh, what is it called?
Basically, it's processors that use light
rather than electrons. Um,
-can't remember what they're called.
-Quantum? Are we talking quantum?
-Nope.
-What was the question?
Not quantum.
Not a question, more of a... Well, so it's
like-
-What was the... What did you say?
-There are these processors that use phot-
I think they're called, like,
photovoltaics or something like that.
Like, photonic. No, photonic processors, I
think is what they're called.
-Okay.
-Photonic processors.
-Right.
-Have you heard of these?
No.
No. Okay.
The, the, we should talk about them next
time.
-Okay.
-The long and short, though, is I was
thinking, like, quantum is gonna be the
thing that's going to help solve all the
problems for, like, AI stuff. It's not.
Quantum actually doesn't apply very well
to,
-to this problem that we have.
-Mm-hmm.
Which is, like, matrix multiplication,
right? Running, running vectors through
these matrices multiplications and then
getting outputs, right? That's all AI is.
Um,
and
obviously, with the current set of
processors we have, there's only so... You
can only flip bits so quickly.
-Mm-hmm.
-And they also generate heat because the
electrons are colliding with actual atoms
and generating heat, and so you have to
deal with that heat, and the speed is an
issue.
And so these photonic processors actually
use light,
and it's really cool because light,
th-
because of the way it works in physics, it
actually does matrix multiplication
natively,
like,
using physics. It's not like a math
operation like it is with flipping bits on
-and off.
-Mm-hmm.
So, like, if you take a red wavelength and
a blue wavelength and you combine them,
they multiply basically to give you, like,
a purple wavelength, right?
-Mm-hmm.
-So it's just, it's like physics, the way
that it does the multiplication,
and it's also insanely fast. Like, it's
faster than the speed at which we can flip
bits.
Um,
and so these things are coming on which,
these, these photonic processors, which, I
mean, will maybe even solve some of these
problems of heat and power and speed,
right? How cool would it be within five
years
if all of this AI stuff and the NVIDIA
hype is just like you don't need it
-anymore? Like, we've got these-
-Mm-hmm
-... photonic processors that basically-
-Mm
... solved
80% of the problems,
and now we just can rip like crazy with
these, with the models. Like, it'd just go
nuts. Um,
I tell you, man, like, it doesn't seem far
off that, that it could be that way. You
know, there's all these problems that are
arising right now because of all the AI
stuff, but it's like it's still such an
early technology it feels like.
-Yeah.
-Um, and it's helping us to innovate so
quickly. I could see having solutions to
some of the problems that it's creating
right now within the next short span of
time. And then it's like, you know, all
these bottlenecks you're talking about,
maybe they become not problems anymore. I
-don't know. It's pretty interesting-
-Yeah
... stuff. But we can talk about that next
time.
Well, I mean, we, we're, we're either
gonna have to overcome or we're just gonna
have to... Well, we've, we've come too
far now to give up on it, so there's gonna
-have to be-
-Totally. I mean-
... some way around it. Someone will
figure it out.
But everybody's... Totally. Uh, they've
got to, but everybody's operating at,
-like, a loss right now, right?
-Mm.
I mean, like, they're gonna... We're not
paying Anthropic enough to actually run
these models, I don't think. You know what
I mean? Like-
-No.
-Um-
-No, we need-
-But it's, it's, it's a capture the market
sort of game right now. So
-anyway.
-Mm-hmm.
Yeah. Somebody's gotta figure it out.
You're right. And, uh, hopefully,
hopefully we do. I think we probably will.
-Yeah.
-But
this is probably a good place to stop this
one. 192.
-Mm-hmm.
-Is that right? Episode 192, folks. Find
show notes for this at
northmeetsouth.audio/192. Rate us up in
your podcatcher of choice. Five stars
would be incredible. We'd really
appreciate it. And hit us up with any
questions on X @michaelthurner,
@jacobbennett, or @northsouthaudio.
All right, folks, till next time. We'll
see ya.
Hey.