Duke: All right.
Well, we almost blew all the punch lines in the green room.
So Corey, why don't you tell them what we're actually, what we for
sure are now going to talk about.
CJ: All right, dude, today we've got a very special guest with us,
friend of the show, Jace Benson.
Jace, man, it's always good to have you here.
Welcome to the show.
Jace: Hey, thanks for inviting me back.
I appreciate that.
Duke: Jace is like the friend of everyone, right?
CJ: Yes.
Duke: nobody's not friends with Jace.
CJ: Dude, like to a person, right?
Acknowledge.
Whenever I saw Jace, they're like, man, is that Jace guy awesome?
Jace: Oh, man.
I'm just another guy I put on, you 1 leg at a time like everybody else.
So we can all be as good or whoever we want to be.
Right.
CJ: Yeah, that's so magnanimous, but still no, man, you are pretty awesome.
And so thanks for taking the time to chat with us today.
Jace: no, I love it.
I love an opportunity to talk , with you fine folks.
You guys always have great insights and inspire me sometimes.
I'm like, well, I really should question how this works or question why
we're doing this or what we're doing.
What value does this add?
Duke: Speaking about what we're doing and value adding, , Jace
just released AI in a box.
Tell us about that.
Jace: Yeah, so I really like owning the things that I want, my blogs.
My content, I like having it as mine and I think that corporations
don't necessarily want to have their data controlled by other people.
Entities, And so I set off to make a self hosted AI solution for service now.
And that's what AI in a box is.
That's like it's core underpinning value that, and it
doesn't cost an arm and a leg,
CJ: so I'll, I'll leave the capitalist angle, for a bit later and figure
out why it doesn't cost arm and a leg when AI is like the number one
through 20, trending items on Google.
But yeah, , tell us a little bit about what you mean in a box.
That's the part of it that I feel like it's really catchy marketing, , but I
feel like there's something there too.
Jace: Well, it was called something else in a knowledge.
Somebody said, Oh, so it's like AI in a box.
And I said, Oh, I'm going to use that because really it's
a self contained unit that.
Does the artificial degenerative AI stuff.
It's not the same as ServiceNavs because it's obviously running on
your own hardware and it's doing its own, they call it inferences.
It's only inferences of the text you give it, but it's self contained.
It doesn't make calls outside to like open AI.
you download the models, you download the software.
And then it becomes a server that you can hit like any other integration you
have internally within your network,
CJ: Nice,
Duke: that.
I love the, in the box for that, right?
Cause this is like a box you take home and it's got your AI in it,
Jace: right?
And you can,
Duke: to you at that point.
It's like
Jace: and, you know, it's not sending stuff out, right?
You know, it's not, getting processed out of country.
I've heard some, shared infrastructure might be running cross country borders.
I don't know what the official stance on that stuff, but , , it
gets weird, but if you have control over it, you control where it is.
When it's on.
Who can access it what it does?
CJ: Yeah, and I could tell you having gone through, , the CTA training
and such, there are definitely some regulations about, like, where you
can store data depending on what country you're in and things of that
nature and the type of data, right?
And so, I don't think that that changes because you're throwing it
into a model and spreading across whatever happens inside the A.
I.
black box, right?
Like it's still data, And it still needs to correspond to those regulations.
So being able to control it, being able to control where it is and
also maybe even being able to deploy it across your organization.
But in ways that in discreetly, align with those different regulations, I
think it's also pretty cool, right?
Because you could deploy.
Several of these things, right?
You can have, say, if there are regulations where, I think Great
Britain has one of these where the data that's produced there needs to stay
there or something like that, right?
Like you can deploy your own AI in the box for Great Britain,
but then you can have another ones , for the rest of the EU, right?
Like that's something that you could do.
Jace: I don't have it set up.
So you can have it distributed.
I have a couple of them at once right now.
I'm still working on just a.
Singular AI in a box per org but that totally could be possible.
Yeah.
CJ: Look at me jumping ahead on you, man.
So put that on one on your, uh,
Duke: Uh, Bye bye.
Jace: it makes a lot of sense, right?
If you have a company that has, entities in multiple countries, If you need
to keep certain data ran against in certain places, like you'd want to
somehow devise a way to say, Oh, this incident is for a United Kingdom.
Resident it needs to be therefore be processed on United Kingdom hardware,
that's not possible today, but it could be like, you could install
it and then you could like, set up the calls to use that mid server.
CJ: Tell me what you get, I just bought AI in the box, right?
I got it set up.
I assumed that this was like a Docker container or something like that.
And, you know, and I got it up and running.
Tell me how I get that connected to service now and tell me what I get.
Duke: a
Jace: All right, so right now it's, I'm not going to name the
price because the price is going to
CJ: No, no, no, no.
You should never name pricing.
Yeah,
Jace: Right.
but right now it's a very low static price Price is just going to go up as
a fixed amount, but when you purchase it, you get an update set and you get
a zip file and install instructions to install the zip file as a server
in your choosing where you are.
you take the zip, Send it over to a server, you SSH into the server, you
extract the zip, and you run an install file, and it installs it after it asks
you a few questions, where it says give me a username and password that will
be used to connect to this machine.
You'll want to note that down so you can set it up in your instance.
And then it'll start on its own and then you can leave the server.
You go into the update site after you've imported into your instance.
There's a settings page.
You build a credential where you put in that username and password.
the mid server.
in the IP address and port to access this machine you just set up and it
defaults to the tiny dolphin model, but you can change that if you want to
use a different model or bring your own model in for the, share of AI stuff.
Once that's all set up right now, it just has a summarize button to show that it
does work, but because it's set up using like the event system under the hood.
You can call it from any back end APIs you can think of.
So I whipped together a subflow, so you can just call a subflow
and say, Hey, summarize this task and it'll summarize that task.
And you can choose to take the output of that, write it to the work notes or
send an email or whatever you want to do.
Duke: can we dive deeper into that because it's just , yes,
I got this new, AI in a box.
It's totally mine.
It's affordable, but the really important thing is what does it do for me?
Jace: thinking about what value does generative AI in general
add to ServiceNow or any tools?
And it's not necessarily that, oh, it can just summarize text
or do these things really.
What are we after when we're looking at this generative AI stuff, right?
We're looking for either feedback or direction or write.
if we're summarizing something, hopefully we're using that summary to save us
time, but not reading the whole record.
But really, maybe we should summarize and reassign like that should be like
the action that happens like only really summarize it when it's going
to get reassigned to the new group.
So, the new group can just read the summary instead of the whole spiel,
CJ: Yeah.
Duke: Maybe I'm getting ahead and putting more on your to do list
like Corey, but can it summarize more than 1 object at a time?
Jace: Yeah.
So.
Duke: incident, can I say, give it a list of incidents and have it summarize
basically what's been going on this week?
Jace: So right now, the only, , functionality that it has is just, I'm
calling it simple summarization, which is you say, give it a specific task.
You give it a template that kind of looks like a mail script, and then you
can use a scripted notation inside of it to give it related records.
So you can't say like summarize all these incidents, but you
could write a background script.
To like generate an event against all those records and have
it summarize all 10 of them.
But I won't summarize them as a collection right now.
If you understand what I'm saying.
Duke: Yeah, it's not 1 summary is 10 summaries.
Jace: It's 10 summaries today.
But if, I need to add like a general inference, option.
So you could say like, Hey, take these things and use
this prompt and deal with it.
Right.
So you could say, Hey, here's 10 summaries, summarize these 10 incidents.
In that case, you could do it, but I like that idea.
I need to add that.
That's a very basic core, like idea that I realized I don't have.
CJ: So aside from Robert and I adding a bunch of stuff to your like to do list,
what I really like about this concept, is the trust.
With AI, there's a lot of, conversation around trust.
like we started off talking about, like, where's the data going
to be, who has control of it?
Right?
Like what am I shipping my data off somewhere?
That's not gonna be used to train a model.
That's gonna be, you know, be, federated out to a bunch of different people.
And then, well, my data starts showing up in their answers, when they ask questions
of the eye with this situation, right?
Like, I think the number 1 selling point of this is that you
have the trust that your data, stays inside of your protection.
I think the number two thing for me once I trust that, you know, okay, I
can use this AI for whatever I want it, to Robert's point, it's like, all
right, so it's like, what next now, what can I have it do for me now?
So Jace, one of the things that, I hear a lot from my clients, Is that, when we go
in and looking into like open AI, right.
Like we kind of know what that does.
Tell me what like AI in a box does.
And then how we could inject some of our data into those AI models, to
make all of this work a little bit better for us versus the generic in
the sky solutions that exist currently.
Jace: When you're asking a large language model to do something.
It's just guessing the next best thing.
It's like the most expensive autocomplete system you can buy.
Right.
CJ: All right.
Jace: right.
That's what, that's what AI is.
But when you give it like 3 paragraphs of suggested best thing, best next
things to do based on the current text that's given, it's going to give you
like the best next answer, best next thing because it has it right there in
its context above the data you're doing.
CJ: Gotcha.
Right.
Jace: a user is asking for a answer to.
How should I, I'm trying to think of like a good example here.
Yeah.
Say, like, you wanted to add, , suggested resolution notes.
Right?
And you don't have it yet because the instance not solved,
or maybe you put it resolved.
But you don't know if it should list a list of steps.
Or just like, thank you, for letting us know this issue is resolved.
You may want to give it a couple examples of hey, here's
an incident that was resolved.
And here was a resolution note of user called and said,
this isn't an issue anymore.
And maybe you'll give it another one that said, have the user reboot their computer.
And then they rebooted it and they had to install a patch, For the same thing.
And if you give it that kind of context, it may use that stuff.
Those things become more likely as the next best token to be included in your
resolution notes, especially if you asked it to, hey, take the last few resolution
notes from the previous incidents and give them as an example for the current
short description and work notes given.
Um, So then it's you're having an inference against content
that makes sense for it.
But building those prompts like that is complicated and hard to talk through.
CJ: I follow you.
Duke: is that the work that you were in a box is taken out of the system doing
Jace: I want to leave some of it in the system to like build those
prompts and then send it over to the self hosted large language model and
vector databases to, get those answers.
So there's
Duke: I see.
So, the configuration of it is still inside service now.
Jace: the configurations inside ServiceNow.
I presented this a few months ago to one of the partners,
and their feedback was clear.
Jace, this is way too complicated.
And so I spent like two months before knowledge, like simplifying it down.
And it's really simple right now, but I'm re enabling things that I had that
were really complicated to set up.
CJ: Yeah.
Duke: been working on this?
Jace: Well, do you remember ScribeMonster?
Duke: Yeah.
So like, you're talking like a year, right?
Jace: When I stopped working on that, about three months later, somebody said
something on Discord and I thought, you know what, I'm going to try.
And I started down this path of a self hosted.
Largely like a self hosted solution for service now.
Duke: Did you know AI before that?
Or
Jace: I knew what I learned.
Duke: your, your masterwork in it?
Jace: I knew what I learned from building that scribe monster project,
which I recently open source.
I learned a ton from building that And I'm using a lot of the things
I learned from that going forward.
Duke: It's a ton of energy, man.
It's a lot of energy.
, CJ: did you intend on scribe monster to be like a learning project to
eventually get to AI in the box or did it just work out that way?
Jace: I built scribe monster to learn AI, that is true.
I started that project and I shipped in like a week, right.
It had a bunch of updates to it eventually.
And then I just stopped updating them because companies became
like adverse to running anything on anybody else's hardware.
Duke: Can we just talk about the appreciation for how much of his extra
energy Jace has , spent on this whole thing, , not only just scribe monster,
but then getting to the AI in the box.
And where do you find the time and energy when you've got a full time job?
We got to get Jace a sample of MagicMind if we can.
MagicMind is a channel sponsor of ours.
It is the world's first productivity drink.
It's got matcha.
It's got nootropics.
It's got all kinds of good stuff.
Sometimes I'll have it with my coffee in the morning, in the afternoons around this
time where I usually have my three o'clock crash, take my magic mind, boom, I'm back
into flow state for the rest of the day.
So Jace, thanks so much for, taking all that extra mental energy and
building these things for us.
And we'll see if we can get you a sample of magic mind.
If you're interested in magic mind, just go ahead into the description below.
You'll see that there's a link.
Use the promo code CJ and the Duke 20, give you 20 percent off your first order.
Jace: I'll definitely have to give it a try.
That sounds like something I could use.
CJ: Piggybacking off of that, I think it is a really good point that you
have used a significant amount of your mental, capacity, to enhance the
ServiceNow ecosystem as a whole, it's not lost on me how much you give back.
To the community, And as I know, it's not lost on a lot of folks out there
because they're the folks that you're giving back to, everywhere I go on
LinkedIn, I see, you know, folks, you know, mentioning your name about just
something really cool that you've built that they've gotten value out of or.
A conversation that you've had with someone or, some help
that you've given someone.
So I just wanted to, pick it back into what, what the Duke was just saying,
express that appreciation on behalf of the community, . For all the work that
you do, you know, that's number one.
and then number two, I do actually want an answer, , where do you find, where
do you find all the time and effort, man?
Jace: when I get into a phase of building, I drop a bunch of other hobbies that I do.
I don't play video games, I don't do other things, I want to see a thing finished.
I want to get it done.
So I spend the time after I could put the kids to bed until I go to bed
working on this kind of stuff, which gives me a couple hours every night,
Some nights I don't get anything done.
Other nights I do.
I wish I got up earlier so I was a more morning routine person, but I'm just not.
I'm not wired that way yet.
But that's where I find the time.
It's before bed, generally.
And then I think about when I'm not working, because it excites me because
these are the passion projects that I, the things that I care about.
Duke: if somebody was interested in AI in a box, tell us roughly how it deploys,
how long that takes and the first things that they would do with it, to make
sure that that money was well spent.
Jace: Very simple.
I've been re simplifying that over and over again, running through it
with different folks, , who I've had trying this thing now, you, you pay
a fixed flat fee, and then you're given that zip file and XML, you
download those two files, you push the.
Zip file to a server with like SCP or our sink or magic wormhole or
however, you want to bring that file to that server, you deploy it, you
install it, like I said before, and then you connect it with the instance.
Once you install the update set and run through the install page.
And right now, the immediate value is, you have a sub flow that can summarize a task.
So you could say when an incident gets reassigned, you could have a
work note written that says, Hey, summarize this for that new group.
That'd probably be the most value add today.
I have it presenting on the form when you load a form, if there's a summary that
exists, that's newer than the last update.
And that's useful for presentations,
, Duke: one thing I'm really curious about is when you say it summarizes a task.
. what all is it summarizing about a task and does it do as good a job
on custom task types as out of box?
Jace: Right.
So I made it.
So the update set includes a table, which consists of a.
The table field, and then it consists of a record template where you say, this is
the template I'm going to use to present the data to the large linked model.
And then you're given a few other fields to build out the few shot.
You can put in like a record.
It'll build out what that data would look like.
And then you can modify that data.
So if you wanted to like, I don't know, scrub something or add some other thing
to the example data, you could do that for the two records that it gets included.
And you can use any custom fields that you'd want within that, just like you
could a mail script and a notification,
Duke: One use case I see you can go right away is, in projects,
there's the project status report.
Right.
CJ: Oh my God.
Duke: And also projects and demand just have slews of.
big text fields
CJ: Yep.
Duke: demands, especially I mean, for crying out loud at vivid charts, we
built something that we didn't like build a summarizer, but we built a report
to present as a slideshow of demands.
imagine going to the demand and just seeing the summary rather than the essay
somebody's written in every single field.
Jace: right?
Or that on a change.
I mean, I cannot believe how much data is in a change,
Duke: Oh, yeah.
Or just, or just like, or the projects
CJ: hold on, hold on, dude, dude, like change, right?
think about the people who approved changes, typically management, often
upper management in some places, right?
Like they don't have time to read all of that crap.
That's on the change record.
Right.
But you need all the crap that's on a change record.
that approval record should have a summary, right?
Duke: bingo, man.
Yeah.
Jace: I know the folks on listening to this can't see, but can I
share a visual with you guys and you guys can describe it?
Duke: you why.
Well, I will, I will get you on the YouTube channel
and we will get you a video.
Jace: Well, here, I'll, I'll,
Duke: I don't know, I don't want to be
Jace: I'll, I'll throw this thing up and we can, talk about it and cause then,
Duke: like an episode of some cooking show where they're just eating in front of
you and you're like, damn, it's so good.
I wish you guys were here.
Jace: Can you guys, can you guys see this AI summarizers table?
CJ: Yep.
I just put it on the stage.
Duke: Yeah,
Jace: Very, very simple table, right?
So let's just take a look at change requests just because it has a little
complicated caveat, So in here you define the prompt of what it's going to do.
You define the record template.
Duke: So the record template is just, it looks like a bunch
of labels and field names.
Jace: yep, so it literally just reads it out like current dot
when it does the evaluation, It just reads out those values.
It gets the display value, which is critical.
But here, I'm going to make a change to this just so it triggers
a change to everything else.
so you can see it updated this record a data and you can
see it builds out this with
Duke: It's just like a sample record, right?
Jace: well, this is an actual record in the instance.
Right?
So it read change 56.
it went ahead and got all of the related approvals.
CJ: dude, dude, okay,
Duke: Yeah, I'm with you.
Jace: But it uses this as the context when it makes that call
to summarize another change.
CJ: There are people at home, dude, there are going to be people at home
listening to this like, what are they talking about, but listen, man,
like the seasonings Jace, oh my god.
Duke: Okay.
So basically like, can it collect data about sub records too?
Yes.
Jace: but I realized, oh, I probably should include , The tasks has like,
well, I'm going to want to do this for other tables, like approvals for changes.
And so I added this syntax.
but effectively anything that's in this double bracket is evaluated.
And if you prefix it with related, it'll do a query against
this table with this query.
And these square brackets are different because it is Evaluating
Duke: management.
It's a spaceman stuff to me, but
Jace: but it, but it reads the current CIS ID.
It says, Hey, give me all the approvals
Duke: right.
Jace: to like with this CIS ID and then return these two fields.
Duke: So, Even if you're not a pro dev, if somebody just explained, listen,
you just put these symbols around it and it's going to work, like you
could totally get by it reminds me a little bit about the squiggly braces,
dollar sign stuff with notifications.
Like I want to put the number in the field.
That's what it reminds me of.
But again,
Jace: right there.
Duke: think about change.
I'm telling you, SPM we'll be kick ass for this because everybody hates
writing their project status reports,
Jace: Yeah.
If you, if you'd like,
Duke: stats,
Jace: if you'd like, I'd like to build an SPM summarizer for you and
project just so we can walk through it.
Not right now, but if
Duke: Yeah, absolutely.
Let's let's do it on video, but
Jace: yeah,
I would love to.
Duke: you can almost get it to just full on automate the project status reports.
Jace: Right.
Okay.
CJ: that summary of , what's going on in real time.
All just condensed into.
The relevant details, some people don't need to know like what port
and switch that you're moving the cord from, they don't care,
we're getting rid of that stuff.
We're bringing all the stuff that matters, put it in front of people and
like a concise, this is what we're doing.
These are the people who have already, weighed in on this, you
know, here's your decision maker pick.
, dude, you have no idea, how much I am seeing
Duke: first
CJ: the analysis paralysis, of, working with the ServiceNow platform.
so much data.
so many different interfaces.
The data is not nearly concise enough.
It's not presented in a way to make it easy for folks who don't
have a lot of time to engage with the system to make good decisions.
All of this stuff.
I'm seeing this thing that this AI in the box, that you built answer
all of those questions for me.
I got my credit card out, dude.
Jace: Well, it's, money well spent, right?
The pricing model is kind of different than a.
Regular pricing model we've seen, cause I'm, I'm tired of the
sasses, just how they're priced.
It's so hard.
Did you guys hear about the new Adobe, like Congress
Duke: insane.
That's
Jace: Adobe?
Duke: yes, I did.
And it's.
CJ: Catch me up.
Duke: from what, from my reading of it, Adobe's new terms and
services is basically you give them an unrestricted license to anything
you create, they can reproduce it and use it in any way they want.
Jace: Oh, that's not what I'm talking about.
Duke: Oh, really?
Jace: I mean, that, that's important too, right?
But , the way that they charge for it, right.
They charge every month for it, but you buy it annually.
So you pay 700, 60 bucks a month.
But if you cancel mid contract, you have to still pay half
of the remaining contract,
Duke: That's cheesy.
Jace: right?
But like, my whole point is these software as a service solutions,
, they're fantastic to get paid from, but they suck as a consumer as a
company needs to pay for these things.
So there's this company I follow called 37 signals.
They're the guys behind Ruby on rails.
and they started this thing called once.
com and they're like, there's something broken in the system and
it's how we sell some software.
Not everything needs to be a SAS.
Let's sell some things and let people own them where they don't have to
keep paying a recurring fee every day.
And so they came up with Campfire as a competition to like Slack and
Microsoft teams, As a chat thing.
Cause like, it doesn't necessarily need to be up all the time.
It will be up all the time, but it doesn't need to be persistently
everywhere all the time.
Like ServiceNow makes sense as a SaaS, But does the generative
AI functionality make sense?
And so it's like a recurring fee that you pay every month for on top of service.
Now, I don't know, I think you should be able to own that stuff.
And so, because of that, I'm selling this at a fixed price.
You pay once you get access to it forever because it's your code at
that point, and I'm going to support it until I get the version 2.
0
Duke: You might want to consider Camtasia's model, which is super awesome.
you buy whatever version you buy
CJ: get this
Duke: and then it's not a monthly fee.
It's 200 bucks or something for Camtasia.
But then they keep on developing it and putting out new versions and
then you buy the next version, right?
Just.
Jace: Right.
CJ: Right.
Second.
I am
Jace: set for version one and then when I get the version two, if I want to add more
features, I'll sell that as version two.
CJ: And then you'll probably do something like a support contract,
because since you're going to be selling this enterprises where they can, opt
into like upgrades and, , certain hours of support, they get you on
the phone and that sort of thing.
Right.
Jace: That's not on the website, but that's planned to be there.
So you pay for a bucket of hours or you pay per hour for support and then also
workshops to help people understand like how this works and how ANN gen works.
CJ: Yeah.
So this is a labor, not, not just a labor of love from the , the standpoint
of doing a technology, but also a philosophical perspective on how we're
selling software and how you feel like it should be done in a different way,
Jace: Yeah, not everything needs to be so difficult to buy I struggle with
the value when everything is a SAS,
CJ: now, for sure.
Duke: It's a great capstone for the whole show.
CJ: Absolutely.
Jace, man, this has been great.
incredibly informative.
I learned so much today and I'm super, super stoked for this.
Duke: The click point for me was once you figure out that the related records
can be part of the summary, there's so many modules that are like that.
I mean, GRC, like, imagine again, summarizing all your control tests,
Jace: right.
Duke: of them, thousands of them.
CJ: Oh my God, CMDB, CSDM, right?
Like summarizing all of those relationships and giving me like a
quick overview of what happens if this goes down, like, can you build, can
Jace: I mean,
CJ: summarize impact management?
Like what happens if this switch goes down?
Duke: we go.
Another 40 minutes.
Jace: Well, so like, you have to realize it's only going to know
to do what you've told it to do.
Because it's, it's like a spell checker.
Right?
So, right now, if you were to , include like a CMDB summarizer to say like,
say you added UI action that said
give you like an impact analysis if something went down and then you pick
a CI in its chain of records, you'd want to give it examples of how that
actually would work with real things.
So it could infer, how the response should look and what it's looking for.
like I said, it's just guessing,
CJ: I got you.
So the answer is yes, but I'll take it.
Jace: Right.
It might get you close, It might give you a starting point.
Like, Oh yeah, this, actually would have this problem or the problem
would happen here, but maybe it would still be available on this port and
not that port, you know what I mean?
Like there may be redundancies.
It may not like suggest.
Cause it's simple.
Duke: The thing is, on this show, we've, come up with, 3 or 4 just.
Fantastic good use cases for this and that's with , a little less than 40
minutes of thinking so we are going to get you the links to get in contact
with chase if you want to see AI in a box and Man, thanks for watching
and we will see you on the next one.
Jace: Thanks again for having me.
I really appreciate this time of yours, , Robert and CJ.
Duke: You're welcome
CJ: Thanks Jane's.