this conversation I speak with Tom Chapland of Canonical AI
now
this conversation was initially held inside of the Synthflow Academy
and was a fireside chat
so towards the end it did involve a bit of a screenshot
so if you get lost towards the end
I wanna see the outcome of that screen
showing what chemical is all about from a UI perspective
please head over to our YouTube channel and watch it there
but hope you enjoyed this conversation with Tom
okay I guess we can we just get a rock and rolling
if anyone's filtering after the fact
we can so thanks everyone for joining
I'm joined today by Tom Chaplin who's building canonical AI uh
which is a mixed part of the voice AI agents
you missed that intro before
so I guess kick things off
how you doing Tom and you wanna give us the quick one line intro
who you are and what is canonical
canonical they say
yeah you did great
you said it right uh
yeah I'm I'm Tom Chaplin
I'm the CEO and co founder of canonical
and and we give you visibility into what your voice AI agent is doing
we also give your customers who you're building voice AI agents for
visibility into what your voice AI agent is doing
and what we're finding in building this product is that um
there's some hesitation around the voice AI
your voice AI customers to expand their usage of the bogus
AIs that you build them um
but once you give them more visibility into what's happening
and they can see that the agent is performing as expected
that's what really unlocks growth
and our mission is to help voice AI developers unlock
the growth in voice AI um
by by helping their customers
feel more comfortable with putting their brand in their sales funnel
on the line with the voice AI agent
you're building them
yeah I like that
um it does seem a lot of the time it's a bit of a black box
and people can say the voice
I do certain things but can you back it up with the actual data
so this this course successful
did it go through did it completely gone etcetera um
you've been doing this for like a year
it's kinda a year and a half August 2,023 if this is correct
so you've been in the space for a little bit
is this is this always been the same product
no actually we started with something else
so I'm gonna go a little bit further back in describing my background
um I started an agriculture technology company back in 2014
and that company got into y Combinator um
it was a great company we
we were building a
sensor that help farmers figure out how much to irigate um
is based on my PhD work and is really neat data feed is
is a good company um
and in the process of building that company
I became more acquainted with nondeterministic
neural network type models
um it was uh
when like the Tensor Flow API came out
we're building cool computer vision stuff on top of our data feed
um and really liked it
and eventually I found a good home for that company
that company was called Tule and was acquired by Crop X
they're great people um
I want to start up journey again
and of course um
my co founder and I who was also the CTO of my last company um
we want to work in LLMS and initially we're building a cache in there
the idea of a cache in there is it's like a smart database
every time a query came in
um from a user asking an LLM something
we'd look in that smart database to see if
essentially the same question was answered
asked before if it was
we could return the answer from the database
rather than calling the slow
lumbering LLM um
and that LED us to reach out to a bunch of voice
AI developers to see if they'd be interested
this was about you know
that year and a half Mark ago
um
and what we found was like yeah
latency was a problem but well
actually I kind of skip to step there
the the thing I really wanna emphasize here is like
we just fell in love with the voice face
we started meeting these voice
AI developers and seeing all the cool stuff they're building
and just seeing the incredible potential for voice AI
that everyone here in this
in the session with us believes and feels
and as examples of seeing it um
and we just fell in love with the voice AI space
and we realized the cash wasn't the right thing to build
it was too slow our growth was too slow on it
I should say um
but what our friends and voice guy kept telling us was like hey
like I've got these contracts with these big potential clients
but they're only dipping their toe in the water
and I need a way to make them more comfortable with my voice AI
and I don't really blame them for not being comfortable with it
because I'm like manually listening to a few calls
I don't have any sort of visibility into what's happening
can you build something that makes it easy to show people like
here's what's happening in your data
and everything's going as expected um
so I started building that about um
I mean we shipped the dashboard in early October um
and it's been it's been really fun
it's been great ever since um
I just love being a part of the Voice Act community
and really glad to be here and
and hopefully get to meet some of you that have joined and
and get to make more friends in the space yeah
that's cool was there one demo when uh
you were kinda exploring options where you thought like
holy crap this is the space that I need to like apply my timer to
ah yeah
like what was the moment where I was like
oh voice AI is so cool
I think for me a lot of it is um
like when I first logged into Sinflow
and it must be similar for so many people
like you first log in and build your first agent as like
it's just a toy and it calls you and you're like
oh my God I can talk to it and it just
I feel like I still build
like voice AI agents just for fun on the side
like little little toy demos just like for fun
I built one that was like a baseball game for my son um
I feel like I'm an artist with a palette
in this really interesting palette that I get to work with
with voice AI and I just think it's such a neat space to be in
and also I see a lot of calls that um
people make um
voice agents make and they just
it's amazing to see how well they can work
you know like when they just nail it
the customer skeptical and they're like
let me talk to a representative
and then the voice guy just solves their problem
at the end they're thanking the machine
it's just like magical it's clearly the future yeah
some of the times when you listen back to a call um
so if you're testing an agent and I've done it a lot of times
you're tweaking a prompt
and suddenly you hear like a recording of a conversation
and you're like damn
that was actually really good
so you're tweaking things like sentence lengths and like do say this
then say that and when when you get that conversational flow right
yes yes
my music to his yeah what was that for you Tom dude
like what was the moment where you like oh
voice the eyes the space I need to be in
um so I've always been have like a
had a community going for a few years called Shiny Object Social Club
where it's just a bunch of us hanging out in a discord
and playing around with any new tech
that's interesting and then uh
I think 11 lads came along when it was like synthetic voices
and you could actually just get into say stuff
and then there's a bit of avatars coming around
and then that got applied to phone calls with things like sin flow
and battery at the time
and uh I wasn't extreme an extremely technical person
so sin flow is obviously the
that the natural fit being a more no code solution and the
the feeling it just gave
that I could now have this power to create these really powerful uh
workflows and tools and functionality was just yeah
it was just wild and then things like um
connecting it with other alternations
which we're gonna have a look at in a minute
where you can not only extract information from the phone call
but also have it trigger other workflows and suddenly
and make internal calls and outbound calls and hooking together
it's just just felt like everything is now on the table
yeah yeah
absolutely yeah
it's cool so yeah and
and it is is been kinda missing for a bit what we seen with chemical
and do you wanna give us the rundown
we're gonna do a screenshot too so you can do that whenever you want
um so basically this base if I'm right
this gives you a really easy way to see
how your agents are performing
that's correct right
yes that's the idea
gives you and your customers an easy way to see what your voice
AI assistant is doing and for you
it helps you improve your agent and for your
your client it helps you feel comfortable with what you're doing
so that they'll start pushing more call volume through it
hmm got it and have you seen have you seen that the take of it
is this meant for um
like small regencies s and B's escaping to enterprise
where where does where does it fit in
enterprises are always the slowest right
I think that um they
they clearly aren't going to um
use voice AI
until they have a lot of comfort with the fact that
the voice AI assistant is doing what you expect
so they're definitely gonna need people selling to enterprises
building voicing for enterprises
and enterprises are gonna need something like what we're building
but it's not really my my initial focus
just because it's so slow to get going with enterprises um
our customer base really is two different categories um
it's vertical SAS specialized vertical SAS
voice AI companies are focused on something like
building a voice AI assistant for um
HVAC businesses um
or they're the AI agencies um
and that's part of why I'm excited to be on this on this
um in the session with you is like
I think Sinflo does a great job with AI agencies and fitting
the needs of the AI agencies
um like with the white label product and that sort of stuff
and the AI agencies
you know the other type of our customer
and we find with both of the those two types of customers
the vertical SAS and our AI agencies
they kept coming to us saying hey
can this is cool
you built for us and it's helping us to hone our agent and refine it
but can you also build this
can you also expose it to our customers
um can you give them a login or can you
we embed what you built into our website so that um
they can see how the what
what's going on with the data
um so that's kind of the direction we're going with things now um
and making progress on that as well
I'm sorry that's kind of maybe a rambling answer
does that that's good
yeah that's really good
yeah I've got a bunch with a bunch of Ad Sheerans and head
so you getting the full white label route or is it more like semi
semi embettable sort of scenario
yeah so let me describe the current state of the world
and then also where we're going
um the current state of the world is you as the voice AI developer can
uh send us calls
um you send us recordings of the calls and a transcript
I mean go through later how to do the integration with make
and also the integration just with code
I'll points that as well um
then you can log in and see all the calls
for the different agents that you send us
in addition you can set up
you can work through with us to set up a login for your client
and your client can only see their agents
so we set it up in such a way that your client only sees
you know the agents are supposed to see
um
that's the current state of the world
then the next thing we're building is embedable components
so if you have a dashboard
like you're using the white label solution for Vincent flow
um Vincent flow
you can take
like the call flow diagram that I share with everyone shortly um
and you can bed that in the
the dashboard so you can show your client that um
as well and then finally um
this is in the works
but I'm talking to the the cofounders of Sinflow about um
doing a more native integration
that just makes this whole thing easier for
for you guys
and that there's not really the step where you have to mess around
they'll just be one click and you can you can get this
this stuff right there and sing flow
got it so I guess you able to trigger different kind of queries
depending on the outcome of the call
like in real time
or is it more of like a uploading batch and then analyze after say
like a campaign
it's a little bit both
so let me let me describe like the the main wait
so we can we can share the screen at any time
so whenever whenever you wanna add context
so you took your own we can do that I'll first describe it um
and then I'll show it and I'll summarize it again
so the the what I wanna describe is the first thing we show you is um
visualizations of what your calls are doing
so most of your calls will likely go down this happy path that
you designed um
but some calls will go off that happy path and take a turn
and those are the ones that you wanna look at
and they'll help you as the voice AI developer to improve the
the agent um
so and that requires that you log in and look at the data
and this is also what um the voice AI developers
customers also like they like being able to look in
and kind of see the different paths that the calls are going down
and get that kind of warm
trusting feeling that they can believe in this
this product and expand with it um
the other thing we do is we have uh
push notifications to slack around insights
so like right after a call ends
if it was supposed to be successful
if it was most likely to be successful
given the fact that of how long it was and what path
what call stages the call went to
but it wasn't successful we push those to you um
and make it easy for you to look at those
and see if you need to do any damage control or um
you know
you wanna make any changes to your prompts or your your integrations
um and then the last part is audio metric
so we we have metrics like um
the latency the number of interruptions and that sort of stuff
and that's really more for um
helping you surface different types of issues and dig
dig deeper into how the agents performing and also like
give reassurance to your clients like look
most of the time my latency is
you know this many milliseconds and that's within the
the human what's normal for human conversation
and things are working as expected
got it I guess
that kinda gives you the opportunity to be a bit proactive as well
so if you're already seeing
like a higher threshold of either foul calls
or ones going off the happy path
or highlight and see
then you could kind of proactively get in touch with your clients just
just to let you know
this is happening and we're on it and we're looking at fix
yeah so my last company
we helped farmers make irrigation decisions
and I think initially my first instincts
especially as a former academic that built this technology
and so close to me was like
if something went wrong I would like
I didn't want to go get the farmer's attention and bother them
and be like hey
this went wrong um
it just didn't feel right to me at the time
but then later I realized that
the key to our success was being very transparent with our customers
so we were always talking to our customers when we saw something good
and we wanted them to like notice how well this thing went
and also when something went bad
we reached out to them because it's better they hear it from us
and that they know that we're working on it
that's what prevents churn um
than if ended on their own
it's all about you know
maintaining and building that trust with your clients
and um
what what is the kind of state of monitoring and analytics tools
especially when it comes to third party
and so we do stuff in 10 minutes since low and
but I haven't checked in too recently on all the available tools
uh they
what what's the existing looking at right now
I think well
I think it
what the ecosystem looks like is indicative of the state of voice AI
right now voice AI is this emerging field
and there's some people getting a lot of traction in it
but not um
most people are still kind of in the early stages of it
so if you look at what other tools are out there for helping you
like third party tools for helping you develop your
your and understand how your assistant is doing there
mostly on the testing side
so the side before you deploy your agent into production
um it's for like
running simulations of lots of different scenarios
and making sure that um
your your agent is trying to automate that
testing that all of you are familiar with doing um
the the problem with that approach is like once you test it
it's great you can feel very confident in it
but there's no way to really simulate human behavior
like humans are the most unpredictable force in nature
and once your agent out into the production
and if you have agents in production
you know exactly what I'm talking about
like people do weird stuff um
and you're you're wouldn't expect what would happen there
and you're going to
really need to see what your agent is doing with real humans
to figure out how to improve it and um yeah
how to improve it so the
in the pre production side
there's companies out there like Vocera and Coval
um in the post production
there's us which is more of like this um
low code way of doing these things
um that
it's really amenable to working with AI agencies and your n customers
um or you can use more higher code solutions
like you can set up a whole
like data dog pipeline and build your own visualizations
uh and
and data flows through with data dog
but and
and the the problem with that is it's not really centered around voice
No. 1 it takes a lot of technical skills
and second there's a lot of problems with
voice that are different from just looking at like
traces from you know
text based lolms um
so I really think this area is kind of Mason and emerging
and we're seeing things emerge like what we're building
got it what's been
like the most common thing that people have found
after implanting your tool that they weren't aware of before
oh interesting
um
well I
think I once heard I was listening to a podcast recently of um
by C Partners and they're talking about what builds a great
what makes a great sass company
uh that really can capture a lot of the market like
like a gusto is that isn't that they do one thing really well
it's that they do a lot of little things
um and they do all of those myriad little things well
um and when I look at voice AI assistance
I can always kind of tell when I'm looking at the data
our customer sent us is like how
how much care and time the developer has put into developing
how much time it's been in production um
because there's always little things along the way that
it's just lots and lots of smooth
at rough edges that need to be smooth so
um finally I'll give examples
it's like a notorious one is um
I should actually more generalize first
there's all these workarounds you have to build
given the current state of the voice AI technology
with text to speech and speak to speech
text and lolms um
even with the audio only models and if you
you're not careful about how to do those workarounds
like how to get email addresses or um
how to be careful about like repeating people's names um
then you're gonna run into problems
um and it's really that sort of constant
careful hearing and craftsmanship that makes a great
voice AI agent I gave two examples there by mostly smoke high level
let me know if you want me to dig in more
no that's good no
I like it um
I think still very few people
I think really digging into digging into analytics
cause there's not too many doing a lot at scale
but presume that's gonna ramp up
now that they're getting more accurate
and they're getting easier to create and deploy so
so I expect that to increase soon
yeah so I'm looking at the chat
and I'm seeing that people are eager to see screen shares
and they're eager to figure out how to connect
especially Nick
I think Nick's very very keen to see a demo of screenshots
let's dive in and we can keep on
sharing my screen um
let's see
give me just a moment and any questions you go file them into
file them into Q&A we can
we can bring them up on screen
or you can even come up and ask yourself whichever
whichever you fancy
okay cool
you can sit if you wanna follow along
I'm gonna drop this in the um in the chat um
so here's the sequence of what I mean go through
I'm first gonna fairly quickly walk you through
our platform and what we're doing
and then from there
I wanna spend more time focusing on how you can integrate with
your sin flow agent with our platform
we have a free tier so it's it's easy to get started
I just put into the chat
also our blog post that really walks you through um
how to integrate with make or with code um
and that's what we're gonna cover now so so without further ado
let me go back to our dashboard
I'm on our demo account now um
and I when you first log in
you see the summary page
the summary page will show you how many calls you've uploaded
and the total duration of the calls you've uploaded
and some high level stats um
where things get interesting is when you scroll down
and you click on one of the agents that you've uploaded
and then we can dig in and look at that
that agent um
so I'm gonna scroll down here
and first gonna talk about how we process the data
you send us for each agent you initially send us excuse me
I should say it differently when you connect a new agent to us
we take the first 15 calls that you send us
and we use those first 15 calls to figure out the stages of the call
what I mean by stages is
an agent will typically go through stages like reading appointment
scheduling um
you know answering a question about a product uh
and then closing the call we figure out what those stages are
then once we have those stages every time a new call comes in
then
we can take each turn in the conversation and assign it to the stage
so when the user says hi
we can assign that to greeting when the user says
I wanna change my appointment
we can change um we can assign that to appointment scheduling
what this enables us to do is to determine
a path that each call goes down um
that all path is helpful in and of itself
but it gets really helpful when you can take that call path
and look at the call paths of all your calls for a given day or week
and really get the bigger picture of what your assistant is doing
um we do that automatically
you don't have to submit a core structure or anything
you can just get that from the course
well I'm really glad you asked that question
that was about to say is that we do it automatically
because some people have so many agents
they don't wanna go in there and edit each one
but also if I were logged into my personal account right now um
you could see a little edit button here
and you can edit the stages to better match
what you think they should be
got it okay
make sense
we do the same thing with the call outcomes
so we use the first 15 calls
and some LOL
magic to figure out what the typical outcomes are of the call
appointment not scheduled
appointment scheduled or product um
inquiry handled
then we use those outcomes
lolms are just like people in that they're more likely be accurate
if you break things down into small steps for them um
they're more likely to be able to solve a problem
when the problem is broken into small steps um
so we take those outcomes and then in another step we ask the LLM
was this a successful call given the outcome or not a successful call
um again
you can edit the outcome names in our dashboard
you can't see that here but if you log in um
you can see that uh
can you still see my screen by the way
uh I just opened your cash
okay
click on the sign up button here
um you can sign up for free and you get uh
I forget how many how many free minutes do you get
you get 4 hours of calls per month for free um
so you can click that sign up button um
and once you sign up you can click the upload calls button
we have a gooey for getting started
so you don't even have to integrate using make or um uh
code initially
you can just download calls from Sinflow and drop them in here
and that'll bring you back to what I was showing you earlier
where you have the uh
will bring you back to our dashboard
so this is it's really easy to get started and and try it out
um you just go to uh
I'll drop this in the the chat
you just go here and click sign up um
and you can try it out um
and upload calls
okay so now that we've uploaded calls I wanted to um
show you some of the things you get so this is the thing that my
the Voice AI developers
customers seem to like the most it's this call flow map um
I think it just gives people reassurance that they can see like oh
the voice the eye is doing what it's expecting
like initially greets the people and then after greeting them
some amount of calls go to objective not met
like yeah people sign out
um people hang up um
that's pretty normal um people hang up right away
but even that's something you can optimize as a voice
AI developer and work on different tricks um
but then the calls are getting you know from their partition
some are going to appointment scheduling
some are going to product enquiry handling
if we go to appointment scheduling
we can see that some go straight from appointment
scheduling to closing call
we can click on either a node
and see all the calls that went through appointment scheduling
or an edge between the nodes
and that'll bring out
a drawer where you can see the calls that went through this um
stage of um
greeting to appointment scheduling and what I'm looking at here
is a call where um it started with greeting
you can see the the stage and who was speaking is the assistant
it's a greeting then the user says
I'd like to schedule appointment
that turn that conversation turn was assigned to appointment
the appointment scheduling stage
and just gives you an easy way to kind of
look at your call volume and see like oh
I see what's happening um
everything's going as expected or something's not going as expected
this is gonna be an easy way for me to click in
and look in and see what's happening
and see what I need to change um hmm
can we pause in there yeah
that was that was really good
that was a really good walk through
so when let's say somebody's looking at this
this report
and they see there's something which has gone off the happy path
and they wanna dig in
how would you kinda recommend someone like evaluate these calls
just clicking in and read through the
read through the transcripts
and you can kinda tell where they've gone off
off path
yeah actually for that
so this first screen is really something that's like
for your quick view of understanding what calls doing
if you want to figure out how to improve your call
I actually like the call map more um
maybe that's a bias because I'm a little uh
it was the first thing we built in it
I think it's neat but um
with the call map
it's a similar thing where we're showing you the stages of the call
but what's different about the call map is we show you
will highlight in red
where most of the calls are stopping with failure
um
it makes it easier to subset to just the calls that are going poorly
the other thing you can do with the call map and
and the demo accounts not as good as an example
but you can see
you'll see something like most of your calls like 106 calls here
47 calls here 17 this is the happy path that most calls are going down
but you might find like this weird branch
and it's when you click on the branches that
that go off in an unexpected direction that you find the insights
you click on those and it will um
and if you click on the transcript part
it'll bring you to the part of the conversation
where it's going off the the rails
it will highlight in yellow
and it will give you ideas about what you need to improve
about your assistant um
we haven't
one of the things we want to do is like
start using diffusion models to turn this into like
a like a comic book panel
so it's quicker to read because reading transcripts is slow
but people can ingest that else of quicker
but we haven't built that yet yeah
we we have like a
the Ellen
Musk judge to just like evaluates the call that summarizes it
but again that's still uh
like a bit of a pain to read all this summaries
especially be doing any sort of significant cool volumes
this might be a bit like a out there question
presuming might be possible and I don't like can I guess features
etcetera but you can almost kind of extrapolate similar
this out to self healing and self recommendations through
call improvements
and maybe if somebody ships like an incomplete assistant
and maybe could use another couple of stages added on to it
you could you could probably even tell by
and having a decent bunch of calls running through your system yeah
that's exactly where we're going with this
so with I'm gonna pop over back to my dashboard
um it was a pretty scheduled comment below because I logged in
it was this thing
um I can't show it
but we have a feature where you can integrate um
with Slack
and what we're doing at Slack in sizes were showing you like
here's a call that went down a happy path
90% of your calls to go through all these stages
the happy path they complete and have a successful outcome
but here's a call that wasn't successful
but it went down the happy path
look at it um
and where we wanna go from there is to then
take those calls that should have been successful
because we've we figured out how to subset just to the calls that um
should have been uh
that are problematic rather than like manually sampling
um and we want to suggest improvements to people system prompts
um to
to help them like you know
figure out like oh
well these calls are going poorly because when the voice
AI repeats their name back
spells their name back it speaks way too fast
you gotta like prompt it to speak slower
um things like that
that we think we we wanna get to the vision that you describe Tom
where we can help kind of build
self healing systems with the human in the loop
hmm
yeah I'm wondering that
you can really pull some really good insights out of this
and instead having a data served up this way
like a visual for any decent sized company
you have to report on the effectiveness of these things
that's gonna be a really useful page to show them
yep
yeah um
so another thing I'd like to show is um
the custom metric so what a lot of people
cause is in line with what you're just talking about Tom
um you can ask custom metrics of call
so the summary is great um
but summaries
one line summaries LLM is a judge
they don't capture kind of what happens within a call
often they all think loss is over
this is a common problem I say Isaac Gloss is over
kind of the details that might happen in particular turns of the call
um and one of the ways to approach that issue is to like
look at call maps and it'll help you identify
the different stages that you need to focus on
and subset rather than random sampling
um another thing you can do is you can define custom metrics
like you may be interested in knowing um
your your voice
I should be asking in the US for consent to record the call um
and you might wanna ask like
did the assistant ask for consent to record the call
and will then for every call that comes in
will ask that question and will surface
you can click on the times it's false um
and we'll show you just the calls where that was false
you can look and be like why did the voice say I not ask for consent
you can see what the issue was and try to figure out how to improve it
um so just one more way to uh
improve your agent and it's also something that like
your customers might be interested in too
because they're like well
how can I get my voice AI assistant to sell better
um you know
how to handle this objection
did the voice AI handle the objection or something like that
yeah nice
there's a there's a couple of like
interesting quirks sometimes come up in assistant calls
one of them is sudden uh
sudden bigger bigger lacency gaps in between questions
so say if you have a script
everything's gonna read nicely
and then suddenly for some reason a question comes up
the user responds as normal
but then there's a big lag and then the user has to repeat themselves
and then maybe the assistant repeats themselves
and then it kinda gets through
is there a way to identify something like that in in your system
yeah so you're this is like
it's almost when I see this happen the most is around interruptions
where like uh
the the person the voice say
I will think the person has finished talking
it'll start talking and then the person will like wait for them
and it just like
the question ends up getting repeated and things get confusing
is that when it's like type of scenario you're talking about
or maybe darkness some sometimes it can be like a
like a rogue background sound which cuts the cuts the response out
but there is uh now and again
I don't know if this happens when you just uh
maybe listening to voice articles all day
so you're more attuned to it and then an old person won't notice
but sometimes you do find that now and again
there comes like a little bug where there's like a
nine out of the 10 questions that the assistant will use on the call
be absolutely perfect like normally it's expected
but then one of them for some random reason
uh just has like a
a double or triple the length of gap and it'd be good
this is maybe like a more minor
like fine tuning use case rather than a high level general one
but it would be good to be able to identify against photos
and I I sit when I'm doing recordings since doing videos like
cause I'm looking at the audio waves between the responses
so I can see the legacy gap on each response
sometimes they're most of the time they're even
but then you can see one
which is like double the length of the other one
uh yeah
I know what you're talking about so
um we calculate on the back end
we're calculating the latency
the for the human response and the latency for the AI response
and we're calculating
how much of the time the person is silent
when it's their turn to speak
which is indicative of confusion
and all these other audio metrics
we haven't surfaced them yet
we just have a um
there when you log in um
this is something we're shipping
hopefully today
is something where you can just download all those metrics
for your calls so you can see the latency for all your calls
but eventually
we wanna surface that and put it in what we call a rain cloud plot
where right now this is the call duration distribution
and you can see like most of the calls this frequency history
it's a frequency um distribution
um you can see most of the calls are around here
around 30 seconds um
and then the individual calls are these dots underneath um
and we wanna create a drop down here for latency
so you'd be able to see like
oh most of my calls have pretty low latency
but you could click on the ones with higher latency
and see what was going on with those
and see when they happen are they happening because of interruptions
or is there some way you can
might be able to improve the flow so that those
those late and sees are less likely happen
or like
there's a long wait and see and maybe you wanna go tell your client
like hey
this thing happened um
that's uh
but we're working on it it sometimes happens
it's okay most of the time your late and sees are over here
you know there's nothing to worry about
yeah sometimes it can be random things like uh
response length or sensitive length
or if they're doing some sort of rag over a knowledge base
which has like an awkward setup
then there's just like a bubbling legacy there too
if they're making like uh
it's easier custom action
which is like a laggy API and then they get a response back in time
um yeah
having that flag would be
would be actually awesome
especially at scale that'd be
that'd be killer
yeah I mean
what you just said speaks to the craftsmanship of building voice AI
is like you can uh
I think um
you know you can get something up and running in
in five minutes and that will get you 80% away
but the next 15% is gonna take you a lot longer and a lot of skill um
you know it's faster to get there if you have a lot of skill
but it takes you know
really understanding how these systems work and how to like
avoid these latency traps and
and other problems they can come up if that
you know if you're not designing your system well
and carefully monitoring and analyzing it
hundred percent there's a if anybody's got a new system flow
we can just cover what we're talking about a bit more is
if you're using something like custom actions
or if you have your system connected to Knowledge Base
if the user asked the question which isn't in the prompt
then it can go and look up through a document or a series of documents
Wikipedia and then bring that um
information back into the call
which may take slightly longer than a pre determined script
or another one
which you can do in sin flows and called a custom action
which is basically just like an API call
which you can either trigger to happen before the call happens
so uh
actually an example I like to use is if you wanna get the weather
so if you're doing anything with like an outside venue for instance
you wanna know the weather to determine what you recommend to the
end customer or if it's like a car service or like a concert
whatever it is
then you can bring that weather forecast into the conversation
you could probably do that before the call starts
then you avoid the mid call conversation
but if you're the mid call later see um
the way moment to cover that if you're doing it mid call
this could be something like
looking up an order ID
in a Shopify store to track where the shipment is
for instance that could be the customer gives the ID
you can look up the database to see the states of the package
and to cover that API call going through to to shopify
you will set in your prompts or in your field in sin flow
you can trigger it and say okay
I'm just gonna look up your information now
and while the this is the same
that sentence is making me a guy called the same time
so that then covers that look up hopefully that makes sense
that was a that was a great intro to um
knowledge bases and rag and tool calling uh
all all the right there very specific
thanks not being at the most advanced custom action developer
we've got you killers and simply does some amazing stuff
but that's as simple as I ha ha I can understand anyway
um story time how to integrate
yeah that's right
no that's it
I was just gonna get good and I'll be I'll be awesome okay
so one of the links I dropped in the chat was um uh
to this blog post that tells you how to connect your sin flow agent
using make um
and the place really just started it
it walks you through all the steps um
uh
but you can also download the scenario
like if you go into this blog post and scroll down
you'll find the scenario where you can just download it
and upload it into make so just backing up a moment um
make is a platform for um doing server side automations um
so it's like instead of writing the server code you can use make to um
take the the the data from a call and do something with it um
like you know send an email with it or
or whatever and we're using
you can use make also to send data to our platform um
so you log into make and the first step is to create a custom web hook
and this walks you through creating that custom web hook
and then you would then go into sin flow and go to your
your agent that you want to um uh
send the data from that the call data from that agent to our platform
and you click on deployment and rest API
and you would paste in that web hook that you got for make um
then the next step is I mean
the easiest thing is to just download the
the scenario from from our website right here
and then you upload that into um
into make and then the final changes they
all you need to do to finish the integration is you end up going into
here I'll go into my make and you have to um
it'll look like this once you upload your
your file into your scenario to make
and you have to put your API key in for canonical
and then it's it's ready to go
I mean I I blaze through that really fast
but it's that's just the bigger idea
you might be wondering where you get your API key to get your API key
you go to our website and you click on the little icon and there's um
the setup button and you can get your API key from in there
um and I don't know what maybe I went through that too fast time
should I slow down on part of that or does that makes
does that look good I think
I think make make integrations are probably a bit more like on the
on the niche technical side anyway
and it probably won't be one of the like beginner use cases for
for using make I think probably a lot of people are still at the
either just trigger a call by a make with some flow
or retrieving the details of the call by some society
to actually produce some more education materials
in this case maybe I can fire up the tutorial
yeah and our tutorial actually you might find it
people here might find it as a good way to get started with me
cause it's like a
walks you through an example that you can get up and running um
and and kind of acquaint yourself with the mix system um
we all in the chat is a monitor synth play voice yeah
and of course cool I'll let my drop this in the Academy too okay
yeah that'd be great um
and you can also we
I also linked to code on how to programmatically to our docks
excuse me so if you're if you're writing server code um
and you you want to integrate with
with our code and send data via the send the calls via the API
that's all in here
there's a tab for sin flow for hitting our sin flow and point um
so that's that's there
for people that are kind of interested in integrating with code
how granulate can you get with the make inspiration
can you
can you animate screen comprehensive when she passing Jason through
can you I guess then you can filter filter by um
if uh
there's like an unintended call which goes through depending on how
you labelling it or a failed call
and you could trigger a slight message too would that be right
yeah
one way if you pick up those dynamically created uh
call stages from the retrieval
so one way to do it I'm gonna start with
the way that kind of better maps to my way of seeing the world first
which is like just send all your calls
use this make um
scenario here to send all of your calls to our platform
and then use our platform to figure out the calls that you want to um
be alerted about and you want to surface
so you can set up the Slack integration for that um
you could create a more sophisticated
the other path is to create a more sophisticated make workflow
where you're using some of the information that you can get from um
the the end of the call
the events um
they get sent to the web the make web hook
and using that to decide what to send to our platform
and what to do other things with it
you know
um along the way I think that's what you're getting at right Tom
yeah yeah
pretty much nice
and one question I didn't wanna ask is when
I guess this is gonna start becoming important
when now everybody knows you can measure these calls
and we have the Synthro analytics
which are just Surf's level now we have this
which set them more in depth
what would we be able to at some point we like create a okay
downloadable report with like a snapshot of the
the last the months worth of course
to be able to send it to send it to a client or is this
this is the one wait
hold on
let me get back you guys should just share this with the client right
yeah
so that's that's what I was about to say is that you can give a login
we can set you up so that your client can login
and only see their calls here
and that way they get
you know the interactive experience rather than the static report um
and then as I mentioned earlier
we're working on I'm hoping that we'll have a native integration with
with Sinflow and that will be these sorts of charts
in the other analytics products
I've shown you along the way
like this stuff that that'll all natively be in
in your white label account um
and then like along the way if people want
you know
something that turns us into a PDF
because they have some old school customers that want pdfs
we can probably make that happen
just FAX it to them and let us all yeah
yeah nice
that's cool uh
so then you got any questions of what you've seen
any alternation folks
and maybe directed someone like Scott
who's very heavy on all the alternation side and like a high level
is this something you could work into your stack for clients
and then if there's any leftfield questions
where you voice AI in general
or questions to Tom about plans far
far away and then we can spend a few minutes answering them
yeah so while we wait for any questions come through
is there anything else that's interesting
um that you're looking at Invoice AI that's kind of separate to this
it could be
um pure racing new models
it could be languages it could be latency related things
what what else is this space you interested in
um you know
it's always kind of tempting to talk about like the
the latest demos that have come out that showcase the
the cutting edge abilities of voice AI and multimodal AI
um and a lot of ways I think that's a little bit
it's just it's not as um
it's almost taking our eye off the ball
the what's really happening in the world right now is we're in
um
we're used to this idea that you can do amazing things with lolms
but some of you might have noticed this during the holidays
that like the Thanksgiving holiday
or you'll notice it at the upcoming holidays
most people that you interact with
maybe they have a chatty BT account and all they've said to it is
hello they're like
we're all living in the future
and we don't realize how much opportunity there is just to
bring people that aren't in the future that we're living in right now
up to where we are now and there's so much money and opportunity in
helping the world go from where it is now
just to the current state of technology
um with these interactive voice assistance
um and uh
I think part of the reason we don't realize is
I'm constantly talking to people like you Tom
or the people in this in this chat
um are in the session that like are living in that future already
but I think that we I think what's really exciting is just like
the opportunity that's here in front of us with with Sinflow
yeah true
you do you kinda get
uh wrapped up in everything that moves so fast
and that curve has been so violent that you do realise that everyday
semi news coming out but still people are still catching up
so that's a that's a great point
we have a question through through
net does it
does it email a report
say once a week or notify errors as they happen for real time alerts
so we have the current notifications we have are the Slack um
Slack alerts um
and uh
that's they'll tell you when a call was
a long call that typically finishes in this success
but failed or if it was a call that was on the happy path
that normally finishes in success
but fails um
we haven't built uh
any other sort of like email reporting or other types of alerts now
um I'd love to hear what you'd like to have built Nick
um you know
it's a fun stage of the company we're at uh
where it's it's just my co founder and me at the stage
and we're working closely with our customers on getting feedback and
you know we love getting to hear and build for things
build things for people
I wonder if that be like a
a good way I think it's pretty pleasant info too
just saying if like a certain amount of cool volume
then kind of falls outside of the norm
in terms of computer core ratio or something
that she just pulls the assistant
um trying to get like that feedback you
uh yeah
that would work really nicely for an outbound agent
right when you have a little bit control over um
you know you know
it's okay
it would be okay to turn it off for a little bit to make some changes
yeah run
just pumping out loads pumping out loads of calls
and realize that 5,000 of them have gone crazy
yeah yeah haha
um chain just asked about booking a call with us
I just dropped my calendar in there for anyone that like to meet me
and also I'd love to connect with people on LinkedIn
I just like seeing people build right
cool demos on LinkedIn of what they're building
and it's like LinkedIn it went from being like this boring thing
I hated being on Twitter when I was running my egg type company um
so like it's kind of a trap for me now
I spend way too much time on LinkedIn and Twitter
just because I get to see like
the coolest stuff that people are building
so I'd love if people um
oops that's the wrong link uh
would connect with me on LinkedIn
Twitter so I can stay up to speed on what they're building
amazing
and uh
any other thoughts just from
you can like wrap up after this if anyone's got any more questions
any more thoughts than just general like AI applications
uh is there any
no pretty much time for side projects
but is there any
anything like hobby related that you're playing around with
it could be video generation images
language translators language learning
know that with my kids
could be spending a lot of time with stuff like Synthesis
Tutor for math which is going really well
so any of those types scenarios you earned
I'm sorry I missed that
are you asking me or are you asking the community
should I answer at first I say to you
I'm the Zike community I see if anyone's got any good ones
um
yeah so one of the things I'm really interested in now is
how do you get emails from voice AI
they they
they bubble it all the time
and I have I have this API that um
I just shipped yesterday but still uh
last night actually but I haven't built the testing around it
but it's basically a way to accurately get an email from a caller um
and what we're doing with it is
we're just doing a lot of data processing to um
extract the the
the proper noun the spoken letters um
and hand that back I think um
I don't know I find it to be a really neat problem
it sort of goes back to this idea that I think
just voices and neat space with so many neat problems
and like even this one's kind of a mundane problem
like extracting an email from a call
but like it's just been really fun to work on um
I'm excited for getting to the point where I can start demoing it um
and hopefully maybe even integrate it with something like sin Flow
that would be useful
because that is a current paying point that requires a lot of um
creative prompting
but attention to detail and it's if you don't explain it well
then a lot of people do slip up
especially when they wanna capture customers
emails and then they come through wrong
so yeah plus 1 for that
yeah yeah
that's neat and the other stuff I just saw um
a post by um Quinn and daily
he's a friend of mine and he use Gemini's multimodal to um
basically I think show us what the
the future is gonna look like
where you're talking to a computer
and the computer is able to see what's on your screen
and it's going to respond to you in text and voice and video um
and I think as humans we
we find it easiest to speak um
but we ingests information with our ears at a slower rate
listening than we do for visually
we're like visual creatures
so I just think it
we're gonna have this future relatively soon around the corner
where people are talking to machines
and machines are talking and back
as well as showing us information back um
and I I think that's a I think that's really neat
I mean I I start talking to my computer too
with Whisper Flow and have really enjoyed yeah
I use Whisper Flow all day every day
yeah
basically doubles the amount that doubles your speed of like output
so any text box is there you can just reply twice as fast
so like Slack replies email replies like DMS yeah
it's incredible
yeah so you're mostly using it for email and Slack
is that the main thing you're using for
um
sometimes I do it for uh like notes
so I wanna take a quick note
I use that um pretty much anywhere actually
sometimes I will use it uh
this weekend or just gone
I was using it in a bolt dot new by stack blitz
and also lovable and it was just building
so while I was building I was doing something else and it comes back
I'll prompt it again with with just my voice and then just go off
so yeah I actually have to type
so you're building software just with your voice
now what what platforms are you doing that with
so one is called bolt.new B O l t.net
oh yeah yeah
uh huh yeah
by staclets and the other one is called lovable L O v a B l e dot gov
I haven't heard of lovable yeah
I've been hearing lots of actually
yeah and another one if anyone's interested in uh
it's the one I keep you interested in NYC
yeah there's another one that she which she may be more interested in
she's open source
works with Llama 3.3 centimetery called Cerebros Coda
um I can share the link in the chat for everybody still here
this will build things in under 1 second which is kinda really good
yeah the whole space is moving very fast and we got lots
lots of stuff to play with
so yeah voice AI is a super cool space to be in right now
I came to said final comment
uh agree on the problem one letter missing an email is a complete void
so we focus only on the right phone number
and then do post after post alternation
getting emails right
in a new flow we always
focus on getting the phone right during the call
caller ID and confirmation by voice
yeah
nice
nice well
I think it's pretty good place anything
anything to add Tom anything you wanna at the end
no I just wanna thank everyone for taking the time to
to listen to me and hear
listen to my opinions about what's happening in voice and
and give
give me the opportunity to tell them about what we're building uh
really means a lot to me thank you
of course I appreciate you taking the time to share with us
and maybe soon we'll see an inspiration inside the synth life as well
yeah that'd be great looking forward to it
hopefully we can figure that out nice
thanks Tom thanks everybody and see you back inside the community
so you posted straight after the call
the replay and posting up on YouTube too if you wanna see
thanks folks catch you on the next one