#004 Tom Shapland - Analytics for Voice AI Agents

Podcast Show Notes: Fireside Chat with Tom Chapland of Canonical AI

Episode Overview:
In this episode, we sit down with Tom Chapland, CEO and co-founder of Canonical AI, to discuss the evolving landscape of voice AI agents. Tom shares insights into how Canonical AI is helping developers and their customers gain visibility into the performance of voice AI agents, unlocking growth and building trust. We also dive into Tom’s journey from agriculture technology to voice AI, the challenges of building voice AI systems, and the future of the industry.

Chapters:
Key Takeaways:

• Visibility is Key: Canonical AI helps developers and their customers gain visibility into voice AI agent performance, building trust and unlocking growth.
• Voice AI is Still Emerging: The voice AI space is rapidly evolving, with many opportunities to improve and refine voice AI agents.
• Integration Made Easy: Canonical AI offers easy integration with platforms like Synthflow and Make, making it simple to start analyzing call data.
• Future Features: Canonical AI is working on advanced features like self-healing systems and more granular analytics to further improve voice AI agents.

Call to Action:

If you enjoyed this episode, make sure to subscribe to our podcast and leave a review! For more insights and tutorials on voice AI, visit our Synthflow Academy.

Creators & Guests

Host

Tom Osman 🐦‍⬛

Growth @synthflowai

What is Synthflow Podcast?

Welcome to the Synthflow Podcast, your go-to resource for exploring the latest advancements in voice AI.

Join us as we dive into insightful discussions with industry experts, share real-world applications of AI-powered voice assistants, and uncover how cutting-edge technology is transforming communication, productivity, and customer experience.

Whether you’re a developer, entrepreneur, or tech enthusiast, this podcast will keep you at the forefront of the voice AI revolution.

this conversation I speak with Tom Chapland of Canonical AI

now

this conversation was initially held inside of the Synthflow Academy

and was a fireside chat

so towards the end it did involve a bit of a screenshot

so if you get lost towards the end

I wanna see the outcome of that screen

showing what chemical is all about from a UI perspective

please head over to our YouTube channel and watch it there

but hope you enjoyed this conversation with Tom

okay I guess we can we just get a rock and rolling

if anyone's filtering after the fact

we can so thanks everyone for joining

I'm joined today by Tom Chaplin who's building canonical AI uh

which is a mixed part of the voice AI agents

you missed that intro before

so I guess kick things off

how you doing Tom and you wanna give us the quick one line intro

who you are and what is canonical

canonical they say

yeah you did great

you said it right uh

yeah I'm I'm Tom Chaplin

I'm the CEO and co founder of canonical

and and we give you visibility into what your voice AI agent is doing

we also give your customers who you're building voice AI agents for

visibility into what your voice AI agent is doing

and what we're finding in building this product is that um

there's some hesitation around the voice AI

your voice AI customers to expand their usage of the bogus

AIs that you build them um

but once you give them more visibility into what's happening

and they can see that the agent is performing as expected

that's what really unlocks growth

and our mission is to help voice AI developers unlock

the growth in voice AI um

by by helping their customers

feel more comfortable with putting their brand in their sales funnel

on the line with the voice AI agent

you're building them

yeah I like that

um it does seem a lot of the time it's a bit of a black box

and people can say the voice

I do certain things but can you back it up with the actual data

so this this course successful

did it go through did it completely gone etcetera um

you've been doing this for like a year

it's kinda a year and a half August 2,023 if this is correct

so you've been in the space for a little bit

is this is this always been the same product

no actually we started with something else

so I'm gonna go a little bit further back in describing my background

um I started an agriculture technology company back in 2014

and that company got into y Combinator um

it was a great company we

we were building a

sensor that help farmers figure out how much to irigate um

is based on my PhD work and is really neat data feed is

is a good company um

and in the process of building that company

I became more acquainted with nondeterministic

neural network type models

um it was uh

when like the Tensor Flow API came out

we're building cool computer vision stuff on top of our data feed

um and really liked it

and eventually I found a good home for that company

that company was called Tule and was acquired by Crop X

they're great people um

I want to start up journey again

and of course um

my co founder and I who was also the CTO of my last company um

we want to work in LLMS and initially we're building a cache in there

the idea of a cache in there is it's like a smart database

every time a query came in

um from a user asking an LLM something

we'd look in that smart database to see if

essentially the same question was answered

asked before if it was

we could return the answer from the database

rather than calling the slow

lumbering LLM um

and that LED us to reach out to a bunch of voice

AI developers to see if they'd be interested

this was about you know

that year and a half Mark ago

um

and what we found was like yeah

latency was a problem but well

actually I kind of skip to step there

the the thing I really wanna emphasize here is like

we just fell in love with the voice face

we started meeting these voice

AI developers and seeing all the cool stuff they're building

and just seeing the incredible potential for voice AI

that everyone here in this

in the session with us believes and feels

and as examples of seeing it um

and we just fell in love with the voice AI space

and we realized the cash wasn't the right thing to build

it was too slow our growth was too slow on it

I should say um

but what our friends and voice guy kept telling us was like hey

like I've got these contracts with these big potential clients

but they're only dipping their toe in the water

and I need a way to make them more comfortable with my voice AI

and I don't really blame them for not being comfortable with it

because I'm like manually listening to a few calls

I don't have any sort of visibility into what's happening

can you build something that makes it easy to show people like

here's what's happening in your data

and everything's going as expected um

so I started building that about um

I mean we shipped the dashboard in early October um

and it's been it's been really fun

it's been great ever since um

I just love being a part of the Voice Act community

and really glad to be here and

and hopefully get to meet some of you that have joined and

and get to make more friends in the space yeah

that's cool was there one demo when uh

you were kinda exploring options where you thought like

holy crap this is the space that I need to like apply my timer to

ah yeah

like what was the moment where I was like

oh voice AI is so cool

I think for me a lot of it is um

like when I first logged into Sinflow

and it must be similar for so many people

like you first log in and build your first agent as like

it's just a toy and it calls you and you're like

oh my God I can talk to it and it just

I feel like I still build

like voice AI agents just for fun on the side

like little little toy demos just like for fun

I built one that was like a baseball game for my son um

I feel like I'm an artist with a palette

in this really interesting palette that I get to work with

with voice AI and I just think it's such a neat space to be in

and also I see a lot of calls that um

people make um

voice agents make and they just

it's amazing to see how well they can work

you know like when they just nail it

the customer skeptical and they're like

let me talk to a representative

and then the voice guy just solves their problem

at the end they're thanking the machine

it's just like magical it's clearly the future yeah

some of the times when you listen back to a call um

so if you're testing an agent and I've done it a lot of times

you're tweaking a prompt

and suddenly you hear like a recording of a conversation

and you're like damn

that was actually really good

so you're tweaking things like sentence lengths and like do say this

then say that and when when you get that conversational flow right

yes yes

my music to his yeah what was that for you Tom dude

like what was the moment where you like oh

voice the eyes the space I need to be in

um so I've always been have like a

had a community going for a few years called Shiny Object Social Club

where it's just a bunch of us hanging out in a discord

and playing around with any new tech

that's interesting and then uh

I think 11 lads came along when it was like synthetic voices

and you could actually just get into say stuff

and then there's a bit of avatars coming around

and then that got applied to phone calls with things like sin flow

and battery at the time

and uh I wasn't extreme an extremely technical person

so sin flow is obviously the

that the natural fit being a more no code solution and the

the feeling it just gave

that I could now have this power to create these really powerful uh

workflows and tools and functionality was just yeah

it was just wild and then things like um

connecting it with other alternations

which we're gonna have a look at in a minute

where you can not only extract information from the phone call

but also have it trigger other workflows and suddenly

and make internal calls and outbound calls and hooking together

it's just just felt like everything is now on the table

yeah yeah

absolutely yeah

it's cool so yeah and

and it is is been kinda missing for a bit what we seen with chemical

and do you wanna give us the rundown

we're gonna do a screenshot too so you can do that whenever you want

um so basically this base if I'm right

this gives you a really easy way to see

how your agents are performing

that's correct right

yes that's the idea

gives you and your customers an easy way to see what your voice

AI assistant is doing and for you

it helps you improve your agent and for your

your client it helps you feel comfortable with what you're doing

so that they'll start pushing more call volume through it

hmm got it and have you seen have you seen that the take of it

is this meant for um

like small regencies s and B's escaping to enterprise

where where does where does it fit in

enterprises are always the slowest right

I think that um they

they clearly aren't going to um

use voice AI

until they have a lot of comfort with the fact that

the voice AI assistant is doing what you expect

so they're definitely gonna need people selling to enterprises

building voicing for enterprises

and enterprises are gonna need something like what we're building

but it's not really my my initial focus

just because it's so slow to get going with enterprises um

our customer base really is two different categories um

it's vertical SAS specialized vertical SAS

voice AI companies are focused on something like

building a voice AI assistant for um

HVAC businesses um

or they're the AI agencies um

and that's part of why I'm excited to be on this on this

um in the session with you is like

I think Sinflo does a great job with AI agencies and fitting

the needs of the AI agencies

um like with the white label product and that sort of stuff

and the AI agencies

you know the other type of our customer

and we find with both of the those two types of customers

the vertical SAS and our AI agencies

they kept coming to us saying hey

can this is cool

you built for us and it's helping us to hone our agent and refine it

but can you also build this

can you also expose it to our customers

um can you give them a login or can you

we embed what you built into our website so that um

they can see how the what

what's going on with the data

um so that's kind of the direction we're going with things now um

and making progress on that as well

I'm sorry that's kind of maybe a rambling answer

does that that's good

yeah that's really good

yeah I've got a bunch with a bunch of Ad Sheerans and head

so you getting the full white label route or is it more like semi

semi embettable sort of scenario

yeah so let me describe the current state of the world

and then also where we're going

um the current state of the world is you as the voice AI developer can

uh send us calls

um you send us recordings of the calls and a transcript

I mean go through later how to do the integration with make

and also the integration just with code

I'll points that as well um

then you can log in and see all the calls

for the different agents that you send us

in addition you can set up

you can work through with us to set up a login for your client

and your client can only see their agents

so we set it up in such a way that your client only sees

you know the agents are supposed to see

um

that's the current state of the world

then the next thing we're building is embedable components

so if you have a dashboard

like you're using the white label solution for Vincent flow

um Vincent flow

you can take

like the call flow diagram that I share with everyone shortly um

and you can bed that in the

the dashboard so you can show your client that um

as well and then finally um

this is in the works

but I'm talking to the the cofounders of Sinflow about um

doing a more native integration

that just makes this whole thing easier for

for you guys

and that there's not really the step where you have to mess around

they'll just be one click and you can you can get this

this stuff right there and sing flow

got it so I guess you able to trigger different kind of queries

depending on the outcome of the call

like in real time

or is it more of like a uploading batch and then analyze after say

like a campaign

it's a little bit both

so let me let me describe like the the main wait

so we can we can share the screen at any time

so whenever whenever you wanna add context

so you took your own we can do that I'll first describe it um

and then I'll show it and I'll summarize it again

so the the what I wanna describe is the first thing we show you is um

visualizations of what your calls are doing

so most of your calls will likely go down this happy path that

you designed um

but some calls will go off that happy path and take a turn

and those are the ones that you wanna look at

and they'll help you as the voice AI developer to improve the

the agent um

so and that requires that you log in and look at the data

and this is also what um the voice AI developers

customers also like they like being able to look in

and kind of see the different paths that the calls are going down

and get that kind of warm

trusting feeling that they can believe in this

this product and expand with it um

the other thing we do is we have uh

push notifications to slack around insights

so like right after a call ends

if it was supposed to be successful

if it was most likely to be successful

given the fact that of how long it was and what path

what call stages the call went to

but it wasn't successful we push those to you um

and make it easy for you to look at those

and see if you need to do any damage control or um

you know

you wanna make any changes to your prompts or your your integrations

um and then the last part is audio metric

so we we have metrics like um

the latency the number of interruptions and that sort of stuff

and that's really more for um

helping you surface different types of issues and dig

dig deeper into how the agents performing and also like

give reassurance to your clients like look

most of the time my latency is

you know this many milliseconds and that's within the

the human what's normal for human conversation

and things are working as expected

got it I guess

that kinda gives you the opportunity to be a bit proactive as well

so if you're already seeing

like a higher threshold of either foul calls

or ones going off the happy path

or highlight and see

then you could kind of proactively get in touch with your clients just

just to let you know

this is happening and we're on it and we're looking at fix

yeah so my last company

we helped farmers make irrigation decisions

and I think initially my first instincts

especially as a former academic that built this technology

and so close to me was like

if something went wrong I would like

I didn't want to go get the farmer's attention and bother them

and be like hey

this went wrong um

it just didn't feel right to me at the time

but then later I realized that

the key to our success was being very transparent with our customers

so we were always talking to our customers when we saw something good

and we wanted them to like notice how well this thing went

and also when something went bad

we reached out to them because it's better they hear it from us

and that they know that we're working on it

that's what prevents churn um

than if ended on their own

it's all about you know

maintaining and building that trust with your clients

and um

what what is the kind of state of monitoring and analytics tools

especially when it comes to third party

and so we do stuff in 10 minutes since low and

but I haven't checked in too recently on all the available tools

uh they

what what's the existing looking at right now

I think well

I think it

what the ecosystem looks like is indicative of the state of voice AI

right now voice AI is this emerging field

and there's some people getting a lot of traction in it

but not um

most people are still kind of in the early stages of it

so if you look at what other tools are out there for helping you

like third party tools for helping you develop your

your and understand how your assistant is doing there

mostly on the testing side

so the side before you deploy your agent into production

um it's for like

running simulations of lots of different scenarios

and making sure that um

your your agent is trying to automate that

testing that all of you are familiar with doing um

the the problem with that approach is like once you test it

it's great you can feel very confident in it

but there's no way to really simulate human behavior

like humans are the most unpredictable force in nature

and once your agent out into the production

and if you have agents in production

you know exactly what I'm talking about

like people do weird stuff um

and you're you're wouldn't expect what would happen there

and you're going to

really need to see what your agent is doing with real humans

to figure out how to improve it and um yeah

how to improve it so the

in the pre production side

there's companies out there like Vocera and Coval

um in the post production

there's us which is more of like this um

low code way of doing these things

um that

it's really amenable to working with AI agencies and your n customers

um or you can use more higher code solutions

like you can set up a whole

like data dog pipeline and build your own visualizations

uh and

and data flows through with data dog

but and

and the the problem with that is it's not really centered around voice

No. 1 it takes a lot of technical skills

and second there's a lot of problems with

voice that are different from just looking at like

traces from you know

text based lolms um

so I really think this area is kind of Mason and emerging

and we're seeing things emerge like what we're building

got it what's been

like the most common thing that people have found

after implanting your tool that they weren't aware of before

oh interesting

um

well I

think I once heard I was listening to a podcast recently of um

by C Partners and they're talking about what builds a great

what makes a great sass company

uh that really can capture a lot of the market like

like a gusto is that isn't that they do one thing really well

it's that they do a lot of little things

um and they do all of those myriad little things well

um and when I look at voice AI assistance

I can always kind of tell when I'm looking at the data

our customer sent us is like how

how much care and time the developer has put into developing

how much time it's been in production um

because there's always little things along the way that

it's just lots and lots of smooth

at rough edges that need to be smooth so

um finally I'll give examples

it's like a notorious one is um

I should actually more generalize first

there's all these workarounds you have to build

given the current state of the voice AI technology

with text to speech and speak to speech

text and lolms um

even with the audio only models and if you

you're not careful about how to do those workarounds

like how to get email addresses or um

how to be careful about like repeating people's names um

then you're gonna run into problems

um and it's really that sort of constant

careful hearing and craftsmanship that makes a great

voice AI agent I gave two examples there by mostly smoke high level

let me know if you want me to dig in more

no that's good no

I like it um

I think still very few people

I think really digging into digging into analytics

cause there's not too many doing a lot at scale

but presume that's gonna ramp up

now that they're getting more accurate

and they're getting easier to create and deploy so

so I expect that to increase soon

yeah so I'm looking at the chat

and I'm seeing that people are eager to see screen shares

and they're eager to figure out how to connect

especially Nick

I think Nick's very very keen to see a demo of screenshots

let's dive in and we can keep on

sharing my screen um

let's see

give me just a moment and any questions you go file them into

file them into Q&A we can

we can bring them up on screen

or you can even come up and ask yourself whichever

whichever you fancy

okay cool

you can sit if you wanna follow along

I'm gonna drop this in the um in the chat um

so here's the sequence of what I mean go through

I'm first gonna fairly quickly walk you through

our platform and what we're doing

and then from there

I wanna spend more time focusing on how you can integrate with

your sin flow agent with our platform

we have a free tier so it's it's easy to get started

I just put into the chat

also our blog post that really walks you through um

how to integrate with make or with code um

and that's what we're gonna cover now so so without further ado

let me go back to our dashboard

I'm on our demo account now um

and I when you first log in

you see the summary page

the summary page will show you how many calls you've uploaded

and the total duration of the calls you've uploaded

and some high level stats um

where things get interesting is when you scroll down

and you click on one of the agents that you've uploaded

and then we can dig in and look at that

that agent um

so I'm gonna scroll down here

and first gonna talk about how we process the data

you send us for each agent you initially send us excuse me

I should say it differently when you connect a new agent to us

we take the first 15 calls that you send us

and we use those first 15 calls to figure out the stages of the call

what I mean by stages is

an agent will typically go through stages like reading appointment

scheduling um

you know answering a question about a product uh

and then closing the call we figure out what those stages are

then once we have those stages every time a new call comes in

then

we can take each turn in the conversation and assign it to the stage

so when the user says hi

we can assign that to greeting when the user says

I wanna change my appointment

we can change um we can assign that to appointment scheduling

what this enables us to do is to determine

a path that each call goes down um

that all path is helpful in and of itself

but it gets really helpful when you can take that call path

and look at the call paths of all your calls for a given day or week

and really get the bigger picture of what your assistant is doing

um we do that automatically

you don't have to submit a core structure or anything

you can just get that from the course

well I'm really glad you asked that question

that was about to say is that we do it automatically

because some people have so many agents

they don't wanna go in there and edit each one

but also if I were logged into my personal account right now um

you could see a little edit button here

and you can edit the stages to better match

what you think they should be

got it okay

make sense

we do the same thing with the call outcomes

so we use the first 15 calls

and some LOL

magic to figure out what the typical outcomes are of the call

appointment not scheduled

appointment scheduled or product um

inquiry handled

then we use those outcomes

lolms are just like people in that they're more likely be accurate

if you break things down into small steps for them um

they're more likely to be able to solve a problem

when the problem is broken into small steps um

so we take those outcomes and then in another step we ask the LLM

was this a successful call given the outcome or not a successful call

um again

you can edit the outcome names in our dashboard

you can't see that here but if you log in um

you can see that uh

can you still see my screen by the way

uh I just opened your cash

okay

click on the sign up button here

um you can sign up for free and you get uh

I forget how many how many free minutes do you get

you get 4 hours of calls per month for free um

so you can click that sign up button um

and once you sign up you can click the upload calls button

we have a gooey for getting started

so you don't even have to integrate using make or um uh

code initially

you can just download calls from Sinflow and drop them in here

and that'll bring you back to what I was showing you earlier

where you have the uh

will bring you back to our dashboard

so this is it's really easy to get started and and try it out

um you just go to uh

I'll drop this in the the chat

you just go here and click sign up um

and you can try it out um

and upload calls

okay so now that we've uploaded calls I wanted to um

show you some of the things you get so this is the thing that my

the Voice AI developers

customers seem to like the most it's this call flow map um

I think it just gives people reassurance that they can see like oh

the voice the eye is doing what it's expecting

like initially greets the people and then after greeting them

some amount of calls go to objective not met

like yeah people sign out

um people hang up um

that's pretty normal um people hang up right away

but even that's something you can optimize as a voice

AI developer and work on different tricks um

but then the calls are getting you know from their partition

some are going to appointment scheduling

some are going to product enquiry handling

if we go to appointment scheduling

we can see that some go straight from appointment

scheduling to closing call

we can click on either a node

and see all the calls that went through appointment scheduling

or an edge between the nodes

and that'll bring out

a drawer where you can see the calls that went through this um

stage of um

greeting to appointment scheduling and what I'm looking at here

is a call where um it started with greeting

you can see the the stage and who was speaking is the assistant

it's a greeting then the user says

I'd like to schedule appointment

that turn that conversation turn was assigned to appointment

the appointment scheduling stage

and just gives you an easy way to kind of

look at your call volume and see like oh

I see what's happening um

everything's going as expected or something's not going as expected

this is gonna be an easy way for me to click in

and look in and see what's happening

and see what I need to change um hmm

can we pause in there yeah

that was that was really good

that was a really good walk through

so when let's say somebody's looking at this

this report

and they see there's something which has gone off the happy path

and they wanna dig in

how would you kinda recommend someone like evaluate these calls

just clicking in and read through the

read through the transcripts

and you can kinda tell where they've gone off

off path

yeah actually for that

so this first screen is really something that's like

for your quick view of understanding what calls doing

if you want to figure out how to improve your call

I actually like the call map more um

maybe that's a bias because I'm a little uh

it was the first thing we built in it

I think it's neat but um

with the call map

it's a similar thing where we're showing you the stages of the call

but what's different about the call map is we show you

will highlight in red

where most of the calls are stopping with failure

um

it makes it easier to subset to just the calls that are going poorly

the other thing you can do with the call map and

and the demo accounts not as good as an example

but you can see

you'll see something like most of your calls like 106 calls here

47 calls here 17 this is the happy path that most calls are going down

but you might find like this weird branch

and it's when you click on the branches that

that go off in an unexpected direction that you find the insights

you click on those and it will um

and if you click on the transcript part

it'll bring you to the part of the conversation

where it's going off the the rails

it will highlight in yellow

and it will give you ideas about what you need to improve

about your assistant um

we haven't

one of the things we want to do is like

start using diffusion models to turn this into like

a like a comic book panel

so it's quicker to read because reading transcripts is slow

but people can ingest that else of quicker

but we haven't built that yet yeah

we we have like a

the Ellen

Musk judge to just like evaluates the call that summarizes it

but again that's still uh

like a bit of a pain to read all this summaries

especially be doing any sort of significant cool volumes

this might be a bit like a out there question

presuming might be possible and I don't like can I guess features

etcetera but you can almost kind of extrapolate similar

this out to self healing and self recommendations through

call improvements

and maybe if somebody ships like an incomplete assistant

and maybe could use another couple of stages added on to it

you could you could probably even tell by

and having a decent bunch of calls running through your system yeah

that's exactly where we're going with this

so with I'm gonna pop over back to my dashboard

um it was a pretty scheduled comment below because I logged in

it was this thing

um I can't show it

but we have a feature where you can integrate um

with Slack

and what we're doing at Slack in sizes were showing you like

here's a call that went down a happy path

90% of your calls to go through all these stages

the happy path they complete and have a successful outcome

but here's a call that wasn't successful

but it went down the happy path

look at it um

and where we wanna go from there is to then

take those calls that should have been successful

because we've we figured out how to subset just to the calls that um

should have been uh

that are problematic rather than like manually sampling

um and we want to suggest improvements to people system prompts

um to

to help them like you know

figure out like oh

well these calls are going poorly because when the voice

AI repeats their name back

spells their name back it speaks way too fast

you gotta like prompt it to speak slower

um things like that

that we think we we wanna get to the vision that you describe Tom

where we can help kind of build

self healing systems with the human in the loop

hmm

yeah I'm wondering that

you can really pull some really good insights out of this

and instead having a data served up this way

like a visual for any decent sized company

you have to report on the effectiveness of these things

that's gonna be a really useful page to show them

yep

yeah um

so another thing I'd like to show is um

the custom metric so what a lot of people

cause is in line with what you're just talking about Tom

um you can ask custom metrics of call

so the summary is great um

but summaries

one line summaries LLM is a judge

they don't capture kind of what happens within a call

often they all think loss is over

this is a common problem I say Isaac Gloss is over

kind of the details that might happen in particular turns of the call

um and one of the ways to approach that issue is to like

look at call maps and it'll help you identify

the different stages that you need to focus on

and subset rather than random sampling

um another thing you can do is you can define custom metrics

like you may be interested in knowing um

your your voice

I should be asking in the US for consent to record the call um

and you might wanna ask like

did the assistant ask for consent to record the call

and will then for every call that comes in

will ask that question and will surface

you can click on the times it's false um

and we'll show you just the calls where that was false

you can look and be like why did the voice say I not ask for consent

you can see what the issue was and try to figure out how to improve it

um so just one more way to uh

improve your agent and it's also something that like

your customers might be interested in too

because they're like well

how can I get my voice AI assistant to sell better

um you know

how to handle this objection

did the voice AI handle the objection or something like that

yeah nice

there's a there's a couple of like

interesting quirks sometimes come up in assistant calls

one of them is sudden uh

sudden bigger bigger lacency gaps in between questions

so say if you have a script

everything's gonna read nicely

and then suddenly for some reason a question comes up

the user responds as normal

but then there's a big lag and then the user has to repeat themselves

and then maybe the assistant repeats themselves

and then it kinda gets through

is there a way to identify something like that in in your system

yeah so you're this is like

it's almost when I see this happen the most is around interruptions

where like uh

the the person the voice say

I will think the person has finished talking

it'll start talking and then the person will like wait for them

and it just like

the question ends up getting repeated and things get confusing

is that when it's like type of scenario you're talking about

or maybe darkness some sometimes it can be like a

like a rogue background sound which cuts the cuts the response out

but there is uh now and again

I don't know if this happens when you just uh

maybe listening to voice articles all day

so you're more attuned to it and then an old person won't notice

but sometimes you do find that now and again

there comes like a little bug where there's like a

nine out of the 10 questions that the assistant will use on the call

be absolutely perfect like normally it's expected

but then one of them for some random reason

uh just has like a

a double or triple the length of gap and it'd be good

this is maybe like a more minor

like fine tuning use case rather than a high level general one

but it would be good to be able to identify against photos

and I I sit when I'm doing recordings since doing videos like

cause I'm looking at the audio waves between the responses

so I can see the legacy gap on each response

sometimes they're most of the time they're even

but then you can see one

which is like double the length of the other one

uh yeah

I know what you're talking about so

um we calculate on the back end

we're calculating the latency

the for the human response and the latency for the AI response

and we're calculating

how much of the time the person is silent

when it's their turn to speak

which is indicative of confusion

and all these other audio metrics

we haven't surfaced them yet

we just have a um

there when you log in um

this is something we're shipping

hopefully today

is something where you can just download all those metrics

for your calls so you can see the latency for all your calls

but eventually

we wanna surface that and put it in what we call a rain cloud plot

where right now this is the call duration distribution

and you can see like most of the calls this frequency history

it's a frequency um distribution

um you can see most of the calls are around here

around 30 seconds um

and then the individual calls are these dots underneath um

and we wanna create a drop down here for latency

so you'd be able to see like

oh most of my calls have pretty low latency

but you could click on the ones with higher latency

and see what was going on with those

and see when they happen are they happening because of interruptions

or is there some way you can

might be able to improve the flow so that those

those late and sees are less likely happen

or like

there's a long wait and see and maybe you wanna go tell your client

like hey

this thing happened um

that's uh

but we're working on it it sometimes happens

it's okay most of the time your late and sees are over here

you know there's nothing to worry about

yeah sometimes it can be random things like uh

response length or sensitive length

or if they're doing some sort of rag over a knowledge base

which has like an awkward setup

then there's just like a bubbling legacy there too

if they're making like uh

it's easier custom action

which is like a laggy API and then they get a response back in time

um yeah

having that flag would be

would be actually awesome

especially at scale that'd be

that'd be killer

yeah I mean

what you just said speaks to the craftsmanship of building voice AI

is like you can uh

I think um

you know you can get something up and running in

in five minutes and that will get you 80% away

but the next 15% is gonna take you a lot longer and a lot of skill um

you know it's faster to get there if you have a lot of skill

but it takes you know

really understanding how these systems work and how to like

avoid these latency traps and

and other problems they can come up if that

you know if you're not designing your system well

and carefully monitoring and analyzing it

hundred percent there's a if anybody's got a new system flow

we can just cover what we're talking about a bit more is

if you're using something like custom actions

or if you have your system connected to Knowledge Base

if the user asked the question which isn't in the prompt

then it can go and look up through a document or a series of documents

Wikipedia and then bring that um

information back into the call

which may take slightly longer than a pre determined script

or another one

which you can do in sin flows and called a custom action

which is basically just like an API call

which you can either trigger to happen before the call happens

so uh

actually an example I like to use is if you wanna get the weather

so if you're doing anything with like an outside venue for instance

you wanna know the weather to determine what you recommend to the

end customer or if it's like a car service or like a concert

whatever it is

then you can bring that weather forecast into the conversation

you could probably do that before the call starts

then you avoid the mid call conversation

but if you're the mid call later see um

the way moment to cover that if you're doing it mid call

this could be something like

looking up an order ID

in a Shopify store to track where the shipment is

for instance that could be the customer gives the ID

you can look up the database to see the states of the package

and to cover that API call going through to to shopify

you will set in your prompts or in your field in sin flow

you can trigger it and say okay

I'm just gonna look up your information now

and while the this is the same

that sentence is making me a guy called the same time

so that then covers that look up hopefully that makes sense

that was a that was a great intro to um

knowledge bases and rag and tool calling uh

all all the right there very specific

thanks not being at the most advanced custom action developer

we've got you killers and simply does some amazing stuff

but that's as simple as I ha ha I can understand anyway

um story time how to integrate

yeah that's right

no that's it

I was just gonna get good and I'll be I'll be awesome okay

so one of the links I dropped in the chat was um uh

to this blog post that tells you how to connect your sin flow agent

using make um

and the place really just started it

it walks you through all the steps um

uh

but you can also download the scenario

like if you go into this blog post and scroll down

you'll find the scenario where you can just download it

and upload it into make so just backing up a moment um

make is a platform for um doing server side automations um

so it's like instead of writing the server code you can use make to um

take the the the data from a call and do something with it um

like you know send an email with it or

or whatever and we're using

you can use make also to send data to our platform um

so you log into make and the first step is to create a custom web hook

and this walks you through creating that custom web hook

and then you would then go into sin flow and go to your

your agent that you want to um uh

send the data from that the call data from that agent to our platform

and you click on deployment and rest API

and you would paste in that web hook that you got for make um

then the next step is I mean

the easiest thing is to just download the

the scenario from from our website right here

and then you upload that into um

into make and then the final changes they

all you need to do to finish the integration is you end up going into

here I'll go into my make and you have to um

it'll look like this once you upload your

your file into your scenario to make

and you have to put your API key in for canonical

and then it's it's ready to go

I mean I I blaze through that really fast

but it's that's just the bigger idea

you might be wondering where you get your API key to get your API key

you go to our website and you click on the little icon and there's um

the setup button and you can get your API key from in there

um and I don't know what maybe I went through that too fast time

should I slow down on part of that or does that makes

does that look good I think

I think make make integrations are probably a bit more like on the

on the niche technical side anyway

and it probably won't be one of the like beginner use cases for

for using make I think probably a lot of people are still at the

either just trigger a call by a make with some flow

or retrieving the details of the call by some society

to actually produce some more education materials

in this case maybe I can fire up the tutorial

yeah and our tutorial actually you might find it

people here might find it as a good way to get started with me

cause it's like a

walks you through an example that you can get up and running um

and and kind of acquaint yourself with the mix system um

we all in the chat is a monitor synth play voice yeah

and of course cool I'll let my drop this in the Academy too okay

yeah that'd be great um

and you can also we

I also linked to code on how to programmatically to our docks

excuse me so if you're if you're writing server code um

and you you want to integrate with

with our code and send data via the send the calls via the API

that's all in here

there's a tab for sin flow for hitting our sin flow and point um

so that's that's there

for people that are kind of interested in integrating with code

how granulate can you get with the make inspiration

can you

can you animate screen comprehensive when she passing Jason through

can you I guess then you can filter filter by um

if uh

there's like an unintended call which goes through depending on how

you labelling it or a failed call

and you could trigger a slight message too would that be right

yeah

one way if you pick up those dynamically created uh

call stages from the retrieval

so one way to do it I'm gonna start with

the way that kind of better maps to my way of seeing the world first

which is like just send all your calls

use this make um

scenario here to send all of your calls to our platform

and then use our platform to figure out the calls that you want to um

be alerted about and you want to surface

so you can set up the Slack integration for that um

you could create a more sophisticated

the other path is to create a more sophisticated make workflow

where you're using some of the information that you can get from um

the the end of the call

the events um

they get sent to the web the make web hook

and using that to decide what to send to our platform

and what to do other things with it

you know

um along the way I think that's what you're getting at right Tom

yeah yeah

pretty much nice

and one question I didn't wanna ask is when

I guess this is gonna start becoming important

when now everybody knows you can measure these calls

and we have the Synthro analytics

which are just Surf's level now we have this

which set them more in depth

what would we be able to at some point we like create a okay

downloadable report with like a snapshot of the

the last the months worth of course

to be able to send it to send it to a client or is this

this is the one wait

hold on

let me get back you guys should just share this with the client right

yeah

so that's that's what I was about to say is that you can give a login

we can set you up so that your client can login

and only see their calls here

and that way they get

you know the interactive experience rather than the static report um

and then as I mentioned earlier

we're working on I'm hoping that we'll have a native integration with

with Sinflow and that will be these sorts of charts

in the other analytics products

I've shown you along the way

like this stuff that that'll all natively be in

in your white label account um

and then like along the way if people want

you know

something that turns us into a PDF

because they have some old school customers that want pdfs

we can probably make that happen

just FAX it to them and let us all yeah

yeah nice

that's cool uh

so then you got any questions of what you've seen

any alternation folks

and maybe directed someone like Scott

who's very heavy on all the alternation side and like a high level

is this something you could work into your stack for clients

and then if there's any leftfield questions

where you voice AI in general

or questions to Tom about plans far

far away and then we can spend a few minutes answering them

yeah so while we wait for any questions come through

is there anything else that's interesting

um that you're looking at Invoice AI that's kind of separate to this

it could be

um pure racing new models

it could be languages it could be latency related things

what what else is this space you interested in

um you know

it's always kind of tempting to talk about like the

the latest demos that have come out that showcase the

the cutting edge abilities of voice AI and multimodal AI

um and a lot of ways I think that's a little bit

it's just it's not as um

it's almost taking our eye off the ball

the what's really happening in the world right now is we're in

um

we're used to this idea that you can do amazing things with lolms

but some of you might have noticed this during the holidays

that like the Thanksgiving holiday

or you'll notice it at the upcoming holidays

most people that you interact with

maybe they have a chatty BT account and all they've said to it is

hello they're like

we're all living in the future

and we don't realize how much opportunity there is just to

bring people that aren't in the future that we're living in right now

up to where we are now and there's so much money and opportunity in

helping the world go from where it is now

just to the current state of technology

um with these interactive voice assistance

um and uh

I think part of the reason we don't realize is

I'm constantly talking to people like you Tom

or the people in this in this chat

um are in the session that like are living in that future already

but I think that we I think what's really exciting is just like

the opportunity that's here in front of us with with Sinflow

yeah true

you do you kinda get

uh wrapped up in everything that moves so fast

and that curve has been so violent that you do realise that everyday

semi news coming out but still people are still catching up

so that's a that's a great point

we have a question through through

net does it

does it email a report

say once a week or notify errors as they happen for real time alerts

so we have the current notifications we have are the Slack um

Slack alerts um

and uh

that's they'll tell you when a call was

a long call that typically finishes in this success

but failed or if it was a call that was on the happy path

that normally finishes in success

but fails um

we haven't built uh

any other sort of like email reporting or other types of alerts now

um I'd love to hear what you'd like to have built Nick

um you know

it's a fun stage of the company we're at uh

where it's it's just my co founder and me at the stage

and we're working closely with our customers on getting feedback and

you know we love getting to hear and build for things

build things for people

I wonder if that be like a

a good way I think it's pretty pleasant info too

just saying if like a certain amount of cool volume

then kind of falls outside of the norm

in terms of computer core ratio or something

that she just pulls the assistant

um trying to get like that feedback you

uh yeah

that would work really nicely for an outbound agent

right when you have a little bit control over um

you know you know

it's okay

it would be okay to turn it off for a little bit to make some changes

yeah run

just pumping out loads pumping out loads of calls

and realize that 5,000 of them have gone crazy

yeah yeah haha

um chain just asked about booking a call with us

I just dropped my calendar in there for anyone that like to meet me

and also I'd love to connect with people on LinkedIn

I just like seeing people build right

cool demos on LinkedIn of what they're building

and it's like LinkedIn it went from being like this boring thing

I hated being on Twitter when I was running my egg type company um

so like it's kind of a trap for me now

I spend way too much time on LinkedIn and Twitter

just because I get to see like

the coolest stuff that people are building

so I'd love if people um

oops that's the wrong link uh

would connect with me on LinkedIn

Twitter so I can stay up to speed on what they're building

amazing

and uh

any other thoughts just from

you can like wrap up after this if anyone's got any more questions

any more thoughts than just general like AI applications

uh is there any

no pretty much time for side projects

but is there any

anything like hobby related that you're playing around with

it could be video generation images

language translators language learning

know that with my kids

could be spending a lot of time with stuff like Synthesis

Tutor for math which is going really well

so any of those types scenarios you earned

I'm sorry I missed that

are you asking me or are you asking the community

should I answer at first I say to you

I'm the Zike community I see if anyone's got any good ones

um

yeah so one of the things I'm really interested in now is

how do you get emails from voice AI

they they

they bubble it all the time

and I have I have this API that um

I just shipped yesterday but still uh

last night actually but I haven't built the testing around it

but it's basically a way to accurately get an email from a caller um

and what we're doing with it is

we're just doing a lot of data processing to um

extract the the

the proper noun the spoken letters um

and hand that back I think um

I don't know I find it to be a really neat problem

it sort of goes back to this idea that I think

just voices and neat space with so many neat problems

and like even this one's kind of a mundane problem

like extracting an email from a call

but like it's just been really fun to work on um

I'm excited for getting to the point where I can start demoing it um

and hopefully maybe even integrate it with something like sin Flow

that would be useful

because that is a current paying point that requires a lot of um

creative prompting

but attention to detail and it's if you don't explain it well

then a lot of people do slip up

especially when they wanna capture customers

emails and then they come through wrong

so yeah plus 1 for that

yeah yeah

that's neat and the other stuff I just saw um

a post by um Quinn and daily

he's a friend of mine and he use Gemini's multimodal to um

basically I think show us what the

the future is gonna look like

where you're talking to a computer

and the computer is able to see what's on your screen

and it's going to respond to you in text and voice and video um

and I think as humans we

we find it easiest to speak um

but we ingests information with our ears at a slower rate

listening than we do for visually

we're like visual creatures

so I just think it

we're gonna have this future relatively soon around the corner

where people are talking to machines

and machines are talking and back

as well as showing us information back um

and I I think that's a I think that's really neat

I mean I I start talking to my computer too

with Whisper Flow and have really enjoyed yeah

I use Whisper Flow all day every day

yeah

basically doubles the amount that doubles your speed of like output

so any text box is there you can just reply twice as fast

so like Slack replies email replies like DMS yeah

it's incredible

yeah so you're mostly using it for email and Slack

is that the main thing you're using for

um

sometimes I do it for uh like notes

so I wanna take a quick note

I use that um pretty much anywhere actually

sometimes I will use it uh

this weekend or just gone

I was using it in a bolt dot new by stack blitz

and also lovable and it was just building

so while I was building I was doing something else and it comes back

I'll prompt it again with with just my voice and then just go off

so yeah I actually have to type

so you're building software just with your voice

now what what platforms are you doing that with

so one is called bolt.new B O l t.net

oh yeah yeah

uh huh yeah

by staclets and the other one is called lovable L O v a B l e dot gov

I haven't heard of lovable yeah

I've been hearing lots of actually

yeah and another one if anyone's interested in uh

it's the one I keep you interested in NYC

yeah there's another one that she which she may be more interested in

she's open source

works with Llama 3.3 centimetery called Cerebros Coda

um I can share the link in the chat for everybody still here

this will build things in under 1 second which is kinda really good

yeah the whole space is moving very fast and we got lots

lots of stuff to play with

so yeah voice AI is a super cool space to be in right now

I came to said final comment

uh agree on the problem one letter missing an email is a complete void

so we focus only on the right phone number

and then do post after post alternation

getting emails right

in a new flow we always

focus on getting the phone right during the call

caller ID and confirmation by voice

yeah

yeah

nice

nice well

I think it's pretty good place anything

anything to add Tom anything you wanna at the end

no I just wanna thank everyone for taking the time to

to listen to me and hear

listen to my opinions about what's happening in voice and

and give

give me the opportunity to tell them about what we're building uh

really means a lot to me thank you

of course I appreciate you taking the time to share with us

and maybe soon we'll see an inspiration inside the synth life as well

yeah that'd be great looking forward to it

hopefully we can figure that out nice

thanks Tom thanks everybody and see you back inside the community

so you posted straight after the call

the replay and posting up on YouTube too if you wanna see

thanks folks catch you on the next one