Unlocking the Potential of In-Vehicle Edge AI | Ep 15 | Driving Innovation by Sonatus

In this episode, Sonatus Product Manager Steve Stoddard and Head of Product Sanjay Khatri, discuss the opportunity and challenges of deploying in-vehicle edge AI beyond ADAS and AD to transform the vehicle driving and ownership experience. The topics discussed here are explained in detail in the Sonatus White Paper, Unlocking the Potential In-Vehicle Edge: The Role of SDV Technologies and Flexible E/E Architectures.  

Creators and Guests

Host

Sanjay Khatri

Sanjay Khatri is Head of Product Marketing at Sonatus, Inc., a leading provider of software solutions accelerating the shift to AI-enabled Software-Defined Vehicles (SDVs). He brings more than 20 years of experience in product management and marketing for cloud and edge software across Fortune 500 companies and early-stage startups. At Sonatus, Sanjay defines product positioning and go-to-market strategy for solutions that help automakers deliver intelligent, adaptive vehicles that evolve with changing market demands. His background includes leading strategy and execution across Connected Car, In-Vehicle Infotainment, and IoT technologies. Sanjay holds a Bachelor’s degree in Computer Engineering and an MBA, along with certifications in AI/ML, telecommunications engineering, and content strategy. He is a frequent writer and speaker on the future of automotive software, edge AI, and the rapidly transforming mobility ecosystem.

What is Driving Innovation by Sonatus?

Dig into wide-ranging technical topics about modern vehicle technology with industry leaders, hosted by Sanjay Khatri, Head of Product Marketing, Sonatus.

Welcome to another episode
of Driving Innovation,

the podcast that explores the
intersection of automotive

software innovation and AI and how
it's driving the future of mobility.

Today, we're diving into
how software-defined vehicle

technologies and solutions
can enable edge AI to

transform vehicles from static
machines into intelligent,

adaptive platforms, a topic we
cover in our latest white paper,

Unlocking the Potential of
In-Vehicle Edge AI that you can

download from our website.

We'll break down why
cloud-based AI isn't enough,

what's holding OEMs
back, and how SDVs unlock

real-time in vehicle
intelligence,

from predictive maintenance to
energy optimization and beyond.

Whether you're a business
leader shaping your company's

digital roadmap or a
technologist driving changes

under the hood,
this one's for you.

Let's get started.

With me today to discuss
in-vehicle edge AI is Steve

Stoddard, product
manager here at Sonatus.

Steve, welcome to the
podcast. Thank you, Sanjay.

Well, let's get started.

What is the buzz around edge AI
in the automotive industry today?

Yeah.

I think there's a number
of trends that are kind of

converging right now.

One of which is autonomy that
has been developed over a

number of years here over
the last decade or so,

and it's finally starting
to really come to fruition.

And so there's a lot
of buzz around that.

But also, you have the confluence
of these traditional LLM models

that are really
expanding quickly.

And while they're mostly
based in the cloud,

it's really bringing a lot of
awareness to AI and bringing a

lot of customer interest and
actually expectation around

having AI in their
vehicles today.

One of the issues is that
cloud-based AI has a kind of a

limit because of the data costs
associated with sending data

from the vehicle to the cloud.

And there's always potential
connectivity issues.

And let alone there's
also the concerns around

PII data,

basically personally
identifiable information or

possible data leaks from
customer proprietary information.

Things like GDPR in the EU
and other regional-based

regulations that would require
a little bit more sensitivity

around control of personal data,

all lend towards wanting to
run AI models in the vehicle as

opposed to in the cloud.

I know AI is all
the rage these days.

What is happening in the
automotive industry today to

make edge AI more possible
that perhaps wasn't there

before in some of the more traditional
legacy vehicle architectures?

Yeah. Definitely.

Software-defined vehicles,

the sort of SDV trend definitely
is a big factor in that.

Of course, when things are more
controlled via software where you can

change certain variables,

then it enables a lot of
capability for AI models that

maybe didn't exist before.

Additionally, sort of a lot of these ML
models have really become more established.

And so traditional ML has kind
of paved the way for things

like recommendation models and things
that have been used in the cloud today,

which now are starting to be
better optimized for running

on limited hardware that
you see in the vehicle.

At the same time,

some of the hardware that's
being developed in the vehicle,

there's more of a trend
towards HPCs and compute

capability that may be better
suited for AI neural compute

and GPUs, especially with the
rise of autonomy as I mentioned.

There's a lot more focus of
these types of silicon that can

be implemented into the vehicle.

And so there's a lot wider
variety of capable ECUs in the

vehicle today that actually
can run these types of models.

So clearly SDV technologies opens
the way to bring edge

AI into vehicles.

Let's dig into that
a little bit more.

What's been holding
OEMs back up until now?

One of the big things is going to be
the network topology in the vehicle.

So the actual architecture of
the vehicle and the signals

traditionally have been
more rigid and static based.

As we move to SDVs, of course,

that flexibility introduces a
lot more of this capability.

Another problem is the
actual data access.

So one of the things that
one has to think about when

deploying AI models into the vehicle
is how do I send the

right data that's required as model
inputs to that model at the right time.

And that's one of the things
that certainly the newer

technologies are
starting to enable.

Additionally, there's a lot of
cloud-based and sort of MLOps

types of tools out there.

Very good for deploying
models in the cloud,

but that's the more
traditional way of doing ML.

There's less fewer
resources available,

fewer tools for deploying models
into embedded devices like vehicles.

And I think vehicles have
a little bit more sensitivity

requirements compared to maybe
just general IoT devices,

particularly for the data control
access and things like that.

Right.

So flexibility and
precision, obviously,

needed for running AI models.

What you mentioned about
more automotive specific AIML

ops tool chain.

Let's explore that
a little bit more.

Can you elaborate on why
all of the tools that are

available out there may not be
suited for an automotive environment?

Yeah. Definitely.

This is this is a tricky one
because the folks who have

access to the data
coming from vehicles,

typically the pathway for
training and deploying a model

would involve
gathering all the data,

building the model from that,

and typically that's going to be
data scientists or ML engineers.

And then ultimately, to
bring it into a vehicle,

you have to make sure you're
connecting it to the right data

sources and optimizing the
model, typically downsizing it,

implementing certain
techniques like sparsification,

then designating where you want
to deploy it into the vehicle,

and ultimately the approval
process for saying,

I'm good with this model.

I know it's not going to harm
any other processes in the

vehicle, and I can go ahead
and actually deploy it.

So we find that ML tool chains
really need to hit upon all of

these key facets.

Otherwise, it's
just a roadblock.

You won't be able to execute.

Right. Yeah. It's not like
deploying AI on a big server.

And plus it's a device
or a platform that has to

do a lot of other things.

Primarily, you know,

transporting people goods
from point A to point B,

keeping them safe and
keeping them, you know,

entertained and happy.

So, there's a lot of
moving parts. Definitely.

A lot of other mission critical
things that have to be considered.

I'd like to explore the
infrastructure a little bit more.

We talked about how the vehicle
has to do a lot of different

things, mission-critical things.

It's a constrained environment.
There's a lot of limitations.

What sorts of architectural
choices can OEMs

make to accommodate
the environment for AI

models while also fulfilling
the primary mission of the

vehicle, which is
transportation and mobility.

So one of the things as
OEMs make the transition to

SDVs, there's often a
changeover from more like

the CAN lower speed
architectures networks within the

vehicle to more of
an Ethernet-based.

So traditionally,

you wanna see like more
of an Ethernet backbone.

And then we see anything
from the more distributed types of

network with distributed ECUs
to domain-oriented architecture

to even a zonal architecture.

In all of these cases,

you generally have the problem
of either bringing the model to

the data or bringing
the data to the model.

And so in those cases,

you need to have the right
solutions where you can

actually deploy that model into
that edge ECU node or even an

MCU to run where the
data is available.

Or conversely, if you're
going to put the model into,

for example, a gateway,

you need to have sort of an
agent to grab the data from the

edge ECU and bring
it to that gateway.

So those are a few of
the critical items.

Generally, we also think that
a service-oriented architecture

is more amenable to deploying
these kinds of models and

providing the flexibility
that's needed to run these in

the vehicle and to
see that capability.

But also the need
to run these in a

containerized environment.

Going back to that safety,

mission critical
type of viewpoint,

we wanna make sure that any
model that's deployed is always

going to keep the vehicle safe
and it's not going to consume

too many resources that
would otherwise prevent those

mission-critical functions
from being able to execute.

Yeah.

So you really need a flexible
platform where you can

deploy models in
general-purpose ECUs

and not rely on
specific high-end GPUs, etcetera.

Because those ECUs have to
do other things as well,

like manage the powertrain,
manage the infotainment systems.

Absolutely. So interesting.

So there's a lot of different
challenges that have to be juggled.

So I can see how you need to
have a much more sort of a

tailored solution for bringing
edge AI into vehicles.

Definitely.

And that's where,
just to that point,

the optimization
becomes so important.

Right?

Resources are constrained and
so you wanna make sure that the models are

as optimized as possible without
giving up the

accuracy that you need
for the model performance.

Great.

So clearly, is an opportunity,

but there are also challenges.

And as you've explained,
SDV technologies can

pave the way for bringing
in more edge AI use cases.

By the way, we haven't really
talked about use cases.

I know people talk about
ADAS and autonomous driving,

and a lot of that
is dependent on AI.

What are some of the other use
cases that is enabled by Edge

AI? And then why really
should OEMs care about it?

Yeah. Definitely.

So a lot of the, we think,

most interesting use cases are
actually totally outside of

ADAS because that's a
very well-trodden path.

A lot of folks have done a lot
of great work in that area.

But using some of the
more traditional silicon,

the traditional hardware that's
available in the vehicle,

we think you can run more
interesting models that will

deal with personalization and
certain types of features.

So I can give a few
use case examples.

Certainly tire wear
and tire management

can come into play from
a safety perspective.

And so there are
some traditional

physics-based models as well as
taking data from other sources.

So certain driver behaviors
where you can even personalize

these tire wear analysis
models or the hydroplaning type

of models as well,

making sure you have
sufficient grip on the road.

Providing an early warning
to both the user or to other

systems in the vehicle to
change the traction control or

things like that.

Another place is
in personalization.

So things like

how a user might actually
interact with an ADAS system.

So this can enable new features
over time, for example.

I may have certain hardware in
the vehicle that is capable of

supporting a new model, but I
don't have time or it didn't exist.

I didn't know of it at
the time that I'm actually releasing the

vehicle into start
of production.

These platforms can bring the
ability to deploy new models to

the vehicle and provide
even more personalization.

So where there's maybe a
personalized distraction.

So depending on who the user is,

I may want to change the
frequency or the way a

particular distraction tone
plays depending on whether I've

been distracted more
immediately leading up to the

current moment in time.

There's a few others as well.

There's certain regulations
that are coming out.

Certain regulations in the
EU around automatic leveling

of headlights and different
things like that are gonna

require these kinds
of capabilities.

And you can add additional
features on top of that while

you may be able to meet the
letter of a given regulation.

Now you have the capability
to add a new function that can

dynamically adjust the headlight
even while driving, for example.

We're starting to see, you
know,ChatGPT or other types of

LLM-based models also
come into the vehicles.

Is there any room for
those types of LLM based

models to come in, you
know, aside from, you know,

asking ChatGPT where the
nearest coffee, you know,

place is and and
navigating to it?

Yeah.

Yeah, absolutely.

And this is what's driving a lot of
the excitement in the space for sure.

And so there's a variety of different
LLMs and use cases for those.

There's the traditional
chat assistants,

which I think a lot of
people are playing with,

to varying degrees of success
and enthusiasm for sure.

These are often
running into IVI.

And so depending on the
particular use case of an LLM,

you may want to run it on a
different ECU in the vehicle.

One other use case for LLMs
that we've seen that we think

is particularly compelling is
around the area of cybersecurity.

A lot of times the intrusion
detection models have very

discreet, very numerous
rules for potential

intrusions and require a lot of
data to be transferred back to

the cloud as well as a
lot of false positives.

With LLMs, we're finding some
model vendors that are able to

significantly reduce those false
positives as well as

cover a lot wider
range of threat

pathways, basically,
threat vectors,

and can be simplified
into these models.

One concern, of
course, is, you know,

what constitutes an LLM and
can it run in the vehicle?

And this is where a lot
of that optimization know-how comes in.

Of course, there are SLMs that
are continuously coming out.

SLMs being Small
language models.

Yes, exactly.

Exactly where you
draw that cutoff,

I think is an open question.

But at the end of the day,

as compute continues to
increase and especially neural

computing GPU type of
capabilities in the vehicle

ECUs, as well as the model
compression and bringing down

or compressing the capability
of those traditional

foundational LLMs into
smaller and smaller tools,

I think there's a convergence
that we're going to be seeing

in the next couple of years
where these tools in the

vehicle will become
a lot more capable.

Excellent.

Let's talk about Sonatus
and Sonatus's role in

this evolution.

I know, we are, a leader in SDV

technologies and solutions.

How does that play into the
things that you've talked about?

Yeah.

So Sonatus has a number of
different products that kind of

fall into these
different categories.

For us, it really starts
around our Foundation product,

which is focused on the EE
architecture in the vehicle.

It's basically dealing with
controlling the network traffic

and ideally dynamically
changing aspects of the network

in order to optimize for the
particular use case or the

needs of the of the
vehicle configuration.

There's also our Collector
product which basically

enables customers to collect
data from the vehicle kind of

when and as needed.

So rather than having to get
everything or get nothing,

I can get exactly what I want
when I need it and no other time.

And those really are key to
being able to deploy AI in the

vehicle along with
the next step,

which is really when a customer
wants to take action in the vehicle.

So we have an Automator
product which enables configurations to

say when certain trigger
conditions are true in the

vehicle, then I want to

do some kind of actuation or
send some signal elsewhere in

the vehicle for consumption.

And that's one of those things
that with the advent of AI

models being deployed
into the vehicle itself,

now the spectrum of possible
outputs or vehicle signals

that you might use to
communicate one place or

another can expand quite a bit.

So you can actually
use the outputs of

an AI model in the vehicle as
if it was a generated signal

in the vehicle and consume it
elsewhere or send it back to

the cloud for other purposes.

Almost like a true agent. Right?

It's connecting
the virtual or the

digital to the physical.

Definitely. Absolutely.

And then with, you know,
AI in the cloud, of course,

on top of that, you can
run some more intelligence.

So things like those LLMs are
a bit more human-like in how they

behave and how they think.

Now you start to add that
capability to start to diagnose

things that are happening
in the vehicle and take more

intelligent actions.

Yeah. So that's where we
see the direction going.

Sure.

And then of course, we have
Updater, which is our own OTA.

So that plays a
role in, you know,

updating models throughout
the vehicle lifecycle.

Definitely.

Not to be left out given
short shrift, by all means,

OTA solutions like Updater are
really important for bringing

that model to where it needs to
run in the vehicle as well as

to offer that containerized
environment to protect all the

other operations taking place.

Well, that was a very
informative discussion, Steve.

I really enjoyed
our conversation.

Thanks again for
stopping at the podcast.

Thank you, Sanjay.
Appreciate it.

Well, there you have it.

We are looking forward to a
future where vehicles are not

just, platforms for
mobility innovation,

but also for
AI-enabled innovation.

Stay tuned for more episodes
where we will dive much deeper

into the intersection of SDV
technologies and AI and how

that's truly going to transform
the future of mobility.