Subscribe
Share
Share
Embed
"The Edge AIsle" brings you to the forefront of artificial intelligence and edge computing, powered by Hailo.ai. In this podcast, we explore how edge AI is reshaping industries from smart cities and intelligent video analytics to autonomous vehicles and retail innovation. Join industry experts and thought leaders as they share insights into how Hailo’s AI processors are transforming the way industries function, enabling real-time deep learning and AI-driven applications directly on edge devices.
This isn’t just a podcast about technology—it's about how AI is empowering industries and improving lives across the globe. Whether it’s smart cameras making our cities safer or AI accelerators driving innovation in autonomous vehicles, the possibilities are endless.
If you're passionate about the nuts and bolts of AI processors and how they integrate with edge platforms to deliver unparalleled performance, "The Edge AIsle" is the podcast for you. Expect detailed analysis and a peek behind the curtain at the future of edge computing AI.
00:00:00:01 - 00:00:15:20
Welcome to another Halo AI podcast, bringing you innovations and insights into AI on the edge. Now let's get started.
00:00:15:20 - 00:00:17:06
thank you for having me.
00:00:17:06 - 00:00:18:06
My name is Yanni.
00:00:18:06 - 00:00:20:08
and I lead automotive in Halo
00:00:20:08 - 00:00:35:18
in the next 25 minutes or so, I'm going to talk about the challenges of scanning in our locomotives. What are the exactly the challenge. And what are the different ways to cope with it at the silico level?
00:00:35:18 - 00:01:02:18
Because at the end of the day, we all know that silicon today's powering AI and automotive and is a semiconductor company. We're happy to offer some of our thoughts on this topic. So moving on. A little bit about us. We are, a fabulous chip maker based in Israel. We founded more than five years ago.
00:01:02:20 - 00:01:32:00
Our first generation is already in mass production. More than $200 million in funding, from both financial and strategic, investors. As mentioned, we're headquartered in Israel, but we have offices pretty much where the automotive industry is, over 200 people today. And we have a growing worldwide partner ecosystem, including many partners in automotive.
00:01:32:02 - 00:01:59:11
So having said that, looking a little bit about the idea of automotive, and I chose maybe the most well-known, domain, which has grew, I don't know what it, what it is today, which is both, Adas, the driver assistance systems, which I believe were responsible for the boom in, in AI, in automotive to begin with, as well as automated driving.
00:01:59:12 - 00:02:25:23
So looking at these two, which many times are combined hand in hand, I think it's shown in the previous presentation by moment that there are some unique challenges related to automated driving. So first, if we look to the Adas, he does is become definitely a market, which is a large market which is, dominated by certain, features that are required.
00:02:26:01 - 00:02:54:18
It needs to comply with specific regulation. It is definitely cost driven. And in comparison, when we talk about automated driving, we see it's mostly performance driven. Today we see an order or two of magnitude performance, in performance more than us in terms of requirements. Regulation is just starting in many places. And it's not well regulated yet.
00:02:54:20 - 00:03:27:10
So cost has not become an issue yet. Of course, we all know it will be. And we look at some of the other, properties. We see that, the, amount of sensors is higher in automated driving. We see that that's something which is pretty clear, automated driving, like a comfort features for the for the, passenger are are becoming more and more common in Adas as well.
00:03:27:12 - 00:03:59:04
And in general, we see that while these two are different as an industry, we need to find the path to scale. For me, just to add, in spite of all these challenges. And one thing that both have in common is the fact that the AI based perception lies in the heart of both, today, and we need a way to scale efficiently between the two, in order to really have a roadmap path toward automated driving.
00:03:59:06 - 00:04:27:17
And I think as an industry, we understand that, we don't have a direct path to automated driving, as maybe some four years ago. The path is gradual. And if the path is gradual, we really need, a scalable will to to to reach it. So moving on. Let's take a look at a very popular term in, in AI, which is tops.
00:04:27:21 - 00:05:03:05
So everyone are, putting their requirements out in terms of what are these tops, which is trillion operations per second and are required for processing for positions in the car. So taking a very, rudimentary basic example about a sensor that everyone would agree is needed due to the cameras. If we look at the number of cameras, that are needed, and then we multiply that by the frame rate of these cameras, frame rate gives us the temporal performance.
00:05:03:05 - 00:05:44:20
Meaning how sensitive will these sensor be for real time changes in the, environment? Indeed. Vicinity of the car. So how fast would we notice those changes? Then if we look at resolution or pixels per frame, that will determine our special performance, meaning how sensitive we will be to small objects around the car. And then there is the element, which is how well this model is designed for the purpose or, or group of models is designed for the purpose that we want to achieve, which is, accuracy.
00:05:44:21 - 00:06:17:06
So in how many, in how many frames would we identify a pedestrian as an example? That's the model accuracy. So multiplying all these factors, we get to a number, which is our requirement. The challenge with that number is not only that resolutions change and frame rates changed is that in the AI world today, we see we we are it's a very fast paced world with a lot of advancement.
00:06:17:06 - 00:06:43:22
And we see models. The just yesterday we had models which achieved accuracy a certain accuracy with and required performance ex. Today they can achieve the same accuracy. Well well consuming the half of the performance as an an. And as an example, in my next slide, I'm showing a wave of, meta architectures called transformers, which are now making their way into automotive.
00:06:43:22 - 00:07:13:01
They're not even automotive. The day to day locomotive. We still have more classical neural network, object detection. But as an example, if we look at Vit, which is short for visual transformer, we can see that for for the same level or for for the same level of compute, it can achieve higher accuracy or it can achieve the same accuracy with less compute.
00:07:13:03 - 00:07:44:21
And this tells us that it's that in AI, it's not just about the, the, the tops or the, the requirement for, a certain level of compute. It's the ability to leverage new, AI architecture or meta architectures that are, that we see that are becoming more and more efficient. And, and when utilizing them, we can actually lower the compute we need in the system.
00:07:44:23 - 00:08:15:08
Continuing, going to the next slide, continuing the view on on what we see in factories in automotive. So definitely when it comes to hardware, we if we look at compute performance in automotive development cycles, which are roughly, 30, 30 to 36 months, we see that roughly it has performance needs have doubled in every, two and a half to three years.
00:08:15:08 - 00:08:46:13
Performance needs have doubled. And that tells us that, of course, both power and cost efficiency are because if we keep doubling our cost and power, we have a huge challenge. And also we see that hardware is becoming more and more centralized. So while we had scattered issues in the car, each one having its own AI, now we see zone controllers, and we see centralized domain controllers.
00:08:46:14 - 00:09:17:19
And that means that we need to we have a range of performance that we need to meet and not another single or, or when it comes to software, openness and flexibility, have become are becoming more common. And, and if we want to allow for controller innovation this world then of of of AI in automotive then obviously, openness and flexibility are very important.
00:09:17:21 - 00:10:09:02
We also know that the software is a major challenge and has been a barrier to entry for many players into this, into this market. And obviously the desired paradigm is, is to develop the software once and run it across multiple platforms. And in terms of the world of machine learning, the world of AI, we see advancement in the neural network models, and we see many tasks that were done with classical, computer vision tools are increasingly being done with AI, and we see that the eyes kind of on a class of its own in terms of compute and has grown and changed more in recent years than other classical forms
00:10:09:02 - 00:10:41:02
of compute, like DSP and CPU and GPU. And as I mentioned, I think one of the clear example here is Transformers, which, is a new MIT architecture for AI, which will will be very dominant in many perception tasks. Going forward. Moving on. Talking a little bit more about software. So, we all know the software is, is today a core competency in the auto industry?
00:10:41:04 - 00:11:17:03
And we all know that software requires a lot of resources. But there are some specifics. When we look at AI so far in automotive, first of all, reuse. So reuse is crucial if we want to see return on the software investment that we making in the eye and on the motors. And, and and as mentioned, since neural network models are changing, and, and and the ecosystem is evolving, flexibility is very important.
00:11:17:05 - 00:11:48:03
And last but certainly not least, many times when we talk about scalability, if we think about it as a hardware requirement. But that's far from the truth. For them to the scalability is just as much a software requirement as it is a hardware requirement. The ability to scale in a linear or sublinear fashion when we scale up our number of sensors or multiple or taking on more AI tasks is, equally as important.
00:11:48:05 - 00:12:22:21
In so, so looking at what we said so far, let's draw some conclusions. The, the challenge of, of an automotive. So we know that Adas is cost driven today and it's a volume product. So we need an efficient way to scale to different levels. We've also discussed the fact that automated driving is a big lead for us, and that scalability will eventually need to get there.
00:12:22:23 - 00:12:53:22
We talked about the progress, in AI and the fact that we will need hardware capable of running the latest and greatest neural network models. And we just talked about the software investment, which is which is really significant, when it comes to a lot of money. So the bottom line is we need a solution which is scalable, which is, which is AI centric, which is centered around AI, and it is flexible in the future.
00:12:54:00 - 00:13:23:15
So these are these this is the conclusion from our discussion. And now in the in in the next part of the presentation, I want to look at it from a silicon perspective and see what are the different options we have to get to a solution which answers this criteria. So when we look at the semiconductor world today and where technology is heading, we basically have three key ways to achieve AI.
00:13:23:15 - 00:13:53:15
How, you know, the more the first one, is monolithic design. So these are some of the, myths. Many of the, SoC players in automotive take AI IP, which is either homegrown or off the shelf, IP, and they embedded into their, designs for, an sec that will go into different, applications in the, in AI.
00:13:53:17 - 00:14:29:09
Will go into Adas, we go to automated driving and other applications requiring AI to be. So this approach, of course, is great because it's a single, it's a single vendor from, from a, the hardware perspective. It's a complete solution. But it is also challenging in the scalability realm because when it comes to monolithic design, you will need to do, you need a wide portfolio of devices to scale from the low into the AI and, it is it is very challenging.
00:14:29:11 - 00:15:02:11
And, and cost to achieve that is very high. And there's the danger from the, OEM perspective or the lock in scenario that when maybe design vendor, then technology automotive is, marching forward and we can look at, chiplet models becoming more relevant, meaning that a die, a basic die of AI for vendor A can go into the associate vendor will be and will be codified in Productized, by them.
00:15:02:15 - 00:15:32:00
This model offers much better scalability and then a monolithic design. But there are some challenges when it comes to qualifying and prioritizing these products for model. And then the third option is to separate the silicon to dedicated silicon for AI and dedicated silicon for other means. This is definitely the approach which has the highest scalability, but it requires multiple device.
00:15:32:02 - 00:16:07:07
So continuing this comparison, on the next slide, here are some kind of, I would say Marquese to what are some example product targets that we see today in the market for monolithic design, for device level integration, for platform level integrations with some target performance numbers that some of you may argue with. I know it's a hot topic, but this is, kind of, an example, representative example of where this can be.
00:16:07:07 - 00:16:48:18
And of course, the, some of the device level integrations or skew technologies are data into making. So this might be more relevant for design in the second part in the second half of this. Thank you. And in the next slide talking about, how we view the proposed solution, we think that in light of what we discussed and the need for flexibility and scalability, we need the dimensional approach which separates the AI dimension from the other dimension of processing in the B.
00:16:48:20 - 00:17:23:18
For that, we need, a creature that we would call automotive grade AI accelerator that accelerates AI independently from other, compute needs. It offers scalability because you can grow or shrink it as much as needed. It is very powerful and can feed, something this use, it needs to be cost efficient, of course. And and even though we have multiple devices, it needs to be cost competitive versus some of the mega sources out there.
00:17:23:20 - 00:17:50:09
And it needs to offer an open software ecosystem and allow for, software control or innovation by the OEMs into tiers. At present, this solution is a chipset, meaning an Ada sesso C and, an accelerator in the future. In the near future, we view it as a chip. That approach meaning a few days that can be packaged as a single silicon.
00:17:50:11 - 00:18:21:17
And but the idea is there should be a single source or investment behind the solution, whether it's a chip set or a LED, that allows monetizing the software investment that is done in this architecture. So this is our our view of the of the desired solution. And now I'll move on to the next slide, and talk a little bit about to summarize with what we do.
00:18:21:19 - 00:19:00:01
So we are very focused on AI performance and doing it, doing our facial, and doing it power efficiently. And moving to the next slide, just mentioning that today we have a this actually, chip has been, out in the market now for more than two years. It's a 26, tops AI accelerator. It is focused purely on AI, and, does not, include other other compute.
00:19:00:03 - 00:19:53:12
Elements. And it is self-contained and can accelerate, the neural network. But the full perception, essentially, models on its own. Moving on to the next slide. We are following the concept, which I just mentioned. We already have reference designs with two major. The is both Renesas and NSP in the same, in the same manner that I mentioned, which is, combining an automotive Multivessel SLC with, powerful AI accelerator today because it is as a chipset approach which allows for which allows for application many hardware which has a dedicated dimension for AI.
00:19:53:14 - 00:20:33:15
And moving on, what we have today is designer device. We have different platforms with multiple, both single and multiple chips for evaluation and, and, and product ization, purposes. And just summarize, the next slide by saying that we believe that, an AI acceleration approach, and a path that includes, separating the AI dimension from the other dimensions allows for one architecture and a seamless hardware concept.
00:20:33:17 - 00:21:21:17
It is scalable. It allows for open source platform, allowing receive reuse, which is a crucial point today since some of the other semiconductor there is. Yeah. And because, all software platforms which are not completely open, which might limit controlling innovation, which we think, is not the right way to go, and a long term hardware concept from driver assistance to automated driving, as mentioned, which starts with chipset and completely with Chiplet and specifically tailored, we are focused on building the AI dimension, and making it power efficient and scalable and high performance.
00:21:21:19 - 00:21:26:22
Thank you very much.
00:21:27:00 - 00:21:51:05
Any things you, if. And we don't have questions in the chat, in that case, I jump in, if I may, if you can, if you would choose your your pitch unit of measure for useful work done on a chip, how would you describe the efficiency increase? You're nodding. Or do you think you know what I'm worried about?
00:21:51:05 - 00:22:24:14
If. Yeah, very good question. So, if you've seen my presentation, you know that I would not measure it, measure it in tops. I would measure I would take, a real benchmark, a real workload, and saying, okay, and running this neural network model for it, object detection, like a pedestrian detection at this resolution. How many sensors can I fit on this AI chip?
00:22:24:16 - 00:23:14:17
And and what would be the power consumed that would be a metric which I think is much better, for benchmarking performance between different, solutions. If you can advertise yourself a bit, how how does your product perform that is allowed to say. Yes. So as a, as an example, if I take, one of the best, or today one of the most accurate object detectors out there from the Euro family, like a yolov5, object detector today, our Halo eight device, you can fit about, seven automotive sensors at 30 frames per second.
00:23:14:19 - 00:23:50:08
Running this model, on it at around three, three, 3.5W. As an example. So giving numbers. This this is an example of real, real, reliable, real world performance. Okay. Thanks a lot. Meanwhile, I encourage the audience to type some questions into the chat. And we do have some minutes I think, because then I will ask again. You were talking about neural network scalability.
00:23:50:13 - 00:24:15:11
If I, if I got that right and then you've connected it to, the number of sensors, for example, if you scale in terms of sensors, can you explain what you mean? Just because there are people who say that one would try to stay away from sensors and keep the dependency on, for example, if the AI is a pilot, it should not care too much about the sensors.
00:24:15:13 - 00:24:54:05
I get this right if the AI is what? Sorry if the AI is that the AI shouldn't be too dependent, should not be unnecessarily dependent on on sensors. The number of sensors, the type of sensors is. Yeah. AI is, for example, the autopilot, which ideally performs, anywhere. Yeah. So the, sensors do, I think provide, an important input because they, they are the basis for, for any, model that we run.
00:24:54:05 - 00:25:25:16
So the, the that the, their input is important. I didn't try to say otherwise. It is very important. The the the the scalability has many, many dimensions. And the number of sensors is one of them. But if we take as an example resolution, then our aspiration system planners would be that the performance would scale linearly in resolution.
00:25:25:16 - 00:26:02:05
And when we multiply the resolution, we will, scale up in a linear fashion. In real world scenarios, it's not always the case. And sometimes when we multiply the resolution, the the required performance is more than x. And as an example, if that's what you meant, you question, I think a requirement should be this is the scalability that when we scale up resolution, the, performance of of the device in question would rise in the same proportion as the resolution.
00:26:02:07 - 00:26:07:19
As an example, regardless of the sensor, if it's a camera or lidar or ray.
00:26:07:19 - 00:26:33:02
Thank you for listening to the Halo AI podcast. If you enjoyed this episode, don't forget to sign up and check out more information at Halo Dot. I keep the conversation going by sharing this with your peers and never stop exploring the future of AI.