Welcome to WebAssembly Unleashed, your monthly dive into the dynamic world of WebAssembly (Wasm). Join F5’s Joel Moses, Oscar Spencer, and Wasm enthusiast Matthew Yacobucci as they unpack the potential, challenges, and innovations within the Wasm ecosystem.
Designed for architects, practitioners, technologists, and Wasm enthusiasts, episodes offer:
Insightful discussions on Wasm advancements.
Practical tips for seamless integration into projects.
Interviews with influential figures shaping the Wasm landscape.
Strategies for maximizing the potential of WebAssembly.
Subscribe now and stay abreast of the latest in Wasm development. Whether you’re a seasoned pro or new to the field, WebAssembly Unleashed is your ticket to unlocking the full power of this revolutionary technology. Tune in for a deep dive into the limitless possibilities of WebAssembly.
Joel Moses (00:04.061)
Hello and welcome to WebAssembly Unleashed, your source for news and views on your very favorite open standard. Our goal is always to help folks understand what's going on in the community, highlight valuable work, and try to translate what this super cool technology means to the world. But because we're not just fans, we're also practitioners in the polyglot world, we'll spice it a little with some red-hot opinion. And so what could possibly go wrong with that? We've got a great topic and a special guest to dig into a huge piece of news for folks working on graphics and UI.
But also for people doing work in those two letters that seem to be everywhere these days, AI. Today's topic is interesting to me, and it tells a very human story about technology standards and their growth over time. When working on something that has a wide industry impact, one of the most difficult parts is deciding which interfaces should become infrastructure that is well defined in lockdown, and which should remain experiments, fast, flexible, changeable, and fast moving.
but the discussion between the lines sometimes can blur between friendly combat and political intrigue. with me today is my co-host, Matt Yakabuchi. Hello, Matt. How are you doing? pretty good. It's been a little while since I've seen you, but welcome and let's get into it. yeah, on June 7th, our guest today was co-author on a blog post that laid out a map between two worlds, graphics and GPUs, and put a pin in this discussion. Mindy Berger is a software engineer and open source contributor working at the intersection.
Matt Yacobucci (01:07.173)
Good, good. How are you?
Joel Moses (01:26.641)
Of WebAssembly, Graphics, and GPU computing. He's a contributor to the emerging WASI GFX graph ecosystem and the renderlit project, if I remember correctly, helping bring portable Web GP capabilities to WebAssembly components that can run across browsers, cloud platforms, and edge devices. And his work at Cosmonic is helping shape how graphics and GPU acceleration fit into the future of the WebAssembly component model. So Mindy, welcome.
Mendy Berger (01:52.12)
Hi, welcome. Thanks for having me.
Joel Moses (01:54.77)
Alright, so yeah, so I wanna I wanna start with the the blog post from from from earlier last month. Wa can you explain to me first of all WASI GFX and and the deprecation of some of the graphics function or the roll up of some of the graphics functions into that and the separation of what web GPU. w what's what's the rationale there?
Matt Yacobucci (01:54.907)
Yeah, thank you.
Mendy Berger (02:19.5)
Yeah, sure. So
We were going initially to do everything under WASI and that would include both Web GPU and a way to actually display to the screen. In the browser, the way this is done is by using the Canvas element. We don't have Canvas, we don't have HTML inside WASI. So we were trying to do our own thing. but since we were de d developing designing an API from scratch, it simply doesn't have the polish that Web GPU comes with Web GPU, even though it's new in browsers, it's been in development for
Quite a few years. I don't remember the actual years started, but like quite a few years. So there's a lot more polish behind Web GPU and it is a lot more ready to be to to be a standard that is not changing or rarely changing, and is mostly at just besides adding new features, you can't really break anything. that is not the case for our surface side, and that's what what we put inside the
Matt Yacobucci (03:09.561)
Okay.
Mendy Berger (03:20.494)
cfx that is the display side of things and that's a lot more experimental still and we're still figuring things out feeling things out seeing what works and what doesn't so that's why the separation just made sense so that web gpu can progress there are lots of use cases web gpu make sense without a without a displaying to the screen at all so that's why we separated them out
Joel Moses (03:43.825)
Got it.
Matt Yacobucci (03:46.608)
Are there's how how significant is the overlap? Are they are they significantly overlapping or or are pretty separate?
Mendy Berger (03:56.145)
there is definitely overlap and it's impossible to do it without it. Like they they both are about how do you display things to the screen. but they're mostly separate, I would say. There is there is a world if people familiar with the concept of a a component model world, there is a world that has both of them and adds a few more things that are like connection points, but for the most part these are separate interfaces that don't actually depend on one another, just only the world that depends on both of them.
Joel Moses (04:25.521)
Got it. Well let's focus on WASI Web GPU for a moment. So can you describe for me what problem WASI Web GPU solves? I mean there there's there's a n a number of different use cases. Some of them are even non-graphics related, correct?
Mendy Berger (04:39.168)
Of course, of course. So Web GPU is to the GPU, what WASM is to the CPU. So the same way WASM gives you close to native speed without actually being native, Web GPU gives you close to native speed without actually being native GPU code. the same way WebAssembly is portable across operating system, across CPU architectures, the same way Web GPU can run on any operating system.
any GPU w whether you have a beefy NVIDIA card or you have a low end mobile or e GPU, Web GPU will just run and not care where it runs. And it also gives you isolation because it's coming from the web and you don't know what people are running in their websites, what they're embedding in there. And Bosm has the same problem, right? it has to be completely isolated. And Web GPU also has to deal with this, where it has to isolate each workload.
and it does that different ways than Wasm does it, but it gets the same result on that different you can run different workloads side by side on the same device without them interacting with one another. So yeah.
Joel Moses (05:50.932)
So yeah, so for people who don't who don't really know, why why would someone, for example, need GPU access in WASI to begin with? What what would they do with it?
Mendy Berger (06:03.15)
So the same reason you would want GPU access anywhere. So you could do it for graphics, as we discussed before. You could do it for AI, which is like the big thing that everyone thinks about right now. You can do it for a parallel compute, a large like scientific compute. so there are a ton of reasons why you would want GPU access anywhere. And if you want to have them isolated in a multi tenancy way, that is
Web GPU is your is your friend.
Matt Yacobucci (06:35.077)
Would so i if this was a couple of years ago, I'd I'd say maybe your primary use case was for display, was for rendering things on the web, perhaps. But are all those use cases equal tier, so to speak? Like are are are you focusing as much on inferencing and graphics and parallel compute like like equally or or is or are there specific use cases that
That are really top of mind.
Mendy Berger (07:06.796)
I I don't think there's any prioritization in that on that way. the same way WASM doesn't care what you're running in it. It just tries to give you access to do what you feel you need to do. And it's up to the developer to decide like how they want to use this. and there are all kinds of interesting ways you want to use it. the one thing I would add is that this d this wasn't from us, this was like from the Web GPU spec. They recently added float sixteen support. They used to have only th thirty two bit floating points, now they have
sixteen bit floating points. And that was specifically because AI started to become so popular. So you can see that there's some movement from things that that people need and like AI is a big thing now. but that wasn't actually on us, but that from us. That was like from the Web GPU spec itself.
Matt Yacobucci (07:46.299)
Mm-hmm.
Joel Moses (07:52.498)
Right. Now can you describe for me how this fits into the WASI component model? obviously that's that's something that that we're watching pretty closely. I know that one of the plans for WASI Web GPU is to to leverage some of it for like async support, but how how does how does WASI Web GPU fit into that?
Mendy Berger (08:12.382)
yeah, sure. So we make heavy use of the component model. our width interface is more than a thousand lines of width. And that is probably one of the biggest interfaces out there. And the reason we feel comfortable with that is because it's based on such such a rock solid API designed by all three browser vendors. it's more than like three there are three engines, but there are more a lot more vendors that actually are are involved and that have been involved and and are still actively involved in the design process. So we feel very confident about it.
But yeah, it's a lot of width we're making use of all the the width constructs resources are very useful in in cases like this. If you want to handle to a buffer, that's that maps on pretty nicely to component model resource. flags, enums, like all these things, a lot of them don't even exist in JavaScript in the same way, but we can make pretty good use of it.
Matt Yacobucci (09:03.825)
So that that makes me think, are are are you pushing the boundaries of of wit as well to to like a thousand lines. A thousand lines is a pretty big wit file from what I've seen. So like what what design decisions or or how have you had to update or changed wit to to to work in this model?
Mendy Berger (09:24.844)
Yeah, well we have come across more than one such case. one of them being bitwise flags, for example. So in JavaScript you don't have bitwise flags. you just in the way WebGP wanted to use bitwise bitwise flags because it's just one number. You can have a bag of bulls and one number, that's pretty performant. so the way they did is just had a bunch of c they created a bunch of constants and you just or them together, bitwise or them. That works. And that is not very unlike what C does.
But in in in width, there's a flags construct. And we wanted to make use of that. And that's great. The problem is you can't in the current component model spec, you can't specify your own bit pattern. And the WebGP was very specific explicit about which bit pattern it wants to see. So that is a limitation we ran into with. We opened an issue, of course, and we hope to get that resolved. But for now, there has to be a conversion between them, which degrades.
Joel Moses (10:03.443)
Mm-hmm.
Mendy Berger (10:24.002)
bit a little bit of the performance. It's not too bad. It's a number conversion. But it's like one of the things w where we ran into ran into like limitations of width because we stressed a lot of corners.
Joel Moses (10:25.427)
Mm-hmm.
Matt Yacobucci (10:35.107)
Mm.
Joel Moses (10:36.817)
Yeah, it d it doesn't surprise me that there's a lot of corner cases which led to the expansion of the wid. I mean anybody who's done graphics programming or worked with GPU primitives knows that sometimes it's down to the right graphics driver version with native applications and sometimes it's down to whether you have to sacrifice a goat to NVIDIA to make it all work.
so so so now that now that you know the wits there and and now we have something that's kind of rock solid, stable and and is is doesn't change too often, what what do you see are the killer use cases for for WASI Web GPU?
Mendy Berger (10:56.805)
yeah.
Mendy Berger (11:10.946)
So of course everyone thinks about AI. that is the big thing now. So let's start with that.
you can run AI. Most of AI nowadays runs in GPUs. So if you try to run in a CPU, it's going to be pretty slow. So we're trying to like lean into that, make sure that the tools are around, the tools for that are around. so one of the things we're actively working on, I have a loc working POC of Llama CPP running that on inside WASI completely and with very minimal changes because they already had a web GPU for in browser.
Browser backend. We just ported that to WASI. It wasn't too hard, thankfully. and we hope to to I hope to upstream the changes into Llama CPP soon. another one is Onyx runtime, which is kind of like the cross-platform way of doing things. it has like it can it can take, you can compile
if if you take models from diff that are written against like PyTorch and TensorFlow and all the other frameworks and you can compile them to Onyx. so Onyx runtime we had a demonstration with Adobe from c with Colin Murphy at Adobe where they have this AI model that
will watermark an image and that is meant for like AI so that the AI can watermark it and you can see, okay, this is AI generated. And it can watermark images, videos, and of course you need a GPU for that, because that's a lot of pixels to to to run through.
Matt Yacobucci (12:39.471)
Mm-hmm.
Mendy Berger (12:41.914)
And yeah, w we together sh showed that this is possible and we compiled Onyx runtime to Web GPU to WASI Web GPU and ran it through that and we had a a very significant performance boost from that. So lots of
Joel Moses (12:58.993)
Yeah, that's pretty cool.
Matt Yacobucci (13:00.197)
With with respect to some of these workloads, like how much how much vendor buy-in is is necessary? Like for instance, okay, I'm sure like AMD, NVIDIA, Intel, like are are pretty popular choices, but what about like, you know, for some of the smaller embedded commute computer community, halos or or other sorts of NPUs like on on like Orange Pies and things like that?
Mendy Berger (13:28.464)
so this is GPU and not NPUs. there's a separate Web N and WASI NN thing. you if you want to dive into that, but
Matt Yacobucci (13:36.934)
okay yeah.
Joel Moses (13:40.465)
Let's let's do that. Let's compare.
Matt Yacobucci (13:41.868)
yeah. Yeah, yeah, yeah. Because I think I I I think I just like, you know, stepped on my own feet. So definitely let's let's do it. Yeah.
Mendy Berger (13:47.183)
Yeah, okay. So WASUF GPU specifically with GPUs. what these NN framework backends interfaces try to do is say I don't care what device this is gonna run on, whether it's CPU, GPU, or NPU, which just tell me what you want to happen and I'll I'll make sure that it runs on the best device, the most efficient device that I have available. so in WASINN it can have a the way WASINN
does it and spicy take I don't think is the best way. The way to do it is basically tell ask you which framework are you writing this again? Is this Onyx runtime model? Is this a PyTorch model? Is this one of the others, but I don't remember the actual whole list. But they have basically an enum of supported of supported backends and you just provide them with your model and then you can do inferencing with that. I personally don't like that. I don't think at
It doesn't feel like very WASM-like. I I I think we should have higher level the NN interfaces, but I don't think that that's the best design out there. it's kinda like having a back like an interface that says, give me your language, give me tell me give me a string of Python or a string of JavaScript and I'll execute it for you. Like that doesn't feel like very wasm like.
There's the other, which is WebNN, which is a lot more interesting. and they're doing it a lot more like Wazin did it, where you they have a bunch of opcodes that they support, they call it operators, and you build up a graph, and then you can execute it. And that is not tied to any back end, like you can compile a TensorFlow model to that, you can compile any of the others to that, and then it executes on the best.
available hardware. So I think that's a better I think it's a better abstraction. And not only
Matt Yacobucci (15:45.699)
I see.
Joel Moses (15:47.367)
Yeah. Yeah.
Matt Yacobucci (15:49.69)
And that's and so that's Web sorry, I'm s Web N N. and then so I take it like WASI or the GPU project has created their own opcodes, so you're doing it more along those lines, or d were you able to use all the opcodes that or let me ask this a separate way. were you did you need to add anything to the WASM standard?
Mendy Berger (15:55.31)
Well then.
Matt Yacobucci (16:19.469)
in order to open up GPU support.
Mendy Berger (16:23.982)
No, but maybe maybe you can talk to like what I think you're getting at. so so there are app codes that are available for like WAS. There are other app codes that are available for Web GPU. If they're called opcodes, it's a little different, it's a higher level language, but it's essentially the same thing. Then there are other app codes that are available for WASINN. Sorry, for WebNN. So
Matt Yacobucci (16:28.838)
Yeah.
Mendy Berger (16:49.322)
WebNN is actually the most the most limited because it tries to run everywhere and MPUs are actually pretty simple in the way they operate. but the way they if you want to extend that, the way they encourage you to do it is to do it with Web GPU. So you can have you can mix them, you can import a buffer from What WebNN to Web GPU. So WebNN is a lot a lot younger than Web GPU. It's not nearly as stable, it just exists behind a flag in Chrome only. Like it's not it's
Joel Moses (16:54.291)
Mm-hmm.
Joel Moses (17:11.272)
Right.
Mendy Berger (17:19.056)
It's not nearly where WebGPU is. Web GPU is basic is available at least in some platforms in every single browser. So yeah, WebGPU is a lot more mature and a lot more robust, but they complement each other, they're not really competing.
Joel Moses (17:28.401)
Yeah. Now
Joel Moses (17:35.762)
Yes. It it it strikes me that, you know, that right now the AI world is kind of struggling with the overwhelmingly massive cost of token generation. you know, a lot of a lot of the the the foundational models run on large GPU complexes that are centrally located and are built to to have multiple tenants use it use them simultaneously, but they compete with each other.
and it also strikes me that the main delivery model for s for a lot of these AI models is is through the web. There's a web client on the other end that's interacting that has a GPU, right? And it it strikes me that perhaps WASI might be the glue necessary to create local first structures for token generation.
what do you think about that? Is it is a cooperative approach possible? Is is is something like that within the scope of of of WASI Web GPU?
Mendy Berger (18:33.874)
yeah, 100% it is possible. but what you described I think is possible today without WASI. just run it locally. But what WASI brings you is the same workload. You can have literally the same thing. You try to run it locally, maybe it falls over on each on itself, maybe it maybe the network like maybe the network is good enough that you don't care. You want to extend it to the data center, like you can have literally the same thing, the same code.
And you just transition between the browser and the server and different servers, like it doesn't care about what hardware you have on your back end. Nvidia GPUs are hard to get now. But if you only have AM AMD GPUs, you could it doesn't care. It really doesn't care.
Joel Moses (19:13.203)
Right.
Matt Yacobucci (19:15.161)
Okay.
Joel Moses (19:16.307)
Yeah, and your your your GPU on your client might be something that's Intel embedded and the one at the center might be a you know, an H series from from NVIDIA, but but because there's an abstraction, that becomes a a usable and transferable workload. Now, thinking about the use cases for WASI Web GPU, are there use cases for which it would not be an ideal choice? Have you have you thought about that?
Mendy Berger (19:39.849)
there pr d there are lots of them. training might not be the best case, the not the best use case.
Because training is usually happening on you know exactly where it's going to run, what it's going to run, and you're probably gonna do CUDA and you're gonna have an easier time there. it is not like it's going to run in the wild where you're not sure what you'll have available. So training might not be the best use case. I'm not saying it's not possible, it almost certainly is, but you're just gonna have a better time doing CUDA. but for things like inferencing, this is becoming more and more of a very viable path.
Joel Moses (20:19.067)
That's great. Now let let's talk about where we are in the specification process. So we've got WASI GFX and we've got WASI Web GPU. what where are we between those two? Where are we in the standards process?
Mendy Berger (20:32.834)
Yeah, so WASI GFX is not in a standards process. It's n it's not the goal is not to standardize it anytime soon. the mental model I I have of it is like there's the standard library of a language that is usually very stable. It's not gonna evolve that much. Maybe just add features. and there are libraries on NPM and creates IO and
pip and whatever wherever you have packages, those constantly rev versions constantly change as they add new features. So that that is the mental model I have. Like Web GPU is the stable thing. It's not gonna change that much. We feel feel very confident about the the base it's bait built upon. Wazi GFX will need a lot more revision still a lot more cycles until we feel like it should be a standard. So for now that's still going to rev a lot. So
WebGPU is the WASI WebGP is the only one that's actually on standard struck. it is in phase two. and recently converted to using P3 and Async. So we got rid of this P2 hack that was called WASI IO. whoever tried using that, that was not fun, but it was a hack to get yeah, it was a hack to use async and p2. We're done with that.
Joel Moses (21:46.215)
Right.
Joel Moses (21:49.971)
Yeah.
Mendy Berger (21:55.811)
We it's completely standalone. We have no dependencies. and we have two implementations. one in Wasmtime. It's not it's not upstreamed in Wasmtime yet. we have to do that soon. and the other one is with JCO in the browser. So you can run it natively or inside the browser, same workload, it doesn't care.
Joel Moses (22:15.111)
That's really cool. That's really cool.
Matt Yacobucci (22:16.929)
So so it's still it's still being worked on in Wasm time. so server side workloads need a little bit of work, but you're saying it's it's usable it's usable in the browser now. Is is that what I heard?
Mendy Berger (22:31.112)
no no it's not upstreamed in Wasm time, but you can connect it from the outside. So like actually Wasm Cloud, which is our product at Cosmonic, that ships WebGP support today, right? It's been s it shipped like a couple months ago. It's been having support. So yeah. It's just not it's just a separate project for now, which first of all like not for strong reasons. We we wanna upstream it.
Matt Yacobucci (22:36.793)
okay.
Matt Yacobucci (22:46.804)
exciting. Excellent.
Joel Moses (22:47.506)
Yeah.
Joel Moses (22:51.505)
Yeah.
Joel Moses (22:55.695)
It got it. Any updates on passing the conformance tests?
Mendy Berger (23:00.422)
I didn't know like y you're asking for updates as if like there was a
Joel Moses (23:05.523)
Yeah, yeah. Where I mean I I I wanna know but you know, where where are we? C can we use this tomorrow?
Mendy Berger (23:12.538)
okay, so one of the goals of mine of late is to get the WebGPU spec tests to pass on YZ Web GPU. So the way you would do this, the spec tests are all written in JavaScript. so we need to compile our j the JavaScript to Wasm and it does become like a gigantic WASM project WASM modules.
But you compile them to to Wasm and then you run the the tests. So I ran I tr like I I I got everything compiled to Wasm, ran the tests, and it turns out a lot of them failed for the same reason. They needed a sync. So we just up updated to async and we're waiting for
Joel Moses (24:03.485)
Now you gotta go do that fun work again.
Mendy Berger (24:06.81)
yeah, yeah, it should mostly translate. The problem is that componentize JS, which is like the JS way of doing components, is not yet doesn't support P3 yet. P three is very no and yeah, the it the language ecosystem has not evolved far enough. But it should have a lot more passing because a very significant amount of the tests just failed at one function that is a sync and I couldn't make it work.
Joel Moses (24:26.184)
Got it.
Joel Moses (24:37.235)
Got it. So so let me we're we're running out of time here. So let me let me ask. if people want to get involved in in work with either GFX or with with Web GPU, how how can people get started? Are there are there areas of focus you'd like to see people help on?
Mendy Berger (24:50.689)
yes.
Mendy Berger (24:54.888)
sure, yeah. So Web GPU the the WASI Web GPU spec is at WebAssembly org on GitHub slash WASI dash web GPU. there are issues there. if you wanna chime in, share your opinion if you have if you have any, of course. and if you wanna fix bugs in the runtime, it's at GitHub.com slash WASI GFX. that is
I guess yeah, th our larger like everything that isn't the actual Web GP spec is is lives in GitHub.com slash WASIG dash GFX, which has the runtime side. If you see anything in the runtime, we would love there are lots of to-dos still there, unfortunately. If you can yeah, we would love to see people fill those in. and if you care about WASI GFX in general, we have a Discord, we have WASIGFX dot dev.
Joel Moses (25:36.857)
sure. Yeah.
Mendy Berger (25:50.537)
where you can learn more and we'll hope we hope to share continue sharing updates.
Matt Yacobucci (25:56.038)
Are there recurring meetings? if if someone would just wants to jump in the light lightly at first?
Joel Moses (25:56.125)
That's fantastic.
Mendy Berger (26:04.492)
Yeah, sure. we meet weekly. Some of us that find it interesting. I'm actually missing it this week because I'm on the podcast, which is cool. yeah, no pressure meeting. it is I can only tell you what time it is at my time. I don't remember. It is tied to UTC, so when daylight savings times goes on and off. I'm not sure. So check check the repo.
Joel Moses (26:13.265)
Ha ha ha.
Joel Moses (26:29.651)
Got it. Yeah. Well, Mindy, again, thank you for thank you for joining us. We we really appreciate your your input on this. And Def the the industry and and the the the community definitely appreciates the work that you're doing to to to get these things better defined and and stable on the Web GP side and more advanced on the GFX side. So with that, that's all the time we have for today. Now what did I learn?
by keeping the low-level web GPU interface stable in WASI while allowing higher-level graphics APIs to evolve separately under GFX, the community is trying to do something really interesting, balance long-term compatibility with rapid innovation. And that's the broader lesson here. The future of WebAssembly isn't about just about making code portable, it's about making capabilities portable.
CPU today, GPU tomorrow, and whatever specialized hardware comes next. WebAssembly components are like Lego bricks. Native applications are like pouring concrete. Both build buildings, but only one lets you change your mind after it's built. So for myself, for Matt, and for Mindy, thanks for joining us. Be sure to hit like and subscribe for future WebAssembly news and views. And as always, it's a big application world out there. It's time to embrace the power and promise of WebAssembly. Take care.
Matt Yacobucci (27:48.401)
Thank you. See you.
Mendy Berger (27:50.115)
Thank you so much for having me.