Subscribe
Share
Share
Embed
Discover the people, ideas, and technological breakthroughs that are shaping the next iteration of human co-experience. Featuring a diverse cross-section of guest speakers from Roblox’s product and engineering teams, we’ll discuss how we got to where we are today, the road ahead, and the lessons we’ve learned along the way. Hosted by David Baszucki, founder and CEO of Roblox.
David Baszucki: Hey, this is Dave Baszucki. Welcome to Tech Talks. Today we're gonna be talking with George Elkoura, welcome George, and Taylor Russ, welcome Taylor, about all things simulation and engine. We're gonna be talking about how does Roblox work, how do we simulate reality? How do we work on performance? How do we work on scale?
All of the fun things that go at really the heart of Roblox, and maybe as we roll into it, I would say. This is a topic near and dear to my heart, and way back in the very first days of Roblox, um, there's a computer programming language called c plus plus that we'll probably talk a little bit about.
In the very early days, Eric and myself were, were in an office and for about a year we were working on the first version of Roblox, and I got to do a lot of this c [00:01:00] plus plus stuff and that was arguably a very happy period in my life. So the first question, do either of you still get to ever look at or dabble or write C plus plus code?
Taylor Russ: Oh yeah, abs, absolutely. Really? Yeah. Still get, still get to mess around with it. Still get to do code reviews, still get to check. Check out the code for bugs. Still get to look at how things are implemented. Yeah, it's pretty fun.
David Baszucki: Okay, so you're running a team. Yeah. You've got two of the leaders of the whole engine simulation team, but still getting to look at code and how about you, George?
Yeah,
George ElKoura: absolutely. Same. Look at lots of code reviews. We do, um, you know, we do spot checks and we just, um, you know, try to get, um, oh, sorry. Do it again. No, that's cool. Um, because
David Baszucki: what's interesting is, as you say this and you talk about it, we, we introduced one of our principles or values in action recently.
And hopefully we're not talking the talk and walk, we're walking the walk. 'cause we talked about we want our leaders to be hands on and detail
oriented. So I just asked you this out of the blue, and it [00:02:00] sounds like you're both in the c plus plus base. That's right. Yeah.
George ElKoura: Looking at the c plus plus stuff is the happy place. Taylor Russ: Yeah,
George ElKoura: yeah, for sure.
Taylor Russ: Nothing's more fun than look during having an operations incident and digging into the code and finding the issue or thinking you find the issue of maybe being wrong 17 times, but eventually getting there.
David Baszucki: Yeah. So. Let's frame a little, what we're gonna be talking about. This is gonna be one of the more technical talks, so I want you both to go as technical as you want.
I'm gonna try to frame it a little so people can kind of cross the bridge and understand, but what we call the engine at Roblox. I like to think of it as a reality simulator, and even since the very first days of Roblox, we did something a little unorthodox. We made the focus of the engine a physics simulator.
At that time, we just recently are baiting acoustic simulation. We have voice simulation and others always with the notion that the more we simulate reality, the easier it will be for people to make games and experiences and the more natural they will look. And the very first cars on Roblox had axles and wheels and motors, and if the wheel would fall off, they, they would do what we expect.
So we're in the reality simulation business. Maybe before we jump in, just a little background first, George, then Taylor. Kind of what brought you to this, this path of reality simulation?
George ElKoura: Yeah. Um, [00:03:00] the thing that, um, interested me most was the, the reality simulation is interesting.
David Baszucki: Yeah.
George ElKoura: And then reality simulation at scale just blows your mind.
David Baszucki: That's right.
George ElKoura: Millions of concurrent users. How do you get, you know, hundreds, hundreds of thousands of people on the same server at the same time? That kind of stuff really excites me.
David Baszucki: So we, we do have more work to do.
George ElKoura: We do have more work to do.
David Baszucki: We cannot simulate a million players photo, realistically all attending the same sporting event. That's a hard problem. Okay, Taylor, what, what brought you to reality simulation?
Taylor Russ: Yeah, I think, um, the biggest thing is just the, the fact that it's user generated. Um, you know, it's not a traditional reality simulation or game-like simulation platform where it's constrained by game design. You know, you can make anything on Roblox, you can make the real world, you can make, uh, card game, you could make a, a first person shooter, and that it brings an unprecedented challenge to the technology. In turn, in addition to the scale. [00:04:00]
David Baszucki: It's actually really hard. Right? Yeah. One way to think about it's, we're trying to build a generalized reality simulator.
Taylor Russ:Exactly.
David Baszucki: Have a wide range of experiences run on that. And people who are familiar with the gaming space maybe have heard things like, well, we gotta bake in the blank, or we gotta do that. I think those are things that you do when you're not running a generalized simulation that we're unable to do. Like we have to have that performance without any pre-baking.
Taylor Russ: That's exactly right. Yeah. And you, you, you get to build your tech around those assumptions, like those kinds of things. But we can't take those kinds of, um, paths on Robloxs.
David Baszucki: So maybe a positive way of looking at it is by trying to build a general purpose, reality simulator. Our work is a lot more difficult, but if we can do it right, we're gonna see some pretty cool stuff.
Okay, so, um, one, one way to think about what we're talking about, and I was thinking about even when we started thinking about Roblox, if [00:05:00] we had infinite compute capability, infinite bandwidth, infinite memory, infinite ram. We would just say we're gonna simulate 10 to the 40th power of atoms, and we're just going to like the, our users are gonna set up these atoms and we're gonna see everything we want.
Water's gonna flow, steel's gonna bend objects that are gonna bounce into each other. We're gonna hear that go through the air. We don't have that. Right. We have a limited set of compute.
Taylor Russ: Yes. Yes, exactly right.
David Baszucki: Maybe the framing of everything you're gonna hear today is how we're trying to get around not having enough atoms and still do what we have to do.
Taylor Russ: Yeah. How do we fake it till we make it, basically?
David Baszucki: So we're really faking it. Yeah. Okay. It's gonna be a while till we have 10 to the 40th atoms or whatever we would need on our thing. Okay. So we're going to, now we're gonna dive into some of the. The things we've been working on and just hop through a few
topics.
One of the things that's unique [00:06:00] about Roblox is we're trying to run on phones, tablets, computers, consoles, whatever, a wide range of environments. We have iOS, we have Android, we have pc, we have Mac, we have Xbox, we have PlayStation, we have Medi Quest. We have Android devices more to come, arguably. We also have our server infrastructure, which is big heavyweight servers, and you are building code that runs on all of them up until about six months ago. There is something that every one of these devices gave us that we just used, um, without thinking about it. And who wants to say what that was?
Taylor Russ: Uh, are we talking about our memory allocator?
David Baszucki: That's correct. Oh, so every one of those devices has a memory allocator.
And when you build a program on those devices. Arguably the, each one of those operating system has built their own memory allocator, which is when a program runs, it gives you all the millions of little objects [00:07:00] you need to run. And I was thinking about it before the talk. They probably are, are really working to make a memory allocator that works well for everyone.
But maybe that means it doesn't work the best for Roblox. So what, can you share a bit, who wants to share what we've done in the last six to 12 months?
Taylor Russ: Yeah, that's exactly right. We, so we, you know, you hit on it exactly, which is the general allocator that ships on, um. With, with the standard implementation, um, it, you know, it has to support every program in the world and every program that's ever been written, um, you know, it's, it's not specialized.
Um, and when you, when you're targeting a single program, you have the ability to specialize it. And so that's what we've done is implemented a special specialized memory allocator. It gives us a lot more fine-grain control over how allocations work, um, how they run, where they run. Um, we also have a lot more control over the individual technical characteristics like the data structures for tracking.
And this lets us do different things around, uh, performance and stability improvements.
David Baszucki: And can you share what fragmentation is?
George ElKoura: We can improve fragmentation with that. So, that's
David Baszucki: exactly right. What, and can you share what fragmentation is,
George ElKoura: Fragmentation is when you allocate small blocks of memory and then, um, you leave a lot of gaps in between.
So when you avoid fragmentation, you have more. Uh, the, the ability to have more spaces in memory.
Taylor Russ: And then the problem is, is that if you, if you don't have enough space because you've left these gaps and you go and ask for, uh, some size of [00:08:22] memory, you may not be able to allocate that memory.
David Baszucki: Right? And we're gonna talk about, uh, some advances we're making in our architecture where the more accurate, we know how much memory's available, the more we think we can make a system that will never crash.
Taylor Russ: That's exactly right.
David Baszucki: And I would, I would say, without throwing any vendors under the bus, um, if we look at all these devices, there's a various range of accuracy in how well those memory managers tell us what's available.
Taylor Russ: That's exactly right. Yep.
David Baszucki: Yeah. And so that creates a big challenge because we want to use as much as we can, but. If in many of the, in some of those devices without saying which one, um, if we do that, all of a sudden sometimes the device will just say, sorry, we can't supply that memory. We're gonna crash on you. Okay. So let's jump to, so we have taken this disparate set of memory managers on every device and put our own on top of that really. Or [00:09:23] a combination of open source and our own. And so we're running a consistent memory manager everywhere now.
Taylor Russ: That's exactly right. Yep. On every platform that Roblox
David Baszucki: ships on. Okay. And then for the users out there, so the people that here, what is, you know, what are some things they might have not noticed, but that we have measured over the last three to six months?
Taylor Russ: You, uh, individuals may not have noticed it, but you would notice improvements in crash rate.
David Baszucki: I, I watched that. Yeah. Yeah, exactly. So I've noticed that.
Yeah. The other thing that's, uh, a big win with, um, specialized memory allocation is performance. Um, that's right. And so what we notice in our macro level metrics is just improvements over, um, frame per second and jitter as well.
David Baszucki: And so what's what's very interesting in what we expect is whenever we raise performance frame rate. Raw performance consistency. In this case, I believe we statistically measured people playing longer blocks.
Taylor Russ: Exactly. [00:10:16]
David Baszucki: Which is very hard to do and
amazing. And is, um, this is technology that every single experience, no matter who created it automatically gets a little bit better.
Taylor Russ: Yep, that's exactly right.
David Baszucki: Okay. So nice job. Step one of engines. Step two. Yeah. So there's another thing now that we're working on that. I'm not sure we're all the way there. We have a huge grid of servers, um, all around the world. Um, I think, you know, we've shared that we have one in Singapore, we have one in India.
You know, everyone's excited 'cause we're gonna build one in Brazil. Brazil. Yeah. This is gonna be really cool. And these, um, this big data grid has a bunch of servers that have a bunch of cores on them. And we've started, we've rolled out the notion that the larger the game on Roblox. The more [00:11:16] cores it's gonna get because we want to make it fair for creators who make a 500 person game to get more cores.
'cause there's more people playing. So there's some notion, I'm, I'm always, I think, you know, I'm always asking, are we saturating all this cores? Can you. Can someone, can either of you mention what that would mean for us?
Taylor Russ: Yeah. If we could fully saturate all that, those cores, it would mean that we're, um, uh, we're, we're serving the workloads that we optimally can serve.
Ideally, we would fill a hundred percent of our server capacity all the time. Um, and if we filled all of our server capacity all the time, that would imply that our, uh, creators and our users are, uh, having a much more optimal experience because data is getting processed on the server and delivered to them more quickly.
David Baszucki: And so what this means is our, our core engine code that runs in the server runs very parallel right now. But isn't perfectly parallel. That's exactly right. And, and, and when we swipe through every system on our, our servers in the [00:13:00] cloud, they have physics simulation, they have scripting, they have data in, they have data out, they have networking, they have all of that.
And. We are working, I believe, to get every one of those systems fully parallel.
Taylor Russ: Yep. And there's a lot more work to do there, but the idea is to make it so that, you know, parallelism in the Roblox engine is ubiquitous. You just add if you're adding code or you write code in, in, in, uh, lua, uh, or luau for your game, that it's just parallel by default and you don't have to think about it or worry about it.
David Baszucki: Okay. So engine project number two, complete parallelism on the way. I'm excited about that. Engine challenge number three that we've been talking a lot about and we've been hinting at about in the, in the traditional gaming space. What I see a lot of when I use traditional games is I download a really big thing.
Sometimes it's 200 gigabytes, sometimes it's [00:14:00] on an app store, and then after I download that. I start playing it. Um, so if you were, if you sent me a, a, a note either on Roblox or someone said, Hey, come play it. I said, give me a few minutes, um, to do that. So maybe before we jump into this next topic, which is pretty rich, either one of you want to share what's unique about our architecture relative to both a download, which is you download everything.
And then the other one is, we've all heard about streaming, right? I thought there's like video streaming. But I, I actually think there's something better in the middle who wants to chat about that.
George ElKoura: Um, so, um, our architecture is fundamentally different than a traditional game engine where, like I said, you have to wait to download, you know, tens of gigabytes of data sometimes, and on Roblox you can go ahead and start playing a game in. Seconds and we'd like it to be even faster than [00:15:00] that. That's not good enough.
David Baszucki: That's right. We want instant join. That's
George ElKoura: right. Exactly. And so the big difference is that we view our engine as our entire platform. It's both the engine that is traditionally called an engine and the cloud architecture that we have, that we built all leveraging all the power of the servers and everything that we have that really once we start.
Pushing on the limits of that, it becomes, uh, breathtaking in terms of what the size of worlds that we could live in.
David Baszucki: So we are, we're doing a, we're not streaming video, we're streaming 3D objects. Uh, some of those objects are simulating on a local device. Which in certain cases we be believe is very important.
One, um, important thing is when doing communication and looking at a, at yourself on the screen, if you don't see immediately what you're doing. It's a little disconcerting, um, if you see latency when you move your hand around. So there's [00:16:00] cases when we want to do that. And then another big part of our architecture, some of those objects we simulate on servers.
At the same time. We're always trying to mix what we simulate locally versus what's on the server.
Taylor Russ: Cool. Another big note about Roblox too is that we could, uh, make and publish a game just sitting here while we're talking. You know, in, in two minutes we could have something online on Roblox. There's no download, no seams.
Um, and that's the coolest thing about it is like, you know, it's just there and playable instantly. You don't have to host, host servers or do anything like that.
David Baszucki: Okay? So we have, um, clients on a bunch of devices. They're all connected to a cloud. They work very closely together. And now up until recently, and I'll share a little about where we're going.
We put certain limits on the resolution of what objects, um, people could use in their ROBLOX experiences, and we arguably didn't fully use the cloud to help us solve some interesting use cases. One interesting use case, in addition to joining instantly or publishing the same game to every device is on a very low end Android.
I'd like to play the same experience that I play on a super court out gaming pc. And, and believe it or not, in in many places around the world, there are still Android devices with two gigabyte of RAM being sold, that that's someone's first smartphone. I. Um, and typically game game creators make a different build.
They make a low in Android build, they make a high-end PC build. We want the same build. That's exactly right. Yeah. To do that. So how is,
how can I have these super small assets on an Android and super high res assets [00:17:00] on a gaming PC with the same build?
George ElKoura: Uh, so there's, um, there's something called level of detail.
Okay. Um, which is traditionally used to like you, like you mentioned, on traditional game engines. They will produce different builds of assets for each different kind of, uh, device. And even sometimes, for the same thing. For Roblox, what we'd like to do is leverage our cloud infrastructure so that stuff happens on the fly lazily automatically.
You don't have to as a creator. Um, you don't have to worry about all of the hassle and the burden that goes with managing your l your own LOD pipeline. And the more complicated that gets, the more people just don't do it. And if you don't do it, you leave performance on the table.
David Baszucki: So I know that some video platforms, now that there's 4K TVs will still run on an old fashioned 1000, you know, pixel monitor.
And so those video companies are probably. Doing something similar. They're uploading 4K video, downsampling it intelligently, and then sending the right video. So I You share what, I know we're doing this for meshes textures like that. We're doing the same thing essentially. Right?
George ElKoura: Right. Yeah, exactly. And so we, we do it for, uh, all of our asset types that we have on the platform.
We do it for meshes, we'd do it for textures, audio, um, video. Um, anything that, an animation in the future as well. Uh, anything that is considered a Roblox asset would. Would be able to run through this, uh, transcoding pipeline that we've built.
David Baszucki: Okay, so this is something we publicly talked about. We're gonna be rolling out incrementally over the next, you know, six to 12 months kind of thing.
There's a, there's an interesting notion here that then for the first time creators, you know, have always bumped up against the limits of the fidelity of meshes or, or the, the detail on meshes. But now we're gonna eliminate those ma those things and store natively. I believe
George ElKoura: that's right. Yeah. Uh, right now what we do is we go through a process of, um, of filtering the data and processing the data before we store it.
And, uh, soon we'd like to start storing your original assets, um, and original data at the native resolution, at the highest resolution that, that you give us. And then we'll be able to up sample and down sample on the fly, depending on the, uh, on the capability of the device viewing it.
David Baszucki: So what this will mean for a creator is a creator could work in very high resolution assets, push to our cloud, and still we believe on a two gigabyte Ram Android device automatically deliver very low res assets and test textures to make that thing function at good perf.
Taylor Russ: that's it, yeah. And be able, uh, to have it scale dynamically up and down so that you get the correct feel, a consistent feel at that lower level of, of, um, of performance. Like, you know, if you're on a two gig Android device that you still get, um, you still get good performance even though you have low memory potentially.
David Baszucki: Okay.
Taylor Russ: And be able to scale it up and down dynamically.
David Baszucki: So we, uh, this is gonna feel like a whole new Roblox, I feel. We are not used to this kind of mix of low end perf and high end quality that we're gonna see. Now there's another benefit, um, that having a wide, um, what we call LOD assets can provide. And so what, you know, to be specific, if we have a, a car that takes a thousand or a million, say a million triangle car.
Internally on our cloud, we're gonna have a hundred triangle and a 500 and whatever type thing. So there's a, there's also a benefit in joining Roblox quickly and having it feel like we're joining quickly. Who, I dunno, which one of you wants to chat about. Um. Our orchestrator and using LOD to help join more quickly.
George ElKoura: Yeah, I I really, I really like an analogy that you use where in the early days of the web where you had, you know, yeah, that's right. The images that show up and they pixelated and then they come in a little bit more crisp, uh, as you get more bandwidth. Our orchestrator effectively does that on world scale in 3D um, for a world.
So when you first join, you download the lowest res versions of the asset as soon as possible, and you get, you're immediately immersed in the world. And then over time. Um, you quickly get the highest resolution things that are up close first and then further, uh, they, they go further out. So our orchestrator tries to, um, optimize for human factors. So we take things into consideration, like the distance, uh, of the object to the camera or to the player. We take into consideration the, uh, pixel coverage we take into consideration.
Um, you know, the, the, the importance of the object and, um, and all of that together makes you feel like you're in that experience immediately.
And also the further away objects you might not notice that they never actually go up in high res. That's right. And so we save a bunch of bandwidth and, and CPU in memory,
David Baszucki: Well, we talked about the ultimate extent of this is an object way in the distance, that million triangle car might only have 12 triangles.
Um, but way up in the distance that's gonna be enough for you to still see that color splotch, which is, and you said something really important. So thinking through, um, what people have on their own. Device phone or a computer. That device has bandwidth, it has memory, it has ram, it has CPU, it has GPU. It has all of these things, so that's complicated.
Then in addition, all of us have a, as you said, human perception of what feels like I'm joining as quickly as possible, and so we're gonna optimize for each device specifically. The mix of the, that streaming performance to optimize the perception of a quick join of the game?
Taylor Russ: Yeah, we're gonna trade off resources against quality.
So in some cases we'll spend more resources to increase quality, and in other cases we'll, uh, increase quality because there's
resources to spend. So it's that constant balance that happens dynamically across all resources.
David Baszucki: I don't think people are used to joining Roblox experiences instantaneously that that would be the thing we're shooting for.
I, I would say in the past we have mocked up versions of our mobile app and we've simulated what it would feel like to join instantly putting a proxy video in there or something. And it does feel like a new product is just like. No one is used to, uh, that kind of a thing. It
George ElKoura: feels amazing. I mean, firsthand, I tried a hack week project where you would join instantly and you just go, you can play as fast as you can swipe, and it totally feels different.
It is completely immersive.
David Baszucki: We’re not used to that.
George ElKoura: We're not used to that.
David Baszucki: Cool. So now as part of LOD, um, we have all these different things in the Roblox world and there's a little bit of a, the more we treat things the same. The more leverage we can get. So we have traditional parts, we have mesh parts, we have voxel terrain, and we have avatars that are kind of in a complex way, layered and composited.
So the, the freebie question would be, will avatars participate in this LOD system as well?
George ElKoura: Yes. Absolutely. Yeah. Yes. Okay. And,
David Baszucki: and if anything, I think. We have, um, work underway to make avatars less avatar ish and more like everything else. I dunno if you, can you talk a little about our digital matter initiative?
George ElKoura: Yeah.
David Baszucki: Where everything is digital matter, whether it's an avatar or a car.
George ElKoura: Yeah, absolutely. So, um, digital matter is basically what makes up the, the universe in, in Roblox, the world and. Uh, for a variety of reasons, avatars have been treated a little bit differently than, uh, than, you know, than the rest of the digital matter.
So yeah, cars and trees and all of that stuff are digital matter, but then avatars have their own specialized pipeline. Now, avatars are super important to the platform and we, and, but we can, um. We can merge code, we can refactor things such that we can maintain the performance of the avatars that we need while, uh, having it be part of, uh, digital matter so that we get LOD, we get occlusion calling.
We get all of the benefits that we're putting into our, our rendering system, and they would apply to avatars just as well as they would apply to vegetation or mountains or anything else that is in the scene as well.
David Baszucki: So one benefit of, um, treating things more and more the same. Cars, trees, and avatars is the same code as running that same thing.
And I, I think that one benefit then is when we optimize or improve that code. We do it one time rather than three times. Exactly. Yeah.
Taylor Russ: Optimizations carry forward. Um, you know, any optimization applies as well as any refactoring, tech, debt, maintainability, APIs, you know, it's just much more beneficial to have one code path.
David Baszucki: I, I feel like we've seen this pattern time, time again in that when we were working on Medi Quest, for example, which has some overlap with Android devices by not um. You know, forking out the code by having, and I don't know what the number is, 99%, 96%, exactly the same code anywhere. I believe when we were working on the Medi quest, really hard to make it perform well.
We got a little. Yeah. Android's getting faster at the same time.
Taylor Russ: Yep. Yeah, exactly. Um, there was some challenges with Quest, that's for sure. But yeah, it is about nine. You know, I don't know
the exact number, but it's, we try to minimize the amount of platform, co specific code that we have.
David Baszucki: Okay. So now I'm going to gonna jump around and stick with the avatars.
So this one will go high level and maybe not give away any stuff that we're working on at too much of a detail. But philosophically, when we look at a Roblox world right now, there's a lot of stuff in that world, right? And a standard avatar I, I believe can be many body parts, many. Layered things, um, all types of things.
And I think in the traditional video game space, rather than having that avatar have 200 separate things, they, they tend to have artists bake it into a single thing. Is, is that fair to say? I think
Taylor Russ: Yeah. That's fair.
George ElKoura: Exactly. Yeah.
David Baszucki: And, and so we've, we've taken an approach that we don't want to bake it because we want to have any clothing work with any avatar, any animation.
Work with any avatar, facial drive, any avatar. So we've resisted that baking. But I guess I would say if there was any opportunity to automatically do some of that stuff, we might see some performance. [00:28:00]
George ElKoura: We, we just might. Yeah. Who knows Dave.
David Baszucki: So there is, there is a elegant future out there of, Ima imagining the flexibility of, uh, open-ended generalized engine.
But by using cloud, more ability to optimize performance at the same time of these things. Okay, so I looked on, um, one of the most, the biggest PC gaming platforms, and there's a bunch of them. And I think, I'm not sure if I said it to you, George or you Taylor. I said, um, here's a top 20 list. Of these games on these PC platforms.
PC tends to be more competitive gamer. More competitive, um, you know, not just pure social. And I, I think I asked, uh, what percent of those were server authority? Was it you or you?
Taylor Russ: You probably asked me.
David Baszucki: Okay. And do you remember what percent you
Taylor Russ: said? I don't remember the percentage, but I think it was, uh, close to, uh, a hundred percent for all the competitive games, but that's correct.
David Baszucki: Yeah. So this, I, I want to take a step back and highlight the complexity of this problem. And this is deep in the lore of all games. We like the notion, if possible. Of simulating the client client side, you can see your avatar act like a mirror. It's very predictable. Um, if you're going one direction, you change very quickly.
It's immediately responsive. But that does create a problem in competitive gaming and maybe tailor you can share what happens in competitive gaming with a client authoritative game. So as we talk about doing server authority. You know, there's a lot of things people do with a client authority. Um.Client to do funny stuff.
Right. And we, we can see them trying to do it on Roblox. Are there certain types of things people try to do? Yeah,
Taylor Russ: absolutely. One of the, uh, common things that we see is like flinging or changing the velocity or fly hacks That's right where you're flying in the world. Um, and these are very disruptive when you're applying.
Um. It's just not fun to play. It's also not fun to create when you're vulnerable to this type of activity.
David Baszucki: Yeah, and I would say the Roblox developer community [00:30:00] has done an amazing job. 'cause Roblox right now is not. Hardcore server authoritative. Um, we run in this more general architecture. It's done a lot of work to do a nice, nice job in, in locking down those experiences.
But we really want, um, the generalized platform of Roblox to work. In various modes. One mode I would, you know, would be a more social, I'm interacting 100% avatar fidelity on what I'm simulating locally. Um, but the other mode is a server authoritative mode for competitive stuff. So I. Can you get a, give a hint of what we're working on, on server authority?
Taylor Russ: Yeah. Um, so there's big challenges there to tackle the server authority. Um, obviously with client authority, you know, you press uh, a w and you feel that reaction right away and in a naive. Sort of approach to this problem that we've talked about. You know, you could sort of press w wait for the packet to go to the server, um, wait for the response, but then the client would feel all that latency.
You know, that's sort of a, a brute force approach. So the approach to tackling this is to predict what happens on the client or to extrapolate, um, in other words, and this is used heavily, um, in, in the games industry, but what we're trying to tackle here is to make it dynamic. Um, we want client authority whenever we can have that, um, because it's better for performance responsiveness and you don't have the corrections from the server.
But in cases where we can, we wanna put the client in, in a server authoritative mode or cases where the game requires the competitive treatment or the competitive, um, you know, full competitive environment.
In that case, it would be server authoritative. So what we're doing is figuring out how to tackle this problem so that our creators really don't have to, to hoist. The cognitive burden on themselves of dealing with this, oh my gosh, is it on the client, is on the server? The complexities of all of that, you know, and where, uh, different clients are on the timeline.
David Baszucki: So the, the demos we've been running and we shared, we're working on this, um, for competitive gaming. It's really satisfying. Um, 'cause we simulate people hacking the client and all of that, but with server authority, that's like how a banking system right works, right? The server's in charge and a lot of these things just don't work anymore.
There's one other thing that's really interesting and that is traditionally for very complex physical simulation, we run a distributed physics environment on our platform and we try to simulate client side as well. The better the extrapolation, the more in more situations it feels natural to run server authority.
And then when it feels natural to automatically run server authority, many of these very difficult things. Bus of a bus driving with 50 people in it over gravel and the gravel's crumbling. Is pretty much impossible to run as a distributed physics simulation. Maybe some people say you're wrong, but I think that's a pretty difficult distributed physics simulation.
But having that work with good extrapolation, um, maybe just share what those demos look like. 'cause I, I think we've seen some demos of. Things we've never seen on Roblox before.
Taylor Russ: Yeah, absolutely. Um, we've done some complex demos. We've, uh, one, um, it seems a little simple, but it's a seesaw. Another is more complicated.
It's called the party bus. It's got a lot of, uh, people driving on it. A lot of complex physics interactions. Um, and these look and feel really good even in this distributed model. Where some of the authorities on the client, some is on the server. Um, and for our demo, we've done some of the server authority, uh, a full server authority version, and then also a dynamic version.
David Baszucki: So what it means to the ROBLOX community is there, there's been a few edge cases that are very difficult, but we expect them to work that way, that are naturally over time gonna start working. That's
exactly right. Yeah. And pretty much whatever we build, we can always go to server authority with good extrapolation and get pretty there.
There's also one other thing, um, that I I'm really excited about, um, for kind of close quarters combat wrestling grappling. That's also something that's pretty much impossible to do. Client authority. But I'm, I'm optimistic we're gonna see more of that.
Taylor Russ: Yeah, exactly. And you don't wanna, uh, like a tackle in a football game is a competitive scenario, you know?
It's Exactly. You wanna solve those kinds of problems.
David Baszucki: We're gonna see realistic tackles. Exactly.
Taylor Russ: We're gonna see realistic tackles, not videos where it looks like you got tackled, but, oh, I didn't, you know, on two different monitors. Like, it's consistent.
David Baszucki: Okay. Um, now I wanna, um, now, uh, the Roblox community and this one, we're not gonna go into too much detail, but I want to, maybe this is more of a, let them know we hear you thing and the sec. Um, for those of us that are familiar with writing code on Roblox.
We know some of this code can be a little complex 'cause sometimes we have code that runs on the client and sometimes we have code that runs on the server and you know, things that I'm using like a flashlight or a crossbow.
The more client side, the more it feels, um, real time. But some of these things for anti cheat and other things need to do that. So is it fair to say we're looking at the whole paradigm of scripting and. I think we're not gonna share it, but I, I believe we made some huge breakthroughs and we're looking forward to kind of looking that up.
Taylor Russ: Yeah, that's exactly right. Um, we, we, this, um, the server authority paradigm introduces a lot of additional. Our candidate introduce a lot of additional cognitive burden on creators or on, um, uh, yeah, they can introduce a lot of additional cognitive burden on creators, and we're looking at scripting holistically to see how we can improve that such that, you know, you, you don't have to worry about the details of where things are in the world or on the client.
Is it on the server that things just work and it's just simpler to write code on Roblox and then we will handle it for you.
George ElKoura: Dave, when I first joined, you mentioned something about the wait for child, um, problem.
David Baszucki: Yeah, that's like a little artifact that a lot of our creators use when they're writing code to handle this complex client server interaction.
So, yeah, we, I was like, could we ever get rid of it?
George ElKoura: Right. So yeah, in a, wouldn't it be cool kind of way? Wouldn't it be cool?
David Baszucki: So, so maybe one way to think about it as a design goal. Why would you think about something like Wait for child? And arguably, if we were, if this was a Roblox world and we were walking around, we wouldn't be thinking client server.
We would just be thinking, attach some code to this thing and that should work no matter what. And I think that's the design goal of where we're going to get to.
Okay. So, um, so I want to take a step back. We went through a lot of technical stuff and so there's probably a lot of people that are going, oh my gosh, like my head is exploding.
But, um, one way to wrap it up is, um, we've, we've said sometimes the specification for what we're trying to build here. Is actually very simple. And so, uh, I'm gonna ask you a few questions about the spec for the ultimate Roblox cloud engine and all of that. Um, a hundred players or million players is scale important?
George ElKoura: Yeah.
David Baszucki: Scale's important.
Taylor Russ: It's always important. Yes.
David Baszucki: Um, many people make arguments that scale's not important. Same argument was made 10 years ago, and then Battle Royale came on with a hundred, and everyone's like, that's great. So I, I do believe million concurrency in a big stadium or, you know, giant recreation of a medieval battle.
Super important.
George ElKoura: Yeah. We need to get to a place where the limitation is not based on the architecture, but just on the raw resources that you have available. That's correct. Yeah.
David Baszucki: That's correct. Yeah.
Um, ultimately support something that feels photorealistic or not support, something that feels photo realistic.
Taylor Russ: I think we absolutely have to support photorealistic.
David Baszucki: Thank you.
George ElKoura: Yeah. Do the hard thing first.
David Baszucki: Same thing, right? One. One might argue, you know, there's a lot of genres in game engines. You know, blocky, anime cell, whatever. I think it's fair to say photorealism is the most difficult spec. Yeah. And to simulate…
Taylor Russ: computationally and just in terms of authoring it to scale on all devices as we talked about earlier,
David Baszucki: um, we've already talked about it.
Uh, join in two seconds or join instantly.
Taylor Russ: Don't, we don't want seams in any, you know, that's like a common theme that we've talked about on this podcast. Like no seam to download, no seam to public.
David Baszucki: Uh, trick question, different build, low end Android pc, or same build, right? Also by language, also by local policy, also by, you know, push wants run everywhere.
Okay, so that, so I think we shared what the one way of thinking about the spec is, right? Million players, photorealistic, instant join, build anywhere, uh, ship. Next year we'll get back to ship continuously, you know? Yeah. We'll, we'll iterate our way there. Yeah. Yeah. Okay. So that, that's part of what makes it fun working with both of you.
There's so much interesting work to do. So, okay. Well it's been amazing talking with both of you, I hope. Um, the techies I think got everything they hopefully wanted. I think the more casual Roblox observers, hopefully we've got a little glimpse of how intensely we're focused on this reality simulation problem and what amazing people we have.
George ElKoura: Thank you for having us, Dave. Thanks.
Taylor Russ: Thanks for having us, Dave. Appreciate it. Yeah,
David Baszucki: Yeah, thanks. Okay. This is Dave Baszucki and that's been another episode of Tech Talks. If you want, check us out at corp.roblox.com and I'll look forward to seeing you next time. Thank you.