In this episode, Tracy talks to John Gaeta about his interests in machinima and real time, The Matrix Awakens Experience, the influence of the bullet time shot, building the metaverse, future of storytelling in immersive environments, the potential of NFTs and his advice for indie creators
SUMMARY KEYWORDS
world, machinima, create, matrix, metaverse, people, question, real, experience, awakens, experiments, ai, game, story, talking, thought, imagine, virtual, playstation, digital
SPEAKERS
John Gaeta, Tracy Harwood
Credits:
Producer/Editor: Ricky Grove
Music credits: frankum's Nebua Techno House. freesound.org Creative Commons
Machinima, real-time filmmaking, virtual production and VR. Four veteran machinimators share news, new films & filmmakers, and discuss the past, present and future of machinima.
SPEAKERS
John Gaeta, Tracy Harwood
Tracy Harwood 00:14
And welcome to the And Now For Something Completely Machinima podcast. I'm joined today by a very special guest. This is John Gaeta, also known probably many years ago now as Dalt Wisney. Welcome to the show, John.
John Gaeta 00:29
Thanks for having me.
Tracy Harwood 00:31
Welcome. For those of you that haven't heard about John, he's probably most well known for creating that infamous 'Bullet Time' shot we hear so much about in The Matrix films, I think especially evident in the Resurrections film for those of you that haven't really seen that either. And of course, he's also known for developing other things such as volumetric cinematography, and universal capture. His resume is amazing, notably, and of course, he won the Best Visual Effects Oscar at the 72nd Academy Awards for his work on The Matrix. And since then, he co founded Lucasfilms immersive entertainment division called ILMxLab, which has been a pioneer in virtual reality, mixed reality, holographic cinema and future theme parks. And he acted as its executive creative director, I think, working on the portfolio of premium experiences, which include Star Wars, and most recently he did, he did a cameo in front of the camera in Matrix Resurrections. And you can check out his film credits on the Internet Movie Database, which we'll provide a link to in the show notes. Now, from our perspective on the podcast, John has also had a long time interest in machinima. At least that's according to Kim Libreri, who, as you probably aware, is the Chief Technology Officer at Epic Games, whom I interviewed for the Pioneers in Machinima book that Ben and I published last year. So John, when did you first hear about machinima? What do you recall of your first thoughts about it?
John Gaeta 02:12
Well, that's a good question. I would, I would suggest that it was around the time that we were making the trilogy I guess, Matrix trilogy, that those many years ago that late 90s, or, yeah, perhaps early 2000s, early 2000s. It gets fuzzy. But it was around that it was around that time. And yeah, it was a, a sort of, obviously a in a, almost like a hacker's sort of disruption of the video game... I wouldn't call it the business, it would be like the sort of format of video games. And that was obviously very interesting.
Tracy Harwood 03:07
Cool. So I mean, one of the things that Kim mentioned when I talked to him was that your interest in it is what kind of led you to develop this demo for this the Sony GSQ PlayStation supercomputer, which, as far as I am aware was shown at SIGGRAPH in 2000. When you were asked to create something Matrix themed, can you tell us anything more about what you created?
John Gaeta 03:34
Sure. I mean, then, it just, it would be good. Put a little context behind it. And I I would guess if you talk to Kim about it, you might know this context, but I'll I'll reiterate it for this podcast. Um, so when we were making the Matrix trilogy, it was obviously had resonance well, beyond just visual effects technique. It was it was much more impactful with regard to the some of the concepts underlying and in particular, there were aside from folks who were obviously triggered in a way to think existentially about what is real. There were other people who were even at at that time, imagining a long road towards creating the capacity to run massive simulations, you know, massive flawlessly real simulations that people could live within play within tell stories within. And so those sorts of folks there were many interesting folks that found us that had nothing to do with the film business. It's many people in different technologies and, and artists and writers, scientists, engineers... there was a person who was very significant though, in the game industry that found us - that's the inventor of the PlayStation, Ken Kutaragi. And we wound up the Wachowskis and Kim and I wound up having a number of good long conversations over very nice meals, to talk about the future. And, you know, as you can imagine, he had a lot of thoughts about that. And in fact, you know, went on to describe in great detail, in a very prescient way, the the coming of the cloud, basically, the ability to serve, you know, unlimited, high bandwidth, exquisite content, not from your machine, but from somewhere else. And so anyway, this led to, you know, a good relationship, and he led us in on this experiment, where he was going to try to create this giant, PlayStation based parallel processing computer, you know, he was gonna band, I think it was gonna band together 16 PlayStations, to, you know, provide that the compute to run that new kind of real time simulation. And that's kind of the beginning, he's like, we, I would like you guys to get involved in and think about something you might want to do with that capability. And it is interesting, you know, to bring up that test relative to today, because Awakens, I think I get what, why you're asked that, because awakens, in a sense, is, is a dot connect back to this, this demonstration we did in 2000. And that demonstration was not a lot different, it was essentially a sort of a premise of creating dramatic content that could be used, you know, in and of itself, you know, as a experience, but also could be used to help understand how to stage story and cinematic scenes using game engines. It was both really. And you know, the future idea at the time was that, well, in the future, this will be something that everybody does, and people will be in world doing things like this, being involved with other characters and with each other. Those were the underlying themes. And we made it, you know, a sort of a fantasy Matrix scene, you know, Trinity being chased on the rooftops of a major city. And, and, oddly, you know, here we are, we were almost 20 years later, and never expected Matrix to return because of the what we thought was conclusiveness of the trilogy. And then we realised it was, we realised that our colleagues, you know, that Wachowskis wanted to tell another story. And so we Kim and I tried to understand what what would be our place in this new in this new thing, because it's so many years later, and we're doing many different things. And not often cinema directly, like working going out and doing shoots. So this is, this is what we arrived at as the right, the right way to participate in resurrecting the Matrix.
Tracy Harwood 08:57
So it really is your Deus Machina.
John Gaeta 09:01
In a way, and it's this, it's not as if we left, you know, we did that the Matrix and then we sort of all left for you know, that many years, we we've always been in touch, we've actually been doing lots of different experiments and innovations along the way. And in Kim's case, he brought what I consider probably, you know, perhaps the most unique, you know, capability to graphics that anyone has. He brought that capability to and his experiences into into Epic itself and has had a role, you know, working with Tim Sweeney to transform Unreal into into the very, you know, the very type of platform one would need to really manifest these things.
Tracy Harwood 10:01
Absolutely. And, you know, why an experience and not a game, at this point with with the Matrix Awakens Experience, I mean 20 years later?
John Gaeta 10:14
Well, it's it is a bit of all, it's a bit of everything. On purpose, it's a hybrid, the, which is what, you know, immersive worlds metaverse-like worlds are really going to be. They're going to have the capacity to, you know, to game in, of course, you could stage full games in these worlds. And a lot of the experiences will be that because that's what, you know, generation, young generations really want. They want interactive engagement at play. But the exact same destination could be used to create, you know, spectacle, and story and experiences of every kind, creator, you know, whole creator communities can. And all of that stuff, things we've learned from Second Life will be directly applicable, are already directly applicable. So there isn't, unless, unless the the folks who sort of, you know, how would I put it launched specific destinations, unless they really want to have a very specific game focus of a particular game, you know, there will be a lot of these environments that are going to be made by the users themselves.
Tracy Harwood 11:42
Absolutely. The thing that struck me about the Experience is the fact that you've made it open so that anybody can actually, you know, to use the assets and create stories with it is, you know, can you tell us a bit more about what your, your thought processes there?
John Gaeta 12:00
Oh, for sure. It's obviously, okay. So two things I would say about that. It firstly, you know, I've had several experiences, and this was, this was like a new version of an old experience. And that would be, you know, building something really big and complicated, like, a studio, at the same time that you're making some kind of entertainment product. I've been through like three versions of that, where whole studios have been built, or four versions, whole studios built, right? While also making a movie, which is a big deal, like you're making companies and organisation structures. And so it's really complicated product plus build. In this case, it's not exactly that. But it's similar insofar as, you know, Unreal 5, is really heading into its beta mode right now. Right? Like it's being used by the first developer ecosystem. And along the way of making Matrix Awakens, they were essentially dogfooding basically trying to figure out how to work with you know, new features and capabilities that were not possible before particularly Lumen and Nanite. And an in the case of the way that the city was made this really extraordinary generative world building tools that are new. The way that a lot of the simulation content was put together the all the traffic and walking people the deployment of meta humans. I mean, this is this was huge lift. Right? It was like bringing together these these different component pieces that have been in the making for years. And it's you know, UE5 is like a departure point. It's always been considered to be a departure point, at least. Tim Sweeney is always described it as you know, the metaverse engine. So this is going on at the same time that we're making the experience. So the experience is to showcase UE5, but it's also to showcase you know what you could do with a cinematic metaverse like destination. So in the experience, you see, yes, you could, you know, put performance storytelling performance by actual known people, actors and such, you know, like cinema, but volumetric so there's volumetric cinema at the beginning. Then it transitions to actual gameplay, you know, seamlessly transitions into we're now in the story We're actually having an interactive experience, it's fairly simple on purpose, because, you know, it's, it was such a big deal just to try to get something so complex to come together in like a year and a half. So, you know, it's a simple sort of chase scene, shooter scene. But they're, you know, you know, all of these things are sort of taking place inside this, inside this simulation world, the last bit, is actually probably the most interesting in so far as, you know, the end of the, the end of the demo is really the beginning of the sandbox of Matrix Awakens. Because once you've gotten this little tease about, you know, types of things you can do there, basically, the city is, you know, yours, you can, you can now wander and drive and mingle around in people, it can be obviously, so much more... much more can go into this, you know, whereby, you know, sandbox experiences can start being put on top of this, you know, infinitely right. And it's a pretty wonderful gift to give to give developers and independence because this is very difficult to build, for even a talented company, it'd be really difficult to make this kind of, to get this starting point. So it'll be interesting to see what happens.
Tracy Harwood 16:31
I think it's a gift for machinima creators, because to me, what you've got there is a is a thematic open world sin, and 1000s and 1000s of playable assets. Which, you know, it's, you know, I saw, I saw this great little machinima which tried to compare it to Grand Theft Auto, which I think we reviewed that for our February films thing. But what struck me about the Experience is the level of detail down to what you can see in the windows of the buildings, to the vehicles and all a lot of it's just absolutely stunning detail. So it's a real, it's a real gift.
John Gaeta 17:11
I agree. I mean, it's a crossover moment, right? Because machinima we've been using this term, it's in a sense, it's like, overtaking a machine that was made for one purpose and turning it towards storytelling. And, but you know, we do this at the same time, you know, fully, you know, knowing that, you know, the real world where we can take cameras and shoot things and tell stories like that, that that seemed like some like really like high plateau in terms of visual, let's say, visual fidelity, compared to where we began in machinima. And we're like, kind of like, there. We're like, we're like, there. Right? We're like, really one little beat. And then, you know, machinima is cinema. Right? It's we're about to have this crossover moment. Yeah. And this is going to be amazing, because it's going to unleash, you know, all manner of new new types of storytelling.
Tracy Harwood 18:11
Absolutely. We'll come on to that a little bit in a minute. But I just wanted to ask you one more question about this. What's the rationale for PlayStation five and an Xbox S rather than the internet? And I guess really? Well, because one of the questions that the audience might be interested in here is, when can we expect to see it on a PC?
John Gaeta 18:32
Okay, it's a great question. There's a reason for that. And that is, it's a practical reason. Those systems are like, bedrock benchmark understandable right there there. There's no fluctuation from one users, one user to the next. So all users of PS5 will have exactly the same quality and, you know, you know, capability and, you know, with PCs, even though we can do extraordinary things on PCs, it's not democratised, not everyone has the most, you know, souped up system to get the most output you know, the, the highest quality experience. So in a really simple way, it was a starting point to stabilise you know, to create a stable predictable outcome when for Awakens. But it also is to, you know, as well catalyse new again, new forms of interactive entertainment on the game on the game consoles. It I can't say about when because I'm not in Epic, right. So I can't say when they will do it, but obviously, they all know that extraordinary results can, you know can happen on PCs just for this first effort? You know, you have to imagine like, okay, it's got to arrive on time, it's got to play reliably it can't break. It's, it's got to arrive on time because it was, you know, coming out within the season of Matrix Resurrections. So they really couldn't, there wasn't couldn't be fail fail problems, or people having a lesser experience that wasn't desirable, right. However, it's not, you know, it's not at all unimaginable that they will allow it to be on PC at some point, because they do want people to sandbox and sandboxing can happen on all these all these platforms.
Tracy Harwood 20:45
Yeah, probably the creators that we're talking about most, possibly most familiar with, with PC based tools and techniques, and what have you, rather than the PlayStation I would have thought. But But yeah, I completely understand where you're coming from on the quality of experience.
John Gaeta 21:02
Yeah, I would say that's a good question to redirect. You know, we could both do this. But we'll redirect that back to Kim. I mean, you know, it's a simple question we could ask him, you know, like, when, when this becomes truly available for everybody, will, you, you know, enable a PC version. You know, they these, this company tends to use GDC is like a inflection point, they tend to, like talk a lot, then. Maybe that would be a good time if it was possible. But I don't really know what they intend.
Tracy Harwood 21:36
Sure, sure. I wasn't trying to catch you out there. And before we move on there, just tell us a little bit more about the Bullet time shot, how that was created and why just why has that become such a seminal moment in our filmmaking history? And I And it's truly inspirational for machinima creators, I think. But just tell us a little bit more about that.
John Gaeta 22:01
Well, I mean, I think everybody on this listening probably would agree with the general statement that you put everything you put all of yourself into everything you do, you don't do anything, thinking it's going to be you know, you know, award worthy, for example, or noteworthy just do what you do, right. And when we made that shot, they were multiple shots, and they fit inside of a particular, you know, sort of philosophy we had during the Matrix trilogy, which was, we wanted to do more than, you know, create a visual trick we wanted to, we wanted to attempt to, you know, create some methods that might be the types of methods one might deploy, if one was making virtual reality, actually. So when we made the bullet time shot, and then the, that was like a word jumble. And then when we made the made the made the, the bullet time shots in The Matrix, and then we did more bullet time scenes in the next two films, we really started thinking about how one would capture reality, such that you could essentially, you know, have something that would be termed these days, computational photography - photography that was transformable into three dimensional. And the reason why you would need that is because only only in a in a simulation, would you be able to cheat time and space, you would be able to, you know, manipulate time, you could be in as many places at once concurrently, if you wanted to, you could have multiple views of things. You could do all these sorts of things that defy physics that you know, you you could never do in the real world. So I think that it wasn't just, you know, the method, it wasn't just the visual. I mean, it read in the concept came across in the visuals of it, right? It was it, you knew that you could not move that fast. Even though slow motion, you can never get around objects like that unless you were in some form of virtual reality, or simulation. And the reason why it resonated is because the whole narrative, and the underpinning, you know, concepts of The Matrix itself we're preceding the moment, right? We're all being led right to this mind over matter. Mind over matrix moment. And, and people hadn't thought about that as, I mean, some people have like Philip K Dick thought about that all the time, but the average person didn't really think about that. And so it hit the mainstream in the right way at the right time. You know, the dawn of the real mainstream internet age, you know, the rise of gaming, and, you know, things like this were going on. And so the audience was kind of softened up enough by that, to be able to comprehend it. So it lasted. But it really was sort of you obviously, carried on the underlying premise of The Matrix itself.
Tracy Harwood 25:49
Yeah, absolutely. And it's, you know, it's the sort of thing that's exemplified in the Resurrections version of well, as well, because you kind of really play on that theme in that in the last part of that film, where the characters are kind of,
John Gaeta 26:04
So to go, so now to go to the next step, you know, so we put it in, even though we didn't have enough time to do all of what we were thinking, but, um, you know, so we had, this is kind of like, actually, a part of the Awakens story that I think is most is really, it's quite interesting, insofar as we essentially went back in the time capsule and pulled out data from 20 years ago. And we literally used these captures that we had, you know, photographic, HD, HD video, scans, all these things, right, and we pulled them out. And they were literally stored in a salt mine, you know, in Utah, somewhere underground, they were like, in an underground storage area. And like, you could just imagine the Warner Brothers assistant like with like a minors, you know, miners headlight, like searching through boxes, and came library with his, like photographic memory, saying, it's, it's probably going to be in this box. And the file names are this that he like, literally, remember, remembers all the file names, he's like, You got to get this. So they like rescue this ancient data, right? And they have to, like, essentially convert it, you know, and they bring back they resurrect this, this material. And they, they use it to remake like, a modernised version of virtual Neo. So, so what what I, what I find really interesting is back then we were like, hey, you know, one day people are going to do this all the time. And they're going to, you know, create virtual versions of themselves. And it's going to give them this strange form of immortality, you know, on the in the, we didn't call it the metaverse at the time, but like, online, virtual, you know, immortality. And this will be a really big thing. We talked about it all the time when we were making The Matrix. And then here we are 20 years later. And we essentially resurrect this old data, we build a modernised version of Neo which is really fantastically made. And now here exists this new Neo, right, there is a Neo now that could be you know, deployed in a simulation or driven by Keanu, right in a world. So Keanu has, you know, he's one of the people who has, you know, a nice, you know, a nice meta clone of himself. So anyway, we use this to reproduce the shock, but really, this time around. Rather, even though even in Awakens again, it was it was part of a 2D shot version of before you got into the playable. But if that material exists, it's possible for us to actually go in and walk around this bullet time shot, we could do it in VR. And more, right, so we're getting closer to actualizing that stuff and it was built upon this ancient stuff. And now it's going to keep going. One thing just is keep it back to storytelling. I mean, one thing that I really would like to do and I don't know if it's gonna get done, because it's it's up to Warner Brothers, which is really difficult, you know, path because it's a lot of people have to make decisions based on economics rather than passion, but like the the passion request that I had was, maybe we need to continue The Matrix inworld you know, Not just on film in 2d. And it could could easily imagine immersive stories that can still be the actors driving themselves. But immersive stories that you know, can have branches into free exploration of the world and participation and all of these things. So here's an example of where, really the vast potential of, you know, machinima or virtual cinema intersecting virtual worlds and open worlds, right, it's a perfect combination. You can consider the consider the story paths, like a beautiful sculpture, you know, that's sitting inside of a dynamic, open world, you could have many, many of these sculptures, threads criss crossing, right on and on. And you could, you know, you could choose to enable the sculpture and sort of follow along, and sort of see and experience it, you know, you could do it, if you wanted to do it through a composition form, you can, you know, you could do cinematography, you could, you could take the composition form out, and you could watch it dimensionally as you choose. And so let's say, jumping around, you could do that in an XR sort of way. And you can just decide to go out the side door, leave the sculpture and just enter the world and start doing things there. Meeting people having experiences inside of that world. And that is definitely we're like on the precipice of that. I, you know, there's little flirtations of experiments like that. But this is what will turn out to be an incredible dividend of people working with game engines to tell stories that they'll actually just be able to have multimodal to be able to create multimodal experiences. So I don't think it's it's not it's, it's, you know, there have been groups that have experimented with that here and there, like Telltale, and what have you, and like you, you do see cool experiments here and there were games interactives handoff like a little bit like we did with with Awakens, but there's, there's definitely a lot that can be done in the new grammar of this.
Tracy Harwood 32:33
Yeah and I guess one of the, one of the areas, because we're now getting to the point when we've got pretty robust movement, gesture and emotion tracking kit, and that, I suppose, coupled with things like AI and predictive analytics, which might fill in the gaps. I mean, I guess you're now talking about how these things can become what we call them NPCs, social entertainment experience that, you know, that kind of things. So they become, like, you know, live characters, depending on your interaction with them. And whatever platform that you're on, I suppose.
John Gaeta 33:06
Yeah, it's a super interesting area. I totally agree. And I'm experiment I'm, I am doing some experiments there myself. Yeah, I mean, you can't expect I mean, like, it's gonna take a while for the behaviour, of, you know, how people engage with virtual worlds to really, you know, mature. You know, you know, right now, it's all sort of new, and people aren't sure, and people, it's not what they're used to doing. So, it's, it's not to be expected that you're going to have worlds that are filled with actual humans driving their avatars, there's going to be the early adopters will go, you know, there'll be enthusiast and early adopters, but a world really needs to be like filled with characters and events and things so there's no doubt that training NPCs to understand world logic, believe they're in the world. You know, not just be tourists like narration, like tourists machines, they, they need to be able to, you know, sustain a place in the world's you know, makeup, a role. And so I can easily imagine large populations of AI guided NPCs that that help you get a lot of depth and exposition. If you're wandering off path. That's definitely coming. It's not I haven't seen it a lot yet. I've seen it. I've only you know, the there's a lot of experiments, it's, it's getting to a point where it's getting fairly clean, like you can, you know, you can, for example, happen upon some article on the internet written completely by a bot and you might not know it, you know, it's getting, it's getting fairly sophisticated.
Tracy Harwood 35:24
I was thinking like Red Dead Redemption, you know, the characters in that they're kind of pretty sophisticated NPCs, I think?
John Gaeta 35:31
Yes, yes, yeah, yeah. And they're well considered well thought out, if that's the whole bit is like how, I mean, it really opens up a huge new area for a form of hybrid writing, sort of, like, writing plus, you know, training, right from other materials. So you could have like, you could have a character that has a role in some, let's say, it's like, some game about the French Revolution. And, you know, you could have a bot that, you know, has a particular purpose within some, some narrow moment in the game, but also have a complete a comprehensive understanding of the French Revolution as well. Right, and all of the things that went on in those times. And so you can kind of toggle between very specific narrative, and more general discussion about the times that that whole experience is set in. So I do I do think that there's like, incredible, like, strategies, so that, you know, one could try were sort of like a hybrid of a traditional and and now kind of more, you know, AI based generative stuff.
Tracy Harwood 37:00
Just to take you back a little bit, and how has the development of machinima from from your understanding of it and real time techniques, which I think you probably far more familiar with, influenced your you and your work over the years?
John Gaeta 37:15
Well, I just put it this way, I mean, the same attraction, that I think folks who are working in being independent side of machinima, I've had that same attraction to that way of working in the potential of working like that, for my whole career. And I would say, I'd like I kind of crossed over, like, I crossed over around 2010 ish, you know, when I realised that, like, I don't really want to work any longer. In a medium, or I don't want to spend the majority of my time in a medium that has an end, you know, at the end of the story, right, I want the world to actually continue to persist until many, many more stories inside the same world, have them accrue. And as we talked about before, be able to experience the world once I've understood what matters by way of a story, that story gets you the emotional bed, it it, it tells you like what matters in the world, it could potentially give you, you know, your own reason to be in the world. And you can't really do that in cinema and cinema. You know, it's a storyteller's mastercraft. And, and machinima can be that way too, except for once the movie is over the world is That's it? The world has stopped. Right? You may have to jump to a whole nother whole different medium, like a gaming version of that world. And I'm not sure I like the having to do the jump, I kind of would like it all to be in one place. And that's what I started realising. So you know, the answer to your question, I think, is that since literally since the beginning of people, coining the term talking about it, today, I've like, slowly sort of migrated my way over into the realm of it, except I'm trying to, I'm trying to, you know, I'm trying to get all these other elements as well, at the same time, which is, you know, the Matrix Awakens project has to some degree shows you little slivers of the elements we care about I care about.
Tracy Harwood 39:39
Absolutely. And I think that does come through. But you know, I tend to think to your point that really we're not talking so much about a metaverse but a multiverse. Um, yeah, I had
John Gaeta 39:51
well, well, there's a debate on the term, right? The metaverse can be the multi, they can be one in the same, right. It's all I think It's a it's kind of open to interpretation. I mean, you don't have to I don't believe you have to say meta verses right? I think the metaverse is the multi, I think they're one in the same infinite worlds inside.
Tracy Harwood 40:13
Well, really where I was going is that you take your take your preferred world with you, your, you know, your, your immersive experience with you across all these different platforms. And you know, then, you know, depending on your mood and your and preferences and whatnot, you can sort of live in that world and across all these different platforms in and it could be multiple different. Yeah, experiences, yes, well,
John Gaeta 40:39
you could, you could be in multiple worlds at one
Tracy Harwood 40:41
digital skins, you could change your digital skins is kind of where I'm going with that as best
John Gaeta 40:47
you can, you can change your skin, you could have the same skin. You know, you there's, I think this is kind of like it gets a little meta too. But like, you don't always have to imagine yourself being in a world whereby it's actually you, in a live human sense. So for example, if, let's say that I, you know, spend a lot of time in some, you know, virtual world that I like, the whole time that I'm in this virtual world that I like, I could be training an AI clone of myself, right? Such that I can leave that world, you know, and enjoy my real world life, which is arguably better, much better. You know, it's fun, I don't want to be like deny the fact that reality is pretty great if you make it so but but I could leave my virtual self behind to continue to participate in that story world or that other you know, that that world, right? Like, I think it would be very interesting. If I left my virtual me in a story world, and then I would basically get like updates like your virtual, your virtual you has wandered, you know, to this region, in the end has decided to join, you know, this group of people, and, you know, got in a fight in a bar and built a farm and you know, like it just like getting this, this Oh, I did that I did that. Okay, you know, and it would be based would be it would be based on the way you behave. You know, I think that's gonna be fun. If it was a good story world.
Tracy Harwood 42:44
It's what it would have to be, wouldn't it have to be to be compelling, I suppose for you wants to want to do it. I got, I suppose another observation really, I mean, kit, to me, seems to me, the biggest barrier to immersive real time, kind of mixed reality experiences. And, and that's not just the cost of it, but the learning curve associated with you know, understand the controllers. You know, access getting hold of it, like PlayStation 5 is, like hen's teeth at the moment. You know, and then other things like battery life and lightfield, and so on. What's your view of things like Disney's I don't know what you call it a holodeck like that an application that they announced a couple of weeks ago? Is that the future? Is it scalable? Or portable?
John Gaeta 43:30
I believe what you're, I think what you're talking about is their AR without glasses. metaverse like announcement. Okay, well, that did just to be okay. And I what I believe that probably is, you know, the Imagineers have, for many years been really, I mean, like, when I say many, I mean, like since like 50s, or whatever. And 60s have been masters of using projection technology to create illusions, and they, they do this mixed media thing that's fantastic. If you go to the parks, where often you it's like projection mapping, in a sense, right? Before projection mapping was the thing in the outside world, they were doing projection mapping. So they're quite good at it, they quite and they do understand how to create all sorts of depth illusion, but not just that they you know, they put characters into little vignettes, and they and all of this stuff, and what I believe that infers, is that, you know, they know that people having glasses and goggles can be a barrier, it can be friction until until you can have them you know, beyond you know, 100,000 guests, you know, and not break and function and all of these a lot of stuff that has to be right to get it to work. They they really have a very have a way of doing that in a sort of hybrid reality sort of sense. So they would do really high end, projection mapping is the best way I would put it, you know, they could if they if they could, they could if they wanted to, they could very likely pair it with augmented reality glasses to create, you know, even more. But I do think that the way that they're using that term is is, is, you know, pseudo holograms, if you will, that are achieved with a fancy projection. But what I would imagine also that they would do is they would make these the content that these pseudo holograms are based on in a particular way, like, Unreal or game engines, right, in a particular way that it would be a short leap for them to add classes to that. Right and like, extend it and have the content escape and run wild. So, but I almost forget your, your your.. What was your question again?
Tracy Harwood 45:53
Well, the question, what do you think it's scalable, but I guess the only way you're saying that it might be scalable, if they match mix and match it with glasses and whatnot?
John Gaeta 46:03
There's okay, there's a few things I think that are scalable. I do I think that there is a way for them, and not just them, but any of us. And okay, this is not going to answer your head to get tools in the hands of everyone question this is this is what is plausible question. Answer, it's plausible to create a an exquisite, flawless, replica, dynamic digital twin of a place like Disney World, it's possible to imagine that world being available on screens of all kind through like, you know, big cloud streaming type of solutions. And it's possible to imagine that it's not just like a model without people, it's possible to imagine the people being represented by way of sensing people in the park and intertwining it with other media, there's a lot of things that I think are possible now, where it's it's, it's, it's conceivable to have a virtual Disney World that actually is a real time mirror of what's going on at the moment. And literally down to the potential of you know, finding one person in the park and talking and talking to them in a sort of co-presence, XR co-presence sort of way. So those the I do think that the tools are all there, if there is a business case for them, that they think that it would, you know, lead to big audiences, which I think could actually lead to like having millions more people in the park in a virtual sense, in a co-present sense. As far as like democratising tools and making it possible, I think a combination of a few things are going to kind of come into into effect across the next five years. And that would be well, I mean, I just saw a story that you can now put Unreal in the cloud, you can just basically have a licence in the cloud. And you can you could work remotely, which is already already quite interesting. But I also am watching closely trends, like, you know, AI, you know, natural language process like GPT3, or maybe the next GPT, three GPT GTP4 enabled super intuitive, how wouldn't put it, UI for creation, right. So like, build me a castle boink, you know, put in you know, I want horses, right. Let's have a snack. What's that?
Tracy Harwood 48:42
That's Canvas now... we review
John Gaeta 48:45
Canvas is a part of it. It's a two it's two dimensional, but looks three dimensional is part of it. But we will be we will get to three dimensional. But Prometheus AI, I don't know if it's I don't know how much of that is truly workable right now. But it seems very interesting. There's other things but I think that's the overwhelming direction that what what where it needs to go simply is that I want to tell a story or I want to create a world or I want to create an experience and I just say it and it's so. So that's what everyone needs to work towards.
Tracy Harwood 49:27
Do that I'm gonna read something to you because because I was fascinated by your comments when you when you did that piece for Paul Marino's book, back in 2004. I don't know if you remember this, but you gave an analogy of what you said was the current status of life as being caught in the cross dissolve from a mammoth bone to a spaceship in 2001 Space Odyssey. And you said then we were roughly 12 and a half frames into the Kubrickiqn 16 frame evolutionary process with with the last few frames you said being supported by machines and that those almost thinking machines were not quite lifeforms. So my question is, where do you think we are now in that cross dissolve?
John Gaeta 50:15
Yeah, that's a good thanks for bringing that up. I didn't know what the metric was. Yeah. Okay. So if we're gonna use six, how many frames that I say 16? All right, and I said we were at 12 in 2004. Okay, well, at that pace, we're probably at golly, I've only got four frames.
Tracy Harwood 50:41
We can add, probably two or three frames further.
John Gaeta 50:43
Wow, we're close.
Tracy Harwood 50:50
We're close, but not quite there. That's the that's the
John Gaeta 50:53
that's, it's coming together. I mean, like, it depends, let's say how many years since 2004. To now? You know, it's an 18 years, or it was out. Right. So I would say if 18 years equals to three frames that we need, approximately? Yeah. Another? Yeah. I think I think that math works. I mean, like, I think it's inside this decade, for sure that we're going to be at a place where I mean, I think it's coming in the next few years. The question is, is it going to be, you know, put in a form factor that is, you know, commercialised so that we can all use this stuff. So I would say it's going to be across the next half decade, that we get to these, you know, almost invisible, super powered tools, you know, to create worlds and tell stories, and I was gonna ask you in real time?
Tracy Harwood 52:05
Well, to that point really was what's your take on Nvidia's Omniverse platform?
John Gaeta 52:10
Yeah, I think it's, that's definitely going to be very significant contributor to this, what we're talking about, I think that's really what's interesting about it is the way they're approaching it is they are finding a, you know, a common way of tying together different spokes, you know, for example, they're they're tying simulation, real time graphics, they're tying AI, you know, digital twins, all of these things together in a way that you can, you know, have those pieces interchange, you know, their, their advocacy of universal scene description, you know, that the fact that they made a decision, which is everyone else's debating, and they seem to have made it to a decision on how to allow all of the pieces to relate. That's only one way of things being able to relate to one another. There's also the world of smart contracts, and all of that, that whole and of things that are going to come into play, but I, I think that the capacity to create flawless simulation that, you know, AI sort of is heavily under the hood enabling is something that Omniverse is uniquely positioned to help us do. So I'm, I'm very excited by it. And I have a number of colleagues experimenting with it. I can't say that I've created a project yet in Omniverse, but it's absolutely a goal a near term, a near term goal of mine.
Tracy Harwood 54:17
Again, changing the subject we're getting to the end here. You posted something I found really fascinating couple of weeks ago, which may or may not be related to that, but it was about creating interactive 3d content and putting it on the market. Can you tell us a bit more about your thoughts on the role of NFT's and WebEx experiences generally? And how might that influence or help machinima and real time creators?
John Gaeta 54:46
Well, I think everyone's trying to understand it at the same time, the need the need for a a premise whereby a digital, you know, object or item, or any kind of digital media, you know, had some ability of being unique and singular, has always had been the barrier for digital art being taken seriously and digital artists, not just digital artists, but like all kinds of artists, right? Being able to, you know, have a piece of media that can have value and accrue and retain value like physical things can. The question is, you know, because before that time, a piece of digital art or media could be copied infinite, it's they can, you know, these things can be copied infinitely, but there, and so people never thought there would be a way to create value in one of those copies. But this, this system has opened the doors to that, and it's been talked about for quite some time, the, the, the need for this. Unfortunately, you know, it's a wash with a, you know, early provocateurs, you know, trying to make a lot of money selling a dream of value that may or may not really exist. The dream that I think is being sold is that if you buy this digital item, then this will accrue. As we move towards really the underlying the underlying messages as we move towards a, a digital life of a life in the metaverse, these, you know, ownership of these things, just like ownership of things in the real world will, you know, you know, will be a valuable and for you in the future. So, the thing is, though, is that those who, the means that people draw attention to which digital things have value, and which don't, is really questionable, it's a very hype base and a lot of manipulation of perceptions of what matters and what doesn't, like, this is important, because, you know, it's the beginning of the metaverse and you can have a piece of that. So I'm, there's a lot of people who really are in the business of trying to create very, very valuable experiences and virtual content, that look at this, you know, with some degrees degree of shock and disgust because it really is, you know, it's, it's a manipulation of people, you know, so, you know, in order to, in order to inflate the value of currency, actually, not the objects themselves. So, so, but does that mean, it's bad, it's us, it's not actually it's like, that eventually will like anything, a lot of that hype is gonna burn away. And people will start being able to determine what might have value in my what might not so that, to me, it's the that is the essential question for everybody is like, what has intrinsic value, and it could be, you know, different for each person, you know, what it may be, like, you know, if you created something yourself, then immediately you have an attachment to it, and emotional attachment to it. And like, and that wouldn't be any different than an artist in the real world doing that. Some people might find the your creation, genuinely compelling. Your creation could be there's, you know, infinite form factors. So, we need to have a much more serious conversation about what has intrinsic value now I think that goes now circling back to your, your, your, your question, I think that there could be a lot of incredible experiences to be had on remarkable works in machinima. And and I think that the, the question is, you know, will others you know, that are in a peer group that we care about, if they if they agree that there is value there, then then there's a future for machinima and NFT. Just like any other For bid NFT I think that it's really more about, you know, what, what's going on under the hood. It can be like spiritual, it could be technological, it could be like a philosophy of design, it could be, you know, you know, it's, it's like anything else, it's innovation. So why is the Picasso worth what it's worth, you know, you know, what was Picasso doing at the time, that was different than everybody else, he starts a movement in a direction, right? And, you know, paintings are interesting, just as a way to, to relate to because, you know, they they off, you know, some are literal, it's some are more metaphor, some are conceptual, and the concept, if you understand the concept, that's, that's where you might believe the value lies is, oh, that's a remarkable concept. And it floats to the top, you know, in terms of, you know, being viewed as valuable. So that's the way I think things have to be viewed.
Tracy Harwood 1:01:03
I was wondering if you if, you know, from your view of it, whether you thought there needed to be some sort of intervention, which might make it a more democratic environment for creators, rather than let the market sort it out, as markets just fail?
John Gaeta 1:01:16
Yeah, I think these particularly these markets at the moment, are highly manipulated. They really are, and I think that they will get sorted out, at some point, they'll either be sorted out through regulation, or they'll be sorted out by the, by the consumers of, of these things, who will who can vote with their pocketbooks with their wallets, right. But what what the real, the real sort of distortion on on, on things, is tends to be how social media and influencers, you know, sort of, you know, cause confusion about what matters and what doesn't, and that's really the main issue to work out is who should you listen to? It's, it's, it's, it's nice to have, you know, some item registered on a blockchain so we know it exists, but it's a little bit different to know if you know, these 10, you know, groups or people that I believe in and trust their points of view, because they have a whole lifetime of, sort of correct points of view, or like points of view I agree with, if I, if I look to these, these 10, these these 10, you know, people and they know what they're talking about, then I'm like, Okay, there's probably value there, right? Because there's consensus in some way, but I don't really want to be only looking at irrational market behaviour around a piece of art, you know, without the consensus of humans. Right. Also in the mix, you know, humans that know something about art or appreciate it at the very least.
Tracy Harwood 1:03:16
Absolutely. Well, with all the kit and software developments coming down the line and with I suppose the attitude of many of the software developers taking a much more open approach to supporting creators I think it's, it's never has never really been a better time to be a real time creator and content fan. Question is what advice can you give the indie machinima and real time creators today? What What big bit of advice have you got for them?
John Gaeta 1:03:46
I heard the question. Sorry. Just checking the calendar. Um, what advice? Um well, I'll give one answer that may seem obvious, and maybe one that's less. The obvious one is to really, don't try to make what you think is wanted by a market or an audience - make what you are passionate, you know, by and it's just, it sounds it's it sounds really, you know, what, it may sound simplistic but passion. Passion is the one of the most important aspects of your work. Like you really need to believe that you need to you want to tell something or you want to express something or, or you know, create something that is, you know, a reflection of the way you feel and the way you think and to not allow, you know, overthinking on commercial, you know, opportunity to interfere with that. I would say that because the the folks, you know, you can break through just trying to make something that's commercially desirable. But the folks that really I feel like personally have a rewarding experience in a career are ones that stick, at least intertwine their passion, as much as possible. And it's easy enough to say, I mean, it's hard, it's hard to always get to work on stuff you're passionate about, but at least work on something at some point that you're actually passionate about, perhaps you have a balance, right, you got to pay the rent. So you have to do a balance, but don't have an imbalance towards it's only strictly, you know, what you think other people are asking for. So that's one thing. As far as the, you know, the other bit, one thing I find fascinating is that things are moving so fast in these tool sets, that I feel like the trend is overwhelmingly in the direction favouring independent artists, in now, and across the next couple few years, I really think this is an excellent time, you can teach yourself in so many different ways now, like literally you could learn through YouTube, a lot of things, but there's a lot of ways you can, you can educate yourself. And if you feel intimidated by technology, you know, and having to learn, you know, a lot of methodology that could be complex, there's also new things coming whereby you, you don't really need to be a master of tools, you just need to be a master of you know, what you imagine you want to come out on the other side, the I think that the, the tool sets are overwhelmingly moving in the direction of intuitive and some things that creatives can use, right? So you can, you can, if you're creative, it's I think we're heading into a time where it's going to be plausible to make something of some sort, you know, with very little, you know, detailed knowledge. Now, you may not be able to get to something really sophisticated without help from friends and colleagues who do master tools. But it doesn't mean that you can't get in the game very quickly, you can get in the game very quickly now, and you should because at the end of the day, what matters is what you're saying, you know, what you're saying matters more than, you know, the grandeur of the of the graphics. So I would, I would I would be encouraged if I was a new, you know, real time artist for that these years now.
Tracy Harwood 1:08:16
That's great advice. Thank you. Got to ask this final question. Are you a red or a blue pill person?
John Gaeta 1:08:25
Oh, you're asking that? I'm
Tracy Harwood 1:08:28
sure he'd been asked him before.
John Gaeta 1:08:31
Yeah, I know. But like in our culture, everyone's trying to hijack the colours. Oh, and I'm gonna say I'm full spectrum. I'll go with every shade of colour depends on the mood.
Tracy Harwood 1:08:46
Fair enough. Thank you very much. I have no more questions. John. It's been an absolute delight to have you on the show. Thank you so much for taking the time to talk to us today. I'm sure our listeners will be absolutely fascinated in the in the observations and comments that you've made. Thank you so much, and bye for now.
John Gaeta 1:09:09
Yeah, nice to see everybody. Cheers.