The Spatial Reality Podcast

Tilt Five founder Jeri Ellsworth stops by to explain how her $359 AR game table has succeeded where many XR products have failed, and why looking to the early days of personal computing is our best bet for understanding the future of spatial computing.

Show Notes

Tilt Five founder Jeri Ellsworth stops by to explain how her AR game table succeeded where many XR products have failed—and at a cost of only $359. She also explained what you can learn about the future spatial computing from the early days of the mouse and the iPhone, the first spreadsheet software, and the Silicon Valley Homebrew Computer Club that birthed era-defining companies like Apple and Osborne Computer.

Links:

Creators & Guests

Host
Sean Higgins
Writer/copywriter/editor/podcast host. Mostly tech, other things too.

What is The Spatial Reality Podcast?

We host one on one interviews exploring the businesses and individuals that are defining the applications of spatial computing. We aim to show you how spatial computing can change your business and your life—not a decade from now, not in a few years, but today.

Welcome to the Spatial Reality Podcast, your resource for authentic conversations about spatial computing technologies. I'm Sean Higgins, your host. Every few weeks I'll share a new in-depth interview with a leader in spatial computing. I'm casting my net wide trying to find experts who can help us understand how this technology is going to change and is already changing a huge variety of.

My goal is to offer hype free information about spatial computing. For these first three episodes, I pulled recordings from my most popular interviews from the last year. These recordings were made so I could publish text interviews on the website, and that means we used Google to record not ideal.

Still, I think it's worth sharing these so you can enjoy the insights that our guests have to offer. Enjoy it and keep an eye out for more podcasts coming very soon.

When we met Jerry Ellsworth at last year's Augmented World Expo, the XR Industry's flagship event, her company Tilt five was having a phenomenal day and their new holographic gaming system was pulling huge traffic. With good reason too. Slip on a pair of their glasses and flip open a game board. And the system projects a crisp, focused 3D world that you control with a wand in short, where many mixed reality products in the past have overpromised and underdelivered Tilt Five's new system does exactly what it says, and at a cost of only $359.

When we caught up with Ellsworth after the event, the reason for that success became clear. She walked us through a career developing consumer gaming devices and toys, heading up the r and d department at Valve, which made key contributions to the tech behind the HTC Vive and Oculus HR headsets. She chatted about the early days of the mouse and the iPhone, the first spreadsheet software, and the Silicon Valley Home Brew Computer Club that birthed the era defining companies like Apple and Osborne computer, and each of these topics unrelated as they might seem offered an important lesson about navigating the present and the future of spatial computing. Enjoy.

Does the term spatial computing mean anything to you? And if so, like what do you think of this idea?

I, I think it's really interesting. I think about it a lot and I try to compare it to things in the past as well. Uh, I have this feeling spatial computing is going to disappear partially out of our vocabulary.

Spatial computing is just gonna be one of these things. It's gonna be like the internet. Everyone just assumes and knows what it is when it becomes Right. I like your point, which is that it seems to be that it, it'll be pervasive and common enough at some point that it's not something we're going to have to name.

It'll just be a way that we interact with data or with computers. Computing has been, Evolving since the sixties. Like you just look at really pioneers in the ar vr space, even back to the work that was at Stanford. And it's this constant evolution and it really feels like it's the next evolution for computing.

Mm-hmm. , we've been stuck 2D screens for a long time. People are used to working in 3D space. I know how to pick up my cup of coffee very well. Why aren't we computing? Yeah, of comfort. I'm really fascinated with this kind of like sudden chatter about spatial computing, but it's been the thing percolating for decades and decades.

There's been little pieces and signs of it forever. A phone's got gps, so now we can move through patients and it augments our life. Or something more on the fun side, like the, we. Controller. It was super primitive, but it was super compelling. The first time people got to use and it was super intuitive.

So it just leads my belief that this is the next computing platform because it has this ability to be so intuitive for even non-techy people. I, That's interesting. Thinking about the idea of virtual reality, of course it's not new, it's been around for decades, Did we? A tipping point with technological development where all of the stuff is scalable and it's actually, it's practical.

The idea's been around for a while. Is that, Is that the change? It's technologically practical. I think that's part of it, right? The stuff they were doing at Sloan third and NASA is probably millions of dollars just to have a pretty broken experience. Yeah. The nineties, we got a kind of a taste of it with Virtual Boy and some of these location based experie.

I also have a different take on it. I think that the excitement around like virtual reality is a little bit misplaced. I think in reality, augmented reality type experiences come first. That's gonna change user behavior. They're gonna become very comfortable with these more mild type experiences. And then somewhere down the road, a decade or.

Virtual reality will be just a subset of what we're already doing. So we'll have our glasses or our magic pixie dust we put on the table. It creates holograms, but we'll be passing in and out of fully immersive, partially immersive experiences all the time. Even when I worked at Valve, I was, there was a lot of effort going into vr.

I'm like, no, no, no. comes after ar you. You have to move user's behaviors and train them up. You can't just drop them into virtual reality. Make expect that it's gonna be mass market. So, This is a similar argument people make about autonomous vehicles, right? Like you, you scale people up slowly, allow them to become comfortable with the idea of autonomy dead on, right?

You look at what Tesla's doing with autonomous assist, comfortable with that after how long it's been going, 78 years now. We're very comfortable with it. Yeah, maybe too much

but yeah, I think it's all an evolution and it's gotta be done practically. Very much like the early days of home computers. Let's look at the mouse, right? Microsoft did something brilliant with windows, like people didn't know how to click and double click and move a mouse around. So they put solitaire and mind sweeper in as simple little experiences, get people doing all mechanics of using a mouse, and then pretty soon it becomes muscle memory and no one.

You don't even have to talk about a mouse or a touch pad anymore. It's just, it's our muscle memory. I originally was planning to ask, given your background, working on the sorts of products that you've worked on, why, why this? But as you're talking, it's pretty clear, it seems the argument you're making is the product you all have developed so far is the tip of the spear.

To get consumers and potentially business people, even enterprise level, like it's about users. It seems more so than it's about consumer or enterprise. Exactly. It's users. My background, I think the most formative part of my career is when I started working in toy design. Toy design. I worked with some amazing mentors that taught me how to think about the end user more.

The device that we're creating. And I actually see this a big problem in an XR and spatial computing space. There's so much emphasis on how many pixels or how much I can walk around the world with it and not enough focus on like, how do we start to acclimate users to this and, and delight them. Our whole thesis is, it's gonna be a lot easier to delight people in the early days than create.

New productivity type interaction. Yeah. So I keep going back Microsoft and mind sweeper, but it was like, that was brilliant, right? They entertained them, that got them, gave them enough motivation to go through the motions and learn it, and then it became a productivity tool. So I might have a little bit different outlook on even how this gets, um, adopted into.

Enterprise environments broadly adopted. I have no doubt that there's gonna be a lot of little narrow niches where it's gonna be adopted right away. It's just gonna take time for productivity to be developed and trained. That's why they're wholly focused on the living room. I know how to delight people.

From my experience in the toy industry, you come up with a minimum viable product, but it's just enough that it's. A game loop in it or an interaction loop that gets you engaged and it doesn't frustrate you, it just keeps you coming back to it. And that's just so critical. In the early days. I was talking to Al Al Horn, one of the original folks at Atari just a few weeks ago, and I was like, My gosh, when you were trying to put Pong in a hundred million homes, how did you do it?

And he's, It was, Horrible. It was terrible. We were just hitting resistance everywhere. No one could understand why someone would want to play ping pong in the living room thinking no further than the tip of their nose, and, but they knew that they were onto something cuz it was a simple game mechanic that everyone could relate to.

Everyone kind of knew ping pong or tennis and they knew it would be a hit. They just had to get over the hurdle, get into distribution, get awareness. The people trying it, it was easy enough that they could play it right, and that's how they did it. They blanketed the earth with P machines and introduced the whole world to in-home video games.

It was amazing. But that savviness that they had, that had to be just this right balance of interaction and ease of use. They didn't over-engineer it, they just made it just right. I could do a near eye display, I could do HoloLens or a magic type type device. I built those things back valve and other, but it's just not really a product yet for the max market.

That's a really fascinating way to think about it, and it's. Often a secondary concern, uh, for people, for people working with this technology. For people who are reading this who don't know what you, what your product does, could you just give a sort of a brief, quick explanation of it? Yeah. So our system is a headset that you slip on and a game board that you flip open on your table.

very, mm-hmm. familiar. Mm-hmm. Everyone knows how to open a game board. And then out of this game board, this magical world springs out of it. So your little game characters running all over. You can move blocks around, you can play puzzle games, you can collaborate, you can have the entire family around the table wearing glasses and playing these games together.

We also have a magic wand that comes with it that, um, you can poke at things and pick things up. Oh, it's funny about this. When we showed it publicly the first time, people were railing on us saying it looks like a barbecue lighter. And that is not a mistake at all. It went through hundreds of iterations of what's the best interaction for audience.

Everyone knows how to pull a trigger on a hot glue gun or a um, Barbecue wire. Everyone knows how to poke something with a stick. It's innate. So that's why we designed it this way. Like even my father who can't even use a mouse properly, when I hand him like our system, he can reach in, poke a game character and slide it around.

He doesn't have to think about it cuz it's so innate. Clever part of our system is because we weren't too wrapped up in trying to boil the ocean. Some of the big players like magic, we're like, Okay, people are not wearing this outside. No one's gonna wear dork glasses outside, so we just take that off the table.

Right? So we started eliminating use cases, so we just threw away like everything that we thought the users wouldn't do, and then we looked at it and like, okay, people will sit in their home and wearing the dork glasses with their friends and they'll feel comfortable. So that's important. People need to have intuitive interactions.

That's important. We need to have this magic wand. We need to be able to fill the table with the experience. We have this massive field of view that just fills the whole game board and table with experience that's important. You don't wanna be looking through little people like some of these more complicated headsets at the ergonomic.

And so we started jetting a bunch of things and we came up with this really clever. Developed this originally at valve software, this optical technique where instead of trying to ve the light directly into your eyes, which is insanely difficult and expensive and can take, could take decades to like master, we turned it inside out.

We have a special optical board that has this optical property called retro Reflection, send you like diagrams that kinda show how it works, but it projects out to the game board and the special game board redirects the light back to you Only. So by doing that, we removed all of these physics constraints around optics, and we were able to solve one of the biggest holy grails for ar, which is virgins accommodations.

So when you look at our image, because the light's actually coming, Through from the game board. Everything's in focus correctly. You can converge on it. Everything looks like it's really present there on the table instead of feeling ghostly or out of focus because it's focused very far away. It's so funny, like sometimes we talk to shortsighted folks and they're like, Yeah, but you have a D board and it's like, no, that's the, of this, it's brilliant.

It's, we call it AR somewhere instead of AR everywhere. You know where the experience is. It works perfectly and. , everyone knows how to fold open a game board. It's not an onerous task to do. Once you hook people this way, once you delight them and show, show them that AR can work in, I don't wanna say limited capacity, but a focused one.

Mm-hmm. . Um, what do you want to do next? If you talk about this being the start, then what comes after this? Yeah, I think fundamentally a lot of these, like optical challenges are solvable. I think in our case that we can go quite a ways in the entertainment space, but developers that are reaching out to us to develop, there's a large majority of them that are non gaming developers, so natural places we can go with it is more enterprise and support enterprise users and more professional users with different features that they might want.

So that'll be low hanging fruit. In the long run, I want us to be walking around the world with being completely immersed in our AR experience and not thinking that we really are necessarily. It's just making our lives more productive and delightful. Much like our phones are, like phones make our life so much better because we have a super computer in our pocket.

That's our goal in the long run, is to do that in even more seamless way, so you don't even have to pull phone out It. Of your life, as you mentioned, these enterprise users. I wonder there's some pretty classic uses for or potential uses for AR and enterprise. People talk about design reviews being one of them, but as somebody who's had their head in this space for a while, what do you see as some alternate possibilities, especially considering you have a technology that actually just works and solves a lot of the issues that might potentially have bogged down other technologies and these sorts of applic?

I think you hit on some of the early applications, like obviously you're gonna be using our system to pull CAD models in and collaborate on them. In fact, at E we had five different booths there with til five. Not us, but we had partners there. We were the only booth showing games. Everyone else was showing, um, various versions of enterprise applications and professional applications.

So, Company that does volumetric capture and video editing. Really cool. You might think about like a Adobe premiere for volumetric capture that can do some really cool stuff there. There's ability to edit and play back Volumetric movies and interactive content is like amazing, right? So you could do their technology to do training where you need to really have.

Spatial awareness about what you're training. We had another group that was working with the European Space Agency, so they use our system for command and control of the teachers that are teaching actual astronauts that are using a VR headset to walk through a life size version of the International Space Station.

They love our system for the teachers, because you don't wanna be in a full VR system. You wanna have all. Situational awareness around you. You wanna see the eyes of your friends and colleagues and so it's super valuable for them even though they, part of their system is fully immersive. It seems to me one of the values of a system like yours is that people potentially would look at it from a variety of industries or with a potential, like a variety of problems.

And then maybe see this is the thing they're looking for and understand how to use it in a way that you would not be able to predict. Do you think that's part of the value of making it so straightforward? Yeah. Yeah. I don't know. Like I always try to draw comparisons and see where we are in the timeline of things.

So I don't know. Let's just use the iPhone exam, for example, the iPhone. Is arguably the world's best enterprise device. Now, 15 years ago or whenever it came out, it played music, it did maps, and let you browse the internet. It was very primitive in today's terms, but enterprise developers looked at this and, oh, there's this really valuable problem I can solve by using this touchscreen device.

Inexpensive to get straightforward to use, and that opens up unimaginable, like uses, I can't even imagine. I wish I had a crystal ball. Then I would just focus on all of these vertical markets that I knew were going to be huge and successful. For us, what are we gonna nurture was just a no brainer for us.

We wanna get into a hundred million homes and the way to do that is just entertain people. Poor markets will follow and it has. Probably 30% of our developer inbound is non gaming. So then we also made a bunch of tools for our system. So one of my mentors years ago told me like, whenever you're making a product and developers are one of your customers, if there's anything like rough or difficult to do, just make a tool for them or an example.

And so our development environment is drop dead simple to use because we put so much time into these. We support the standard game engines and native plugin if you have a custom app. But yeah, say take Unity. You have a game that already exists, or let's say even enterprise that exists in Unity. It's basically drag and drop.

You plug it, take our plugin, drag it into Unity. Within five minutes after you click the play button, you see you're seeing rendered on the table. And. It's really important that this development cycle goes really fast, so it needs to be real time. So folks that are working in VR have take their headset off all the time.

They can't really edit the scene in real time. Very, yeah. Make that seamless. You can just wear the glasses, you can be programming, you can be moving things around on your 2D screen, and then you look over and they're moving up and down in the space. It's like really magical. It sounds delightful. . Yeah. I love, like when we have a new partner come.

They're like, Oh, is it gonna be difficult? Hey, do you have your project with you? I'll get it rendering in five minutes. There you go. It's rendering now. Now you just need to tweak it, and if you wanna take advantage of our cameras or our wands, then you can go off. I think this is super smart. Like I said, I edited a trade publication, but I didn't mention it was like 3D scanning te.

Did a lot of writing about lidar and photogrammetry and unity and all of these, all these various sorts of visualization technologies. And one thing I'm struck by the people developing these tools, Often don't realize that all of the, all of their customers are also people who maybe would use technology with their family.

They're, they would all play games if it's something that they could afford to bring home. Maybe they're playing a game with their kids and they say, Wait a second, this is exactly what we might need for this sort of, For this problem that we've been having, I could show, I could show people how to do this thing using this technology.

And then also by making it easier to develop you, solve the problem of adopting a new technology, helping people to understand it, developing applications or experiences for the tool in a way that sits into their workflow. So I guess that's all to say by focusing on gaming. In the way that you have, you've actually kind of cracked a lot of problems related to getting enterprise on board.

Oh yeah. I think we go back and look through history, like home computers, right? Home computers was pitched as you're going to do your home finance, you're gonna use this at work, you're gonna use a modem and you're gonna be more productive between work and home. Big dreams of what it was gonna be. But yeah, back in the early days, the Atari and Commodor and Apple and computers, they sold hundreds of millions of these things.

But what do people do with them? They play games primarily . They weren't. They weren't necessarily as productive as everyone hoped. You weren't dialing over a modem to work on a daily basis. You weren't buying your groceries. All these things existed like in the late seventies and early eighties.

Everything we do now, like door dash, like you dreaming about that then, or just downloading a game instantly to your computer. They were dreaming about that, but it just technology and the user. Muscle memory wasn't there and it just wasn't there. But kids like us cut our teeth on playing with home computers and improved them over time, and then it became like what it is today.

The same thing's gonna happen in XR and spatial computing is, it sounds like what you're arguing is that the potential for an intuitive interaction with this sort of technology is so high that. There may in fact be more people who are not young or who didn't grow up with the technology, who can develop things based on it.

I, I think it's gonna be a broad spectrum of folks that are visionaries, right? It's people can see a little bit further into the future than some of the myopic folks. So the Home Group Theater Club here in Silicon Valley, like it was a wide variety of people going to home brew computer Club, and out of that Apple, Microsoft.

Osborne, just tons of technology companies like sprung out of these groups of very diverse genders, races, ages. It feels like we're in that similar phase with facial computing and it's exciting. Usually at this point in the interview, I'd like to start asking people, where do you think spatial computing is on the hype cycle?

I'm. Very happy to say I believe that we're on the long tail of growth on the hype cycle. I don't know if you much of my background, but I was at Valve software on the apex of virtual reality. I had a startup prior to this where we made a ton of mistakes. I mean, this is part of the reason I have like laser focus now is my first startup trying to do.

In reality, I fucked it up really bad. You know, I learned a lot and that was, I think on the, uh, The peak of the hype cycle. Magic Leap was raising billions of dollars. And we weren't focused on user experience, we were just like, we'll build it and they'll come. And we got to the bottom of the hype cycle and my startup had to close because we didn't have a viable product, we couldn't get investment.

And wow, that was a learning experience and we're refocused, and now we're like on that nice long tail, which is the best place to. Do you think this is the time that enterprises can potentially look at this kind of technology and say, I can start thinking about applying this. I don't have to worry about being an early adopter and getting burned on a 6,000 headset that has a two degree field of view.

No, I. For all the enterprise developers and companies that are buying our tech. It's like a no brainer. Like our kids 3 59, that's background noise to them. They could buy it, try it, educate themselves. That's super valuable in itself. Like maybe even if there isn't like an immediate application for it. So that's one I'd say piece of advice.

The other thing is there's going to be really strong verticals where systems like ours are just like perfect, right? If you can identify this narrow problem that it just solved, then yeah, they should grab a system, solve that problem, and, and start using it right away. I don't think if I were an enterprise in a Fortune 500 company, I wouldn't just make an A mandate.

The future is XR and we need everything over. That's probably a fools errand at this. What's the biggest problem right now with this sort of technology? What's the biggest challenge that we still need to overcome? I think in the industry on a whole, our big problem is this notion that there's a silver bullet right around the corner that's gonna solve every one of our XR problems, which causes people to make bad choices.

So we've all heard it. Apple in two months is going to ship XR glasses that are perfect and they're gonna do everything. And I've been hearing that for seven years. It hasn't happened, honestly. And it's not helping as like group of people working on XR to be like bleeding, that Facebook is going to be releasing glasses that are gonna do everything we wanna do.

Like it's such a broad, open space right now. Our system can grab a big chunk of it. Facebook can do whatever they want. Facebook can do whatever they want. We're not playing in the same space because it's just so wide open. What's something we didn't talk about that you have a strong opinion about, or is there anything important that doesn't get much coverage?

Ethics. Ethics. Ethics, . I am terrified about companies like Facebook that want to use xr, Metaverse. Facial computing to manipulate users, they can call it whatever they want. Their business is manipulating users. And my experience working at Val Software is, and this is also, this is another example of my kind of naive growth as a person.

So at Val, they had me start the hardware lab and so they're like, Your mission is bring the whole family together to play. That's it. They'll figure it out. And so I hired a bunch of people. We started researching. We were doing things like we were hooking electrodes with people's heads and feeding it back into games to see if we could get more engagement.

We had psychologists there helping us get more engagement, and we actually could make through like eye tracking and sensors and yeah, then skin response and even posture we could make. Loops that were more addictive, which is what it should have been called. And at the time, being a naive engineer, I was thinking to myself like, these are some really interesting challenges I'm working on.

And I wasn't thinking about the ethical implications of it in the early days. And then as we started to develop it more and think and work on it more, I started learning about what say the casino industry does and how they need people. Blow all their money and I start feeling really uncomfortable with those types of addiction inducing game loops in particular.

For instance, the casino industry, their big innovation in maybe the eighties or late nineties is they went from one coin per pole on a slot machine to, you could put four or eight points, and there's a sound when you lose and there's a sound when you win. So if you put a dollar in and you lose 75 cents of it, but they still play you the wind sound, and our primitive brains are just like, I'm winning.

So now we fast forward to companies that like, I guess I'm pulling punches. Facebook that just is purely profit driven. Will do anything for profit. Now they have headsets that are taking all your biometrics. They're recording your world around you. They know you better, they know your friends better, and then they send it to a super computer, a super brain out in the cloud to figure out AI, to manipulate you to look at some ad or do something that's really scary.

I think about that a lot, and I really hate that there's a lot of money going into this kind of research.

Thanks for listening to today's podcast. Check the episode notes for links to the books, reports, articles, and other media we discuss today. You can find more episodes of spatial reality in your usual podcast spots. Leave us a review if you enjoy today's interview. And so, you know, I'm always looking for more experts to talk to, so hit me up on LinkedIn if there's anybody you'd love to hear.

See you next episode.