Subscribe
Share
Share
Embed
His work broadly focuses on creating new theories and algorithms that allow for the perceptually optimized streaming and sharing of visual media.
AM/FM RADIO/PODCAST & TV SHOW
With millions of listeners a month, Building the Future has quickly become one of the fastest rising nationally syndicated programs. With a focus on interviewing startups, entrepreneurs, investors, CEOs, and more, the show showcases individuals who are realizing their dreams and helping to make our world a better place through technology and innovation.
Kevin Horek: Welcome back to the show.
Today we have Alan Boic.
He's a professor.
Alan, welcome back to
Al Bovik: the show.
Glad to be back.
I had fun last time.
Kevin Horek: Yeah.
Me as well.
And I, I honestly felt like we ran out
of time and so I really wanted to do a
part two with you because I think the
stuff we're actually gonna cover today is.
Kind of a little bit more up and coming
a little bit, maybe more futuristic
and then a few things that have
kind of been around for a few years.
But maybe we, before we get into
that, do you maybe wanna just give
people a quick intro of yourself,
kind of your background, where you
grew up and, and maybe your teaching
career and being a professor?
Because I think.
That's where it will be a really
good segue into what we're talking
Al Bovik: about today.
Absolutely fine.
I, uh, grew up in Illinois,
uh, north side of Chicago.
Um, I went and got all my degrees
at the state school, the University
of Illinois in Champaign.
Um, a very fine school by the way.
And, um, I worked there
and I, I completed.
PhD dissertation with a brilliant man,
uh, named Thomas Wong, and I had a
second advisor named Dave Munson as well.
Thomas Wong was one of the inventors of
video compression, so it was a great honor
to work with him and to learn from him.
Then I took a job, you know, I looked
all over the place, uh, to interviewed.
You know, many places at the beginning
of the high tech boom and I had like
20 interviews for, you know, positions
in national labs and universities
and, uh, and I, well I wanted to
come to Austin because my advisor
said they have a great future, Alan.
And, uh, I took his word and he was right.
Ut Austin's been a great place to be.
Kevin Horek: Very cool.
So give us a bit of background on the
type of stuff that you teach and the
research you're doing, because some
very big brands, and probably a lot
of people listening actually use your
technology every day without even knowing
Al Bovik: it.
Well, no question.
Um, so, you know, I work in the field
called image processing and video
processing, but my particular interests,
um, are also in visual neuroscience.
So really I work in a cross-disciplinary
realm between the two, uh, where.
I have become a trained
visual neuroscientist.
I've learned, uh, quite a bit about
how we see, and my greatest joy
in research is finding theories
of how we see and bringing them
into algorithms that can be used.
In practical ways to affect things
like streaming video, social media,
pictures, uh, virtual reality
and, and other visual things.
Uh, today we're very fortunate that, you
know, a bunch of our algorithms actually
are used throughout the streaming and
social media industries, and it's safe
to say that about 80% of the bits that
are crossing the internet are processed
by algorithms that we developed.
What do they do?
Largely, they monitor.
Measure and control the visual quality
of what it is that you're seeing.
So if you're enjoying your, the, the
quality of experience that you're
seeing on whatever streamer you're
watching, uh, that is at least
in part owing to the work of our
laboratory, my wonderful students and I.
Very
Kevin Horek: cool.
So for people that are maybe doing
something, in the software tech space
that could leverage your technology?
How do they actually get in contact
with you and, and then how can they
leverage your technology and research
to actually implement it into their
Al Bovik: product?
Well, I mean, as far as the bigs, uh,
when I say the bigs, I mean the big
companies like, you know, Netflix and Meta
and Amazon Prime and YouTube and so on.
Well, they, we, uh, I've always had the
philosophy that, you know, the work we.
Uh, that means all of our, we do
conduct huge human studies of, you
know, quality of experience and so on.
Uh, we put all the data
out there for the world.
We don't hold it back or try to sell it.
When we create algorithms, we
publish them immediately and
we put the code on our website.
So we've always been super accessible
and we kind of pioneered the free data.
Idea.
Uh, interesting.
Before that, people were trying to
make money off it, even as academics,
uh, in algorithms they wouldn't share.
They'd say Tough luck.
You know, you figure it out.
So we've, uh, always trended against
that and because of that, however, uh,
people learned about what we do very
quickly and we were approached by all
the companies I mentioned, uh, over time.
Starting with Netflix, who was
one of our first, uh, you know,
partners who's funded our research
for the better part of a decade now.
Wow.
In creating algorithms that are
important for, you know, streaming
and social media workflows.
And they were joined by
several of the other companies.
I even worked with a company in India
called Share Chatt, who reached out to us.
They're sort of like the TikTok of India.
Cool.
So basically you pop me an email.
All right.
It's
Kevin Horek: just very, very
Al Bovik: difficult.
.
Kevin Horek: That's awesome.
So I wanna dive a little bit deeper into,
obviously like the video stuff makes a lot
of sense, but I, and I know like AR and VR
and kind of mixed reality and, you know,
the metaverse and all this stuff, people
hear about some of those things have been
around, um, for a number of years now.
But how do you see.
The future of those things and.
How does that play into your research
and working with some of these brands and
Al Bovik: companies?
Uh, sure.
Well, you know, not all companies
are working in the XR space openly.
Sure.
But most of them are in one way
or another, because everybody
realizes this is really hard to do.
Right.
But everybody also realizes
this is gonna be really big now.
Right now there are a few barriers.
The main barrier is.
One of the biggest barriers anyway
is there doesn't exist right now.
A true.
Augmented reality or AR device.
You can't go on Amazon right now
and buy augmented reality glasses.
Um, right.
So it's hard to, you know,
commercialize something doesn't exist.
Um, you can't of course buy VR helmets.
A whole bunch of, you know,
companies have those and they keep
advancing and that sort of thing.
Um, so that's one.
Now.
Uh, one thing though is that there
is an announcement by Apple that,
you know, by next year they'll be
announcing something and they're
calling it the Apple Reality Pro.
I hear, which is terrible name by the way.
They could probably come up with
something better than that, I think,
and I'm sure they, they sure will.
They probably surely will.
But there's some exciting things I, um,
Read into the, the rumor mill One is that
their device will have eye tracking in.
. Okay.
Okay.
So we have VR helmets where we've
had eye trackers built in and
we've conducted research on that.
So what is an eye tracker?
So it's a device which you wouldn't
even be aware of in whatever, say
glasses you're wearing and so on.
Or you might inspect and find it, but
it's, you know, very small form factor.
It's something that can accurately
track the direction of your visual gaze.
Okay, now that's, uh,
pretty amazing technology.
It's been around for 20 years and we've
researched it for more than 20 years,
and I have been waiting about that
long for it to finally come into, you
know, commercial, uh, scale products.
And I think we're just about to get there.
Now, why is that so important to
know where somebody's looking?
Well, there's a number of reasons.
One, you can, you know, in a
mixed reality, if you know where
somebody's looking, then you
can put an object of interest.
Right.
Okay.
So if you are, suppose that you have,
uh, augmented reality glasses and you,
uh, just took a flight to, uh, Paris.
You're looking around and your
eyes are, look, you know, looking
at this and that and so on.
Well, the glasses could.
Sort of do a help map the environment
thing and show you well what you know,
I'm looking at this building, what is
it UP pops the name of the building?
Notre Dame Cathedral.
Right, right.
Uh, because just my eye looking
there if I wanted to turn that on.
But there's a more important and
little bit subtler technical reason.
That would be much less obvious to
the user why it is so important,
and that is that if in your goggles
or your glasses or whatever you
are, having videos streamed to it.
Okay?
Video is the biggest data.
Okay.
It's more than 80% of all internet
traffic is video cuz it's such big data.
So if you have videos streaming into
your helmet, say from another user,
maybe you're talking to each other
in like a virtual, you know, phone
call, kinda like FaceTime, but you're
both 3D and looking at each other.
Mm-hmm.
. Sure.
You know, then you're sending video
to each other over say, wireless.
It's really hard to transmit that.
So that's why you need, you know, video.
, you squeeze down that video and
it only takes up one, 100th, maybe
of the space it originally did.
Um, and, but nevertheless, when it arrives
and they decompress it, it looks the same.
They hope.
Okay.
And often it does.
Right.
But it virtual reality
needs bigger videos.
So you've got 4K television, right?
Right.
Everybody does.
But for vr, we think
we need eight K or 10.
. Okay.
So suddenly the bandwidth goes up by,
you know, five to 10 times as much.
Uh, and it's just ridiculous.
Over wireless, you can't do it.
So that's why knowing
where you're looking helps.
Oh, interesting.
So you can already start to get it.
So one is, well, you only need to, you
know, show what you are looking at.
Well, it's not as simple as that
because we have a field of view
that goes, you know, 180, right?
In every.
. So, and you want all that,
you want the periphery.
You don't wanna be looking
at, like, you're looking
through a toilet paper tube.
Right?
Right.
So what you do is you, you discover
through eye tracking where the
person's looking and you compress.
So it's very high quality, right.
Where they're looking.
And then away from where they're
looking, the quality goes down is you can
press more and more and more and more.
Okay.
Right.
But because you're being, I tricked.
You won't know the difference.
So to exemplify this, suppose you're
reading a book, you know, I think
whoever's listening to this could
pick up a printed, you know, page.
And if you're reading that page,
only the letters that your gaze
is on are clear enough to read.
right?
The ones out in what we call
the periphery, you can't even
read it because it's too blurry.
That's because your retina out there
where the light strikes around the
peripheral, you know, outside regions
away from this center are much, you
know, the resolutions much smaller.
So we can exploit that with
compression and maybe boost
compression by five to 10 times.
So, wow.
We need five to 10 times as much
resolution, but maybe we can
compress five to TI 10 times as much.
That's kind of a nice match there.
Right.
Right.
Kevin Horek: and then I would assume
that you'd potentially know just
based on where I am and what I'm
looking at, where I'm gonna look next.
So you could load that in the background,
and then maybe the full scene in the
background after like a minute or two kind
Al Bovik: of thing.
Well, less than a minute, Kevin, I, I
must comment that you are very per indeed.
Uh, so, uh, predicting where a person
will look next is very much of.
But not so easy.
Right?
So imagine, you know, you've got
a , you know, you've got a picture,
and it's got two attractive people.
Some people will be attracted to one,
some will be attracted to the other.
Right?
Right.
And well, you know, that's why
deep learning's gonna have a
hard time in predicting where
people look and it won't be.
A fast way to replace eye trackers
because every person's different.
Okay.
The deep learner won't know, you know,
who puts on the headset and who they
wanna look at or what they wanna look at.
It's a hard problem, but
eye tracking solves that.
You know, it just actually
measures where you're looking.
So can you predict where you're gonna
look next based on like, Sort of the
momentum of your eyes to some degree, yes.
Okay.
Okay.
Um, there's another factor.
I've talked about eye tracking, but
there's also head movement measuring.
Okay?
So suppose you're wearing a augmented
reality, you're walking down the street,
you know your eyes are moving, but yeah,
your head may be moving significantly,
and that creates tremendous problems
because if you move your head quickly,
If you don't, as you just suggested,
have a bunch of that video ready to
go of the real world around you, then
there's gonna be a latency issue where
you said, uh oh, what am I looking at?
Just gray or something.
or an old, an old version
of the video or something.
So you wanna be up to speed, and that has.
Big implications for, you
know, memory, so much data.
Uh, how do you represent it?
Do you aviate that somehow?
And also communications link, um,
because you know, you have to send
more data if you're also saving the
data that you might look at next.
So that was a great question.
Kevin Horek: Okay.
But then, so, okay, so you
built, say you build this, I, I
want to use your eye tracking.
Um, algorithm and head
tracking and whatever other
algorithms you guys have built.
Obviously there's a
hardware component there.
How do those two fit and play together?
Because, That's gotta add a
tremendous amount of complexity on
top of all this stuff that you just
Al Bovik: outlined.
Not so much actually.
Well first of all, let's
look at just the iTracker.
Okay.
iTracker is a device.
Uh, it basically casts, say an ir,
you know, kinda like your remote
control for your television.
That's infrared radiation.
It bounces off, you know, various surface
on the interior of your eye, such as the,
um, the inner surface or the outer surface
of the cornea or the crystal and lens.
Uh, and it compares the
positions of the reflections.
Uh, those are called kinji reflections
and, um, that could be very accurate
first of all, but, Very fast.
You can do that easily in real time.
60 or 120 frames per second.
You could do it with, and it's not
very expensive and then, but the data
you collected is, is very low volume.
Okay?
The eye tracking data just tells
you what point you're looking at.
Okay, just the point.
It's just two coordinates.
You know, for those that had a little,
you know, al, you know, high school Math
x y, coordinate one of 'em, it tells
you where you're looking at that moment.
So the data volume is not much increase.
There isn't really much computation there.
So then when you take that coordinate
and use it to guide compression, Okay.
Yeah, that might be more along the
lines of what you're talking about.
So does it increase the
complexity of a Kodak much?
I don't think so.
I think we can go inside a standard,
uh, MPEG coder, say H E V C, um,
and we can modify it to foyate the
compression with very little extra over.
Kevin Horek: Interesting.
Okay, so then just because like I
just picture, well, obviously any
of the AR glasses I've seen so far,
they don't really look like glasses.
It's like a full thing, right?
Like it's clear they're more like ski
Al Bovik: goggles or something.
Like the holo lens, you know?
Yeah, exactly.
You can't really buy that anymore unless
you're like military budget, you know,
Kevin Horek: Yeah, exactly right.
And so like I, I guess maybe the better
way to ask the question is like, All this
technology that you guys are doing, could
you put in just like a regular pair of
like Ray Band sunglasses or something?
You know,
Al Bovik: that's a really great.
Question.
And by the way, um, I have, I
recently purchased the thing
called the Ray Band Stories, right?
Uh, just because it's the most advanced
AR device the consumers can buy.
It's not really ar but what it
can do is obviously it can, you
can take phone calls with it.
You can play music from your
iPhone or whatever phone you have.
Uh, you can take pictures with it
and it's got storage in it, and
you can take short videos with it.
I mean, the glasses,
the glass part itself.
Standard Ray Band sunglasses, right?
But, uh, I bring that up because,
you know, the form factor is
not substantially increased.
And anyway, for, I mean, we'd
eventually like ar glasses to be as
lightweight as, as any, you know,
sunglasses we might wear today.
Could we do it today?
Probably not.
But I sincerely believe that within
five years, uh, we'll be able
to put the capabilities of an.
Into a pair of glasses.
Right.
You know, it might be a little bit,
you know, heavier and so on, but I
mean, people will buy it and then
they'll, you know, it'll be popular.
Billions more will be built into it.
Miniaturization will
continue as it always has.
Has it ever, I mean, things
have never gotten bigger, right?
That's fair.
It's
Kevin Horek: always the thin Costas iPhone
Al Bovik: they've ever made.
Yes, that's right.
And so, um, you know, I am convinced,
you know, and I don't think I'm just
being my usual optimistic self that
within that timeframe we'll have, be
able to have very powerful goggles.
I think the bigger issues are, you know,
in terms of perception and deciding what
to do with AR and also the communication
issues, you know, having enough
bandwidth, um, over wireless links.
Right.
Kevin Horek: Well, and then
the privacy stuff too, right?
Al Bovik: Oh, that too.
Golly.
You know, I , I don't think anybody
knows when I'm wearing these sunglasses
that I could take their picture , right?
Yeah.
Or a video of them and so on.
And I have no plans unless a
cop pulls me over . Fair enough.
Um, but yeah.
Yeah, I think that's always
been an issue, of course, but I
mean, um, you know, a long time.
Uh, I remember in the 1960s when
I was a kid, my dad took me to the
Prudential building in Chicago,
which was then the tallest building.
Okay.
And, you know, uh, he took me to see the,
the exhibition for the picture phone.
And, you know, it was huge lines for this.
And everybody's amazed that you could
talk over the phone with pictures.
And I thought it was like
next year in the 1960s.
Well, it didn't ever succeed even though
there was many products along the way
that tried, didn't succeed until face.
Right, right.
And that was the teenagers
who did it, right?
Yeah.
And that's because nobody, it wasn't
because it wasn't technologically
feasible, it's cuz nobody,
you know, wanted to answer the
phone in their jammies or hadn't
shaved or put on their makeup.
And it wasn't appealing to
people who could afford it.
Well then the kids could
do it and they didn't care.
You know, they just, you know, whatever.
That's the best.
So what am I getting at here?
Is that I think that these things will
be overcome and if everybody's got it
and everybody can take a picture of
anybody, you know, um, well, so what,
Kevin Horek: Yeah, I guess it's not really
any different than the phone, right?
Like everybody's.
Go anywhere.
Everybody's got their phone out.
They, they could be potentially taking
your picture, but nobody assumes that.
Everybody's just constantly
taking your picture.
Right.
Al Bovik: It's a little
bit more surreptitious.
However, when your glasses true
and you can't tell, you know, and
I mean, all you do, you put your
hand up to your glass to adjust it.
Well, you're really taking
the picture , right?
Yeah.
That's fair.
You know, so I mean, there's might
be issues and I mean, I remember, uh,
there's a fellow named Steve Mann.
Okay.
Steve Mann.
Okay.
Very famous for, what he
did was he wore a, you know,
goggles, uh, for years . Okay.
You know, , he lived sort of a, kind
of like a, a vr ar experience, you
know, looking through the goggles,
but it wasn't really augmented.
Um, and he weren't around and he, I
mean, he'd get up in the morning, put
'em on and walk around with these goggles
and, you know, can you imagine how.
Did things to his brain, I don't know.
But I mean, he would, I met, there
was a famous incident at McDonald's
where somebody became very angry at
him because he was clearly videoing
him because that's all he saw was
what came through the camera, right?
And was presented to his eyes.
And, you know, people
can get angry and so on.
So it will be a societal issue.
Um, You know, in regards,
but we'll get past it.
Kevin Horek: Yeah, that's fair.
Cuz we had Google Glass a number of
years ago when it came out and just
like wearing it around, you could
tell people were a little like, what?
What's happening?
So yeah, it was interesting.
But it was a first try.
Yeah.
. Yeah.
That's fair.
So I'm curious, what's your
thoughts on virtual reality?
Because it seemed to be the next big
thing, you know, three to five years ago.
It kind of petered out.
It seems to be a bit back, like,
where are we at and where do
you think the future of that is?
Al Bovik: Oh, good question again.
So, uh, I think a big aspect
of the big problem with virtual
reality is a perceptual one.
Okay.
And that is that most
people who put it on.
And play a game or whatever.
They start to feel tired or head achy
or eye strain or just want to get out of
there and that kind of thing because some,
some kind of, you know, physical or visual
discomfort, uh, that's the biggest thing.
And, uh, we understand why
that happens to a large extent.
There's different sources of the
discomfort, but the biggest source
comes about because of what is called
the accommodation, Virgin conflict.
. Okay.
Okay.
That's a vision science vi
visual neuroscience term.
But let me explain.
Uh, anytime you are looking around the
world with your eyeballs, your eyeballs
are constantly focusing on what you're
looking at at your point of gaze.
Okay?
So what you have is in your eye, you have
a lens, it's called the crystalin lens,
and it's constantly deforming in shape.
in response to trying to focus on things.
There are muscles that do that.
They're called the Cary muscles.
They control the shape
of the crystal and lens.
All right?
Now the interesting thing is not only
is the shape of the lens controlled,
so you focus, but once you focused, it
tells you something about how far away
that object is, and that is sent to.
Certain area of the brain that focuses
on 3D perception, it's called Area mt.
Or Area V five has different names, okay?
And it's used to help gauge how far
things are away, how much, just think
of that, how much your crystalin lens
focuses is used by your brain to help
figure out how far away objects are.
Now another thing happens.
And that is the two eyes.
Okay.
When you're looking in the
three-dimensional world at objects,
try putting in, know your finger in
front of your face and put it closer
and farther while focusing on it.
Your eyes start to cross
well as it gets closer.
That is called virgins.
Okay?
Your eyes naturally verge as you focus on
objects in our three dimensional world.
Now that's controlled.
Other muscles called rector muscles,
they actually, you know, control,
you know, the direction of the
gaze of the eyeballs, especially
when they verge in this context.
So when they verge, that means that the
images from the two eyes fuse into one.
Image.
And so you see one image instead of two.
You don't see double
in your natural vision.
Uh, that's called by way a cyclo image.
Okay?
You can think of cyclops and the
two eye images sort of create one.
But anyway, the amount that the two
eyes verge creates another signal,
a neural signal that goes to the
back of the brain in the same area.
M.
Okay.
Okay.
Now why is this important?
Two signals, one from the lens,
one from the ver to the eyes.
Well, normally it's great they work
together, um, because the crystal,
the, the signal for focusing helps
you decide how much to verge.
In 3d and the signal from verging
helps you to decide how much to
focus, change the shape of the lens.
Right.
Okay.
So it's a, what we call an
engineering, a feedback control system.
There's a lot of feedback back and
forth between those two different
and separate modalities of seeing.
Now why is this super
important and virtual reality?
Because when you've got a helmet on, that
little screen is one inch away from your.
Yep.
And no matter what, you
need to focus one inch away.
Okay?
You can't change it.
You're focusing on that high
resolution display one inch away.
And that doesn't change.
And so the signal sent to you back of
your brain is, this is one inch away.
. Okay.
Wow.
Uh, however, when you're seeing 3D
content, yo, you're looking way a hundred
yards away or right in front of you.
All these different distances
and depths, and those signals
are sent, so you have a conflict.
It's called the accommodation.
Virgin conflict, by the way.
Accommodation is that focusing
of the lens, focusing lens
is called accommodation.
So it's a conflict.
It sets up an instability with
conflicting signals going to your
crystalin lens, into your virgins,
and this can actually oscillate.
I mean, you don't see your eyeballs
shaking or something like that, but
it's an oscillatory mechanism and it
gets unstable and it creates all sorts
of, you know, physical discomfort
in your ocular motor control.
You know, the muscles get tired,
basically trying to figure out what to do.
Kevin Horek: Okay.
So, so does that what can make people like
Al Bovik: nauseous?
Yes, nauseous.
I mean, there's other things like,
you know, too much motion, uh, you
know, vertigo, that kind of stuff.
There's, it's, this is not, you know,
these are all co-related and they're
all issues, but, you know, you can
control the motion by having less motion.
Okay.
Uh, but this doesn't go away.
It doesn't go away.
You know, the only thing you
can do is to flatten out and
make it less three-dimensional.
Okay.
Kevin Horek: Yeah.
Cuz the first version of the Oculus,
I remember getting it and putting it
on and I remember like trying out the
little like rollercoaster simulation
and some of the like, faster moving
things and I was completely fine, but
I, I got nauseous just walking around.
a simple 3D world, and I was like it.
It didn't really make sense to me.
I was like, I'm just virtually
walking in this 3D world.
But your point a second ago,
it totally makes sense Now.
Why?
That was the case.
Al Bovik: It could have been
partly the conflict, because you're
seeing all these different depths.
Yeah.
And your brain is not able
to process it properly.
You know, it could also be just,
you know, proprioceptive issues.
You know, I mean, here you are moving
legs, you can't see and uh Right.
You know, and it's kind of,
it's kind of a, you know, you
shouldn't really use VR outdoors.
Right.
Because you might step in a hole.
I mean, if you don't run into a car,
.
Kevin Horek: Right, yeah,
Al Bovik: yeah.
Fair.
You know, but also for perceptual reasons.
.
Kevin Horek: Okay.
Interesting.
So, Where do you think
the industry is going?
Like, do you think VR has a future?
Is it gonna die off again?
Is it just not ready yet?
Where are we at?
Well,
Al Bovik: I think VR does have, I mean, VR
definitely has a future in gaming because
it already exists and gaming is different
because you can control the content.
It's all computer generated, right?
Right.
It's all computer generated.
So you can control the content to
try to ameliorate things like the
uh, you know, the AV conflict.
Right.
Okay.
So there's different ways of controlling
it so that they're, you know, maybe
they flatten out the background,
you know, and just the object you're
looking at has depths, things like that.
So, I mean, if there's no
depth perception, then it
kind of goes away, no issue.
Um, your eyes kind of look straight ahead
and the virgin doesn't really create
this, you know, conflicting signal.
Um, but, um, I think that technology will
help VR to allow us to watch movies in vr.
Eventually, cinema,
television and all that.
But the displays will have to change.
There's a type of display, um,
you know, that has lenses in it
called, you know, a very focal lens.
Okay.
Uh, presented to your eyes.
So really there's lenses as part
of the display, and so it can
change the amount of focusing.
Uh, and that could be responsive
to what it is you're looking at.
So that could, you know, help with
the AV conflict, uh, so that the
crystalline lens can be more aligned
with the virgins to put it that way.
Um, however, that's.
Cumbersome and expensive
right now, . Right?
So it's hard.
The only other answer really right
now, at least when the immediate
timeframes of five years say, would
be to movie making, would have to
be specially designed for this.
So you can control the
content even of real cinema.
Uh, a, a friend of mine
named Marty Banks at.
Was consulted on the making of Avatar and
he basically did that in the making of
that movie cuz that was the big 3D hit.
Right?
Right.
And people generally didn't get sick
as much watching that as they did a
lot of other stuff that came after.
Um, and that's because he was
there, uh, helping them arrange the
cameras to accurately reflect our
own, you know, visual apparatus,
the way the cameras verge.
Two huge, you know, cinematic cameras.
Would be pointing it in a certain
direction and also help them to
flatten out the depths in it.
So if you're watching Avatar and you're
in the forest scene, you know the blue
beings or you know the animals, you might
notice that the background is pretty
flat depth free, but it didn't matter
cuz you're looking at the blue people.
But it did help with
avoiding AV conflicts.
. So that's a, that's another way,
but I gotta tell you, you know,
uh, these things that's, you know,
that means you have to inter, you
know, you can't watch old stuff.
It's all gonna be new stuff specially
made, and maybe that'll happen, but I
think industry is very interested in ar.
Okay.
Okay.
Yeah,
Kevin Horek: yeah, I, I would agree.
It seems that there's
way more applications.
And almost just for the everyday person
that maybe doesn't game right, you could
actually use AR probably throughout your
day, just doing many things, you know.
Al Bovik: Could be.
Definitely, you know, I'm, I'm not sure
I'm a proponent of people wearing an
augmented reality all day, unless they
can like switch it off and then, you know,
actually look through just glass or Right.
Flip it up.
I don't know.
Um, all these are good
questions going forward.
Um, you know, I, I think that the
idea of somebody wearing a device
like that all day where they don't
need to, you know, somebody worrisome
to me because you know it's gotta be
done just right cuz lemme explain.
I'll give an analogy.
One thing that's worried
me about VR is kids, right?
Okay.
Kids sitting there playing games
all for six hours in a row, okay?
And their, you know, their
vision systems there.
There used to be a thought that the
vision system was locked in after like
six months of life, but it's not true.
It's just as plastic as the rest of our.
Okay.
Okay.
It really is, um, I was talking
to Miguel Nicolas, one of the
world's most famous neuroscientist.
In fact, he's a brain
interface pioneer about this.
And, you know, he affirmed this in
his worry as well in this direction.
And, um, you know, if you have like
a five-year-old watching the wrong
stuff where the 3D depths are wrong
and the shapes are, you know, the
distances are wrong or so on, they're
basically learning an alternate
reality and their brain adapts to.
Oh, interesting.
Uh, potentially if they do it too much,
and I worry about that because even today,
kids who are looking at flat screens sure.
You know, I mean a lot of them are have,
are maybe losing their ability to, you
know, really be active in the environment,
be proprioceptive in 3d, and be athletic
and explore nature and that sort of thing.
It, it worries me and
it worries me much more.
If, you know somebody's wearing
a device for an extended period
all day long, that right.
You know, I think we have to really
watch and it's not just babies.
It's, you know, kids and who knows,
maybe even couch potato adults.
.
Kevin Horek: Yeah.
No, fair.
You're right.
Well, if I wear glasses at alternate or
my, gives my brain different perception
of depth and stuff because of ar.
Yeah.
Interesting.
That's actually, I never really,
that never even crossed my mind
Al Bovik: or dawned on you may, you
may adapt to, you know, the device
and then you get out in the world,
you're not seeing the way you used to.
A little bit off.
I don't like that too much, you know?
But I think responsible companies,
uh, you know, like meta, you know who,
they have very powerful d division
in the meta reality where they have
many, many visual neuroscientists and
visual psychologists are, they're c.
These things, certainly.
And, uh, that's good.
Kevin Horek: Interesting.
So I, I'm curious then, do you have
any other predictions or examples
to give people of potentially how to
leverage the research that you and your
team have, have done and are doing?
Al Bovik: Well, you know, with each
new advancement in imaging and video
and applications of it, there's
new modalities and new aspects.
So to give you an example,
it's not a huge leap forward.
Uh, I think that, uh, High motion is
becoming very important cuz people are
getting really interested in live sports.
Right?
Right.
So, I mean, Amazon, YouTube, I have, you
know, a lot of efforts in this direction
and money they're pouring into it.
And, uh, you know, live sports on
big screen means, um, you really
need higher frame rates, right.
So, you know, I mean, I grew up, most
of my life, everything was 30 frames
per second, and then suddenly, here
we are, 60 frames is pretty standard.
Who saw that?
Just because they could do it from
the increased bandwidth, but for live
sports you need higher frame rates.
You know, 90, maybe one
20 to really get it right.
And by the way, especially if you're
gonna have high motion, In virtual
reality, you'll need higher, you
know, high end high frame rates
cuz the eyeball's right there.
If you don't have enough high frame
rates, you get these sort of really
annoying, uh, distortions that are
temporal or, you know, happen over time
where, you know, An object that's moving.
It looks like it's sort of stuttering
along, and that's why it's called stutter.
When that happens, uh, you know, you can
see it's kind of a discreet, you know,
and people kind of overlook it now, right?
Uh, even though you see it a lot
in, you know, live action, right?
Um, but people are getting more
and more discerning all the time.
They expect more and more and as their TVs
get bigger, that's like eight k coming.
I can tell you that's another topic.
Bigger TVs.
Um, then, you know, it's gonna be
more and more of the experience in
the, you know, we talk about immersion
in the helmet or the AR device.
We're gonna have immersion in
your house, in your living room.
As well.
And you'll be more attentive to
what it is that you're seeing.
So, high frame rate's, another
direction, uh, that's important.
And so, you know, if you're in that
space, uh, if you are N F L, uh,
or you know, basketball, if you're
the NBA and so on, you should be
looking at this kind of technology.
So I'm speaking to those
kind of people right now.
Kevin Horek: Got it.
Okay.
So I, I think the challenge that
I've always had with like eight
K or whatever, even 4k, arguably
there's just not enough content and
obviously like it has to be shot at.
You know, those resolutions are
higher, but how does that, Okay.
Like I get that TVs are getting bigger.
People want bigger tv,
especially for sports.
N B A and big organizations like that
can afford to shoot their stuff in eight
k as long as people have the technology.
But, but how do, I guess, like
regular companies kind of.
Do that?
Like is it just they
need to invest in that?
Is it affordable or, or like how does
maybe like a startup or, or a company
that's not like a Netflix or the N B a
leverage some of that technology, can
they, or is it still kind of years out?
Al Bovik: Well, if we're talking about,
you know, higher resolution Yeah.
You know, it's gotta be the big guys.
Okay.
Okay.
Who are really rolling out that means,
you know, uh, not just, you know, Silicon
Valley, but the big television makers.
Okay.
Right.
Yeah.
Uh, the display makers
and that sort of thing.
I mean, you know, eight K is here.
I got.
Uh, an email from Amazon Prime, you know,
prime, you know, Amazon Prime shopping.
Uh, it was an advertisement for
eight K televisions, you know,
which kind of surprised me.
But, you know, Samsung is offering
them now for about $4,000.
Yeah.
And they'll be good quality.
Sure.
So, you know, I remember what before
4K we're watching what we call
HD back then 10 80 p uh, 4K is,
you know, twice the size in both
directions, and we all kind of were.
When I say we, I mean us, the
population in the United States.
Oh yeah.
What do we need 4K for?
First of all, there's no content
and their televisions are, you know,
$4,000 . Well, look, now, you know,
you both go buy your 4K television
for 200 now, or go to your target.
That's all they sell, and
there's plenty of 4K content now.
It's streaming all the time, and.
Even in real time.
So, you know, that is a done deal.
In fact, the compression standard
H E V C is tailormade for 4K too.
Okay.
And it gives, you know,
increased compressions on
that makes it all possible.
It was very high quality.
So eight K.
Oh goodness.
Yes.
I mean, recently they were
down, they're getting ready.
Or maybe they did it already.
They're imaging the Titanic in eight K.
So you can buy cameras.
Those cameras are expensive.
Yes.
I mean, 4K cameras now, you know,
oh 5k, you know, buy it cinematic.
But eight K is much more, the
televisions are much more expensive,
but hey, all you need to do, all you
need is a larger population buying it.
You do need more content.
Very limited.
But remember, I remember when
I couldn't get my hands on 4K
content to do science research on.
Right.
Okay.
No, it's hard to get any
from any of these companies.
They're all very protective and so on.
Well, that's not, that's totally changed
now and it will with eight K as well.
And even, yes, dare I say 16 k,
you know, last time did I, did I
tell you about Fahrenheit four 50?
No, uh, no . Okay.
Let me tell you about that.
I mean, Fahrenheit 4 51 is a, it's a
book by Ray Bradbury, a great visionary.
Okay.
And, uh, he, I mean, it was
famously about book burning
and a dystopian society, right?
Okay.
Um, and you know, that's what, why
they give it to school kids in high
school to read, you know, Fahrenheit
4 51, you know, well, let's avoid
this, you know, totalitarian society.
But when I, I read.
Too many years ago, maybe 10 years ago.
And what really struck me
was another visionary aspect.
So he describes, you know, people
in these horrible jobs and they
come trudging home, you know, with
no rewards in life to their house.
They walk in the front door and
all four walls of every room in
the house are covered with a.
Display screen.
Okay.
Okay.
And what's happening?
It's the Jerry Springer show.
of everybody in the world can you know,
because there's cameras hiding behind it.
And you walk in and people start.
Yelling at you.
You know, why don't you
do a better job at work?
You know, you should treat your
wife better, you know, whatever.
And it's just, you can't escape
it because all four walls, there's
somebody there who's criticizing
you, you know, or whatever.
You know?
Imagine the, the social media
gone to the absolute extreme.
Now, of course, that
was a dystopian novel.
I don't think that's gonna happen.
I.
Larger screens will be used in
beneficial ways, not only for
entertainment, but for many other things
in science and that sort of thing.
But my point is, you know, Ray
Bradbury saw it around 1960
maybe when that was written.
Right?
1950 And yeah, he was right.
It's coming.
We're gonna have wall size screens.
Interesting
Kevin Horek: one, and I don't know if
you have the answer to this, but I, it's
always kind of fascinating to me how come
TVs and computer monitors aren't the same.
Like there is, there's just one you go.
Like the technology is different, but
you know, we, they share so many things.
So similar or am I totally wrong
Al Bovik: there?
Well, you're not wrong because you can go
up and buy an expensive computer monitor.
Yeah.
Or you can buy a cheap tv.
Yeah.
Okay.
Now on my desk, my computer monitor Yeah.
That I use is a big 4K televi televis.
Okay.
Okay.
It's a, you know, huge monitor.
I, I have a standing desk and my
monitor is, you know, a 50 inch screen.
I love it.
You know, I can walk around
the room and still see and read
everything, walk back over.
It's a television.
I mean, what is a television?
It's a monitor with a receiver.
Okay.
And the receiver has, you know, video
code accidents on all that kinda
stuff, but it made the antenna and so
on, so they shouldn't be different.
And displays shouldn't be more expensive
unless there's special purpose for
like science studies or something.
So I'm very happy with my monitor,
which happens to be a television.
And so, you know, hey, I have no
problem connecting with my computer.
I just.
Plug in the H D M I , so it's easy.
So, you know.
Yeah.
You can talk about a, a monitor having
potentially higher resolution Yes.
Or higher frame rates and so
on if you're doing graphical
research or something like that.
Yeah.
Biographical display for sure.
But if you're, you know, You know,
Joe Schmo at home, and I don't mean
to pick on Joe, he's a wise guy.
Buy a television, spend $200 on
it, put it on your desk, and it's a
fantastic monitor for anything you're
likely to do, uh, with your computer.
And you can watch TV on it.
, a hundred
Kevin Horek: percent, no.
Okay.
No, I've done similar stuff to that
before and depending on where I'm
working in the house, sometimes I
like, you know, just air airplay.
TV and use that as a monitor.
It's great.
Like I love it.
So yeah, I was always kind of, I was
wondering if there was like really a big
technical difference, I guess, or from
Al Bovik: there can be.
Yeah, there can be if you need, I
guess, special capabilities, but Sure.
Look at me.
Here I am.
Mr.
Video.
Yeah.
I use a television as my monitor.
Kevin Horek: Interesting.
So, uh, we're kind of coming
to the end, but is there any
advice that you would give to.
Entrepreneurs or startups or even
people that work at big companies
that maybe want to get into this
space or are in this space because
this stuff's coming, it's here and I.
obviously, to your point, like
really all the big companies are
using your technology, right?
Al Bovik: Yeah, they are.
And so let me, I, I think you've
asked that a couple times.
I haven't really answered
it quite right yet.
So let's think about, you know, a
small enterprise that's trying to
enter the space, broadly speaking.
Okay.
Okay.
So, I think number one is there's no
denying it that, um, with almost anything
we're doing, and the big companies
realize this, the streamers, social
media, everybody is that, you know,
you can't really get away from ai.
And I'm using that term very broadly,
you know, so you, you've gotta, you've
gotta have a strong component on your
team of, you know, AI type, um, you know,
researchers and developers, and thinkers.
Thinkers.
Now the thing is, you know, ai, uh,
is my friend Nickolas, uh, says AI
is just curve fitting on a massive
scale with millions of parameters.
And he is right.
So you need to be able to enter a.
Uh, if you're trying to do something
different with, say, television or
with social media, you need to be
able to access lots and lots of.
If you're not gonna, if you're
gonna try to do something without
being able to gather a lot of data,
whether it's crawling the internet
or whatever, you know, or working
with a, a content partner who's
willing to provide data or something,
you're not gonna succeed because
you're not gonna be able to compete.
So that's, uh, I mean, This is kind of
entrepreneurial advice that has, you know,
nothing to do with like the business side
or being, you know, proactive and never
giving up and all that kind of stuff.
This is a practicality that I think
would have to be incorporated anybody's
business plan, right, uh, of a, of
a company entering the visual space.
Okay?
That's number one.
Number two, don't forget, you're, you're
offering things that are for humans
to watch, so you can't forget the.
That means a lot of things.
Not only the content that's
attractive or repealing or a fun
game or a fun show or whatever, but
also from the neuroscience level.
Okay?
You need to design whatever it
is you're doing, if appropriate.
Some things that won't be appropriate
for that human receiver, right?
After all, our brain, half of
our brain is implicated in.
Okay.
Really is, and it's all
calculation at the neural level.
So that's the receiver.
You should account for it if appropriate.
Not everything needs that, of course.
Kevin Horek: Interesting.
No, I, I think that's really good advice.
And that was kind of one I think.
The, my earliest criticisms of, of
some of this stuff is like, just, just
because you have access to VR doesn't
mean it should be in VR or, or whatever.
Right?
And just because you, you
build it in VR doesn't mean.
, it's good.
And it doesn't mean that you're gonna
get a ton of people using it, right?
Mm-hmm.
, it's like just to use VR just for
the sake of using VR sometimes
makes sense, sometimes doesn't make
Al Bovik: sense.
Absolutely.
Yeah.
I, I think it'll have so many
beneficial applications as it
keeps developing entertainment as
well as, you know, like medical.
Just imagine what you can do, you
know, uh, that surgeon is, you know,
we already have microsurgery where they
were these things and so on, but what if
you know, you are inside the patient's
body with a little robot tool and you
are seeing it very high resolution in
three dimensions cuz of the VR device.
I mean, wonderful.
Right?
Kevin Horek: Yeah, that, that's very cool.
But we're, we're kind of outta time,
so how about we close the show with
mentioning where people can actually
maybe go see some of these algorithms,
start playing with some of this stuff,
and, and get in contact with you and,
Al Bovik: uh, your researchers.
Well, you know, if you Google my last
name b o v i k, you will land on my
webpage very quickly cuz there's only
about five of us bovis in the world.
And, uh, you land on my website.
Then you'll immediately find my
laboratory where we have a wonderful
webpage that lays out, uh, there's
a section on algorithms, there's
a section on human study data,
uh, there's a section on papers.
And, uh, theory and all that.
And my email.
Happy to talk to anybody who wants to talk
about these wonderful, fascinating topics,
you know, and, um, that's the easiest way.
Kevin Horek: Perfect, Al well, I really
appreciate you again, taking the time
outta your day to be on the show, and
I look forward to keeping in touch with
you and have a good rest of your day,
Al Bovik: man.
Thank you so much, Kevin, as usual.
It was a lot of fun.
Thanks
Kevin Horek: very much, Kate.
Bye.
All right, bye-bye.