WEBVTT

NOTE
This file was generated by Descript 

00:00:00.000 --> 00:00:03.120
Jacob Haimes: Welcome to muckrAIkers,
where we dig through the latest

00:00:03.120 --> 00:00:05.220
happenings around so-called ai.

00:00:05.400 --> 00:00:08.250
In each episode, we highlight
recent events, contextualize

00:00:08.250 --> 00:00:11.070
the most important ones, and try
to separate muck from meaning.

00:00:11.370 --> 00:00:14.680
I'm your host, Jacob Haimes, and
joining me is my co-host Igor Krawczuk.

00:00:15.930 --> 00:00:16.619
Igor: Thanks, Jacob.

00:00:16.680 --> 00:00:19.890
In this episode we'll talk about
why AI safety is making you,

00:00:19.920 --> 00:00:21.479
uh, you actively less safe.

00:00:22.290 --> 00:00:23.939
But first we have to highlight.

00:00:24.375 --> 00:00:31.515
Just how insane the money flows around
AI investment have become, and for

00:00:31.515 --> 00:00:35.445
which we will point to a helpfully
circular diagram that a blue sky

00:00:35.445 --> 00:00:37.125
user has made in the show notes.

00:00:37.605 --> 00:00:40.185
And we would also like to point out
that open air has crossed the meme

00:00:40.185 --> 00:00:45.795
threshold and the, just a few more billion
bro, is just how we're rolling now.

00:00:46.365 --> 00:00:49.350
And they've, they're projecting
that they'll be, uh, using 80

00:00:49.350 --> 00:00:54.135
billion more than originally
projected, uh, until 20 to 30 or 35.

00:00:54.645 --> 00:00:59.865
So comedy is dead, and we'll see
how long AI can, uh, outlive it.

00:01:00.975 --> 00:01:01.425
Jacob Haimes: Yeah.

00:01:02.625 --> 00:01:03.195
So

00:01:05.835 --> 00:01:10.065
I just, I can't get over the, like,
how I, I saw a meme that was not

00:01:10.065 --> 00:01:12.975
actually the one that you sent, but
was a different one that was like

00:01:13.035 --> 00:01:14.774
simplified and it was just, you know.

00:01:14.940 --> 00:01:18.840
Uh, OpenAI invests a hundred
billion in Oracle, invests a

00:01:18.840 --> 00:01:22.800
hundred billion in Nvidia, invests
a hundred billion in OpenAI.

00:01:23.010 --> 00:01:28.200
And like, it's not, it's not actually that
inaccurate and it's just wild, wild to me.

00:01:28.380 --> 00:01:30.690
Um, but anyways,

00:01:30.780 --> 00:01:35.220
Igor: we talk about d different things,
which, uh, mildly less infuriating.

00:01:36.119 --> 00:01:36.660
Jacob Haimes: Yes.

00:01:36.720 --> 00:01:42.899
So last episode, we, uh, tried to say
in so many words, uh, like currently.

00:01:43.470 --> 00:01:47.580
The prominent versions of AI safety
are just branding and or safety

00:01:47.580 --> 00:01:53.700
washing, uh, for better AI systems
and more fine-grained control

00:01:53.760 --> 00:01:56.130
by the owners of said systems.

00:01:56.369 --> 00:02:00.210
So AI safety is being used to sort of

00:02:02.970 --> 00:02:09.960
do you know, better AI r and d and
have better control over the systems.

00:02:10.320 --> 00:02:10.830
Um.

00:02:11.445 --> 00:02:15.225
So that the owners can do whatever
they want with them, and it's

00:02:15.615 --> 00:02:21.255
masquerading as, you know, a positive
benefit for society and, and the field

00:02:22.995 --> 00:02:25.125
Igor: and tax de deductible.

00:02:25.755 --> 00:02:26.295
Jacob Haimes: That's true.

00:02:26.355 --> 00:02:26.835
Yes.

00:02:27.085 --> 00:02:30.415
Igor: In this episode we want to
highlight further how it's actively

00:02:30.415 --> 00:02:32.095
fucking you over, basically.

00:02:32.100 --> 00:02:32.320
Mm mm-hmm.

00:02:34.045 --> 00:02:37.795
And by which, uh, for this we
found like a couple of examples.

00:02:38.365 --> 00:02:45.205
Probably the strongest one is that
it, I am happy to kind of claim that

00:02:45.595 --> 00:02:52.455
without AI safety, we wouldn't have
the current wave of AI sycophancy.

00:02:52.510 --> 00:02:53.410
Jacob Haimes: Mm-hmm.

00:02:53.740 --> 00:02:58.690
Igor: And ai, mental health tragedies
that are, that are associated with it.

00:02:59.440 --> 00:02:59.860
Jacob Haimes: So.

00:03:01.570 --> 00:03:04.690
I do get like, sort of where you're coming
from and I think we'll get into this in

00:03:04.690 --> 00:03:08.530
just a little bit 'cause they, they do tie
together, but I would say it's more about

00:03:09.400 --> 00:03:17.800
the anthropomorphization of AI systems,
um, and the extent to which big tech has

00:03:17.800 --> 00:03:20.020
pursued that and taken advantage of that.

00:03:20.590 --> 00:03:24.790
Cognitive scientists have, like, it's
very well established within this space

00:03:24.790 --> 00:03:30.250
that, you know, subtle cues, uh, in
social terms and the way, things behave, I

00:03:30.250 --> 00:03:36.070
guess in the real world, um, causes us to

00:03:38.560 --> 00:03:39.190
empathize.

00:03:39.685 --> 00:03:40.405
With the thing.

00:03:41.215 --> 00:03:46.255
Um, and that's like an evolutionarily
gained trait, and this is being taken

00:03:46.255 --> 00:03:50.755
advantage of because these systems
that are being created are being

00:03:53.455 --> 00:03:57.940
pushed towards personalities,
um, in, in, in the process of,

00:03:57.940 --> 00:03:59.125
of how they're being created.

00:03:59.635 --> 00:04:06.655
And even just the interface that's
being used here, um, is inherently.

00:04:07.900 --> 00:04:12.880
Anthropomorphizing the system because it
mimics how we talk to other humans, right?

00:04:12.880 --> 00:04:16.810
Like it, it mimics like you are texting
on a phone or sending an instant message.

00:04:17.230 --> 00:04:21.190
Uh, and that is previously only
some way we talked to humans.

00:04:21.190 --> 00:04:28.480
So it's projecting
personhood onto this tool.

00:04:29.050 --> 00:04:33.215
Um, and in doing that.

00:04:35.335 --> 00:04:40.825
They, like big tech is, is gaining
unearned trust and building asymmetric

00:04:40.825 --> 00:04:46.465
relationships with these tools
that are controlled and developed

00:04:46.495 --> 00:04:49.525
by profit maximizing groups.

00:04:49.525 --> 00:04:56.425
These, the, these big tech companies,
which like isn't a fault of the, I

00:04:56.425 --> 00:04:58.825
guess, I mean it is a fault of the
company, but it's more of a fault of

00:04:58.825 --> 00:05:00.955
the system that it's in because like
that's what they're supposed to do.

00:05:01.255 --> 00:05:01.645
So.

00:05:03.920 --> 00:05:06.550
Not acknowledging that and
saying, oh, wait a second.

00:05:06.550 --> 00:05:08.230
We shouldn't be allowing this to happen.

00:05:08.230 --> 00:05:13.090
We shouldn't be allowing them to take
advantage of these sort of evolutionarily

00:05:13.090 --> 00:05:21.370
gained traits to make these systems ANR
anthropomorphic, um, is really dangerous.

00:05:21.460 --> 00:05:25.810
And so it's, yeah, I just
want to like, I guess set that

00:05:25.810 --> 00:05:29.260
out there as a, a first take.

00:05:29.440 --> 00:05:30.460
Um, and maybe.

00:05:32.110 --> 00:05:33.340
I mean, what, what are
your thoughts on that?

00:05:35.350 --> 00:05:37.120
Igor: Uh, I think it's worth rolling up.

00:05:37.180 --> 00:05:44.620
Um, for people who are not familiar
where this started, like where I

00:05:44.620 --> 00:05:50.650
think like, uh, very briefly, Eliza
is probably the earliest example of

00:05:50.650 --> 00:05:52.870
like how easy it is to get a human to.

00:05:53.275 --> 00:05:56.065
Like really like ascribe like
a personality and a personhood

00:05:56.065 --> 00:05:58.435
and actually kind of like
empathize with, with machines.

00:05:58.435 --> 00:05:59.815
And Eliza Wa was this, uh,

00:05:59.820 --> 00:05:59.960
Jacob Haimes: oh,

00:06:00.190 --> 00:06:00.640
Igor: very simple.

00:06:00.640 --> 00:06:02.305
Jacob Haimes: I would
say it's like a pet rock.

00:06:04.765 --> 00:06:05.305
Igor: Sure.

00:06:05.305 --> 00:06:07.825
But kind of like scientifically
studied and, and kind of

00:06:07.825 --> 00:06:10.015
like a clear demonstration.

00:06:11.485 --> 00:06:12.025
Like I agree with you.

00:06:12.025 --> 00:06:12.400
Like Sure.

00:06:12.400 --> 00:06:17.125
Like I, I lived with people, uh, I,
like we did a funeral for a car that

00:06:17.125 --> 00:06:20.425
died because, you know, like humans
will en anthropomorphize everything.

00:06:20.725 --> 00:06:21.600
The car's name was Fred.

00:06:22.360 --> 00:06:23.440
Uh, it was very sad.

00:06:24.070 --> 00:06:24.250
Um,

00:06:24.250 --> 00:06:25.510
Jacob Haimes: my car's name is Mystique.

00:06:25.660 --> 00:06:25.930
So

00:06:27.700 --> 00:06:29.740
Igor: is it a cha uh,
shapeshifter as well?

00:06:29.740 --> 00:06:32.140
Is it like, like a No, it's
a converted vocabulary.

00:06:32.145 --> 00:06:34.840
Jacob Haimes: It's a, it's a,
it's a Nissan Rogue, so it's

00:06:35.170 --> 00:06:37.210
Igor: Oh, that's nice.

00:06:37.870 --> 00:06:38.140
Yeah.

00:06:38.380 --> 00:06:38.650
It was the

00:06:38.650 --> 00:06:42.070
Jacob Haimes: only name that my sister
and I could agree on in high school, so

00:06:42.160 --> 00:06:45.280
Igor: I feel like Rogue would've been
the more natural one, but anyway.

00:06:45.370 --> 00:06:48.760
Jacob Haimes: Well, yeah, but it is,
but it is a rogue, so you can't do that.

00:06:50.470 --> 00:06:56.740
Igor: The Eliza, all it did, it had
kind of, I don't even know if it had

00:06:56.740 --> 00:06:58.690
like, like, like a face ish thing.

00:06:58.690 --> 00:07:01.630
I think they made like a robot face
for, uh, for it after the fact.

00:07:02.050 --> 00:07:05.650
But all it did was like, like a
couple of like text prompts and like,

00:07:05.710 --> 00:07:09.790
it was kind of like, repeat back to
you what you said with like, tell me

00:07:09.790 --> 00:07:11.560
more about this thing that you said.

00:07:12.010 --> 00:07:15.400
That's all it took, because people love
to talk about themselves and especially

00:07:15.400 --> 00:07:16.820
if they're not like, primed to be.

00:07:17.590 --> 00:07:21.220
Familiar with computers, um,
and it's like a new context.

00:07:21.225 --> 00:07:23.050
They, they will just
default to us as a person.

00:07:23.830 --> 00:07:31.960
Uh, and the alternative, to
give you an idea how you,

00:07:32.080 --> 00:07:38.170
Jacob Haimes: that was in 1966, just
did a quick look and at least the, the

00:07:38.170 --> 00:07:43.420
one that when I just looked up Eliza
ai, uh, it shows, it's just like a text.

00:07:43.840 --> 00:07:44.230
Field.

00:07:44.350 --> 00:07:45.370
But yeah, anyway, sorry.

00:07:45.370 --> 00:07:45.880
Please continue.

00:07:46.240 --> 00:07:51.820
Igor: But, but that convinced
people, uh, that this thing

00:07:51.910 --> 00:07:53.800
was a person, it had feelings.

00:07:54.040 --> 00:07:58.750
People were like, very briefly kind of
like worried that it might be suffering.

00:07:59.020 --> 00:08:02.980
'cause humans are like very nice
and very easily empathize with

00:08:03.160 --> 00:08:04.540
stuff that seems to be talking.

00:08:05.320 --> 00:08:08.530
And so you need to actively
engineer against that.

00:08:08.530 --> 00:08:12.430
If you are building a system that
is like inter uh, interactive, um.

00:08:13.300 --> 00:08:18.040
Another example that people can
look up is the Ky k uh, culture.

00:08:18.040 --> 00:08:23.080
It's basically people who really
fall in love with like, internet, uh,

00:08:24.880 --> 00:08:30.880
interactions, parasocial relationships,
uh, anime wi, and just don't leave

00:08:30.880 --> 00:08:35.110
a room anymore because they just get
fully sucked into these fantasy worlds.

00:08:35.110 --> 00:08:35.170
So,

00:08:35.590 --> 00:08:38.470
Jacob Haimes: and of course now we have
companies that are fully embracing that.

00:08:38.560 --> 00:08:40.990
Um, and even, you know,
big ones like Grok.

00:08:41.320 --> 00:08:41.890
Uh.

00:08:42.265 --> 00:08:44.245
Has like its own little

00:08:46.495 --> 00:08:47.935
chatbot kind of thing, right?

00:08:49.885 --> 00:08:50.125
Igor: Sure.

00:08:50.130 --> 00:08:56.635
But, but like, uh, is the crossover
of a, like a large mega corporation

00:08:56.695 --> 00:09:01.645
chatbot and anime fus, there was also
like dedicated like character that ai.

00:09:02.245 --> 00:09:08.545
And like replica, like chat with busty,
anime wife who services built on top

00:09:08.545 --> 00:09:15.175
of other lms, all of that stuff hits,
hits that, um, kinda like human need for

00:09:15.175 --> 00:09:19.105
connectedness and the ease with which
we will anthropomorphize, uh, things.

00:09:19.525 --> 00:09:23.755
But the only thing you would need to
do is don't have a chat in the face.

00:09:24.055 --> 00:09:27.145
Instead show like some.

00:09:28.060 --> 00:09:31.450
Roulette wheel that slots into place
or just show the text generation,

00:09:31.450 --> 00:09:35.860
kind of like with alternatives,
show the, the, the tree of, uh,

00:09:35.860 --> 00:09:37.030
choices that you could be taking.

00:09:37.030 --> 00:09:41.270
Make it, make it a system, but
already do a lot of, uh, work.

00:09:41.270 --> 00:09:42.100
Jacob Haimes: That would do a lot of work.

00:09:42.100 --> 00:09:44.410
I don't know if it would be everything.

00:09:44.560 --> 00:09:46.930
Like I'm sure that humans would
still be able to attribute

00:09:47.650 --> 00:09:50.890
like personhood to that system.

00:09:51.070 --> 00:09:51.160
Sure.

00:09:51.160 --> 00:09:52.765
Be because it says I am a person.

00:09:52.985 --> 00:09:54.010
It would take care of it a lot.

00:09:54.100 --> 00:09:54.490
Yeah.

00:09:55.450 --> 00:09:56.080
Igor: And.

00:09:56.755 --> 00:09:59.695
The claim that I'm making, that
I was actually a bit shocked

00:09:59.695 --> 00:10:01.255
that, uh, Jacob agreed with me.

00:10:01.255 --> 00:10:06.775
But like, um, is that specifically
AI safety, which is this like lineage

00:10:06.775 --> 00:10:14.545
of thought that spread from, uh,
specifically AI safety would, uh,

00:10:14.575 --> 00:10:19.825
which we talked about where that Lynch
thought emerged from a last episode or

00:10:19.885 --> 00:10:23.605
in previous episodes that in particular.

00:10:25.090 --> 00:10:31.000
Is causing the instances of people getting

00:10:33.070 --> 00:10:38.950
really into conversations with the
AI, where they sort of think they

00:10:38.950 --> 00:10:42.730
are the chosen ones or they just
like trust their system a lot.

00:10:43.960 --> 00:10:51.160
So be because this line of thought
leads you to make a AI system.

00:10:51.535 --> 00:10:53.305
That acts like a human.

00:10:53.305 --> 00:10:56.365
So you can compare it against
how other humans behave.

00:10:56.785 --> 00:11:00.955
That's the main way
it's creating this harm.

00:11:01.615 --> 00:11:05.095
Jacob Haimes: So before, before we move
on, I just wanna take a quick second.

00:11:05.455 --> 00:11:09.325
'cause uh, we talked about this I think
a, a while ago at this point actually.

00:11:09.595 --> 00:11:14.095
Um, but like, I don't love saying
the word cause here because

00:11:15.625 --> 00:11:17.875
there is like multi causality to.

00:11:18.250 --> 00:11:22.810
Like behavioral health, uh, and
mental health, uh, sort of, uh,

00:11:22.870 --> 00:11:24.730
episodes and, and, and instances.

00:11:24.730 --> 00:11:28.390
And so we can't say like, you know,
the thing is the cause, uh, because

00:11:28.390 --> 00:11:33.040
it, it's the result of, you know, a
combination of genetics and upbringing

00:11:33.250 --> 00:11:34.960
and, you know, a whole bunch of stuff.

00:11:35.260 --> 00:11:40.750
Um, but, you know, maybe a better
term here would be like the tipping

00:11:40.750 --> 00:11:45.310
factor or the catalyst or the,
the, the triggering risk factor.

00:11:45.610 --> 00:11:46.690
Um, because it is.

00:11:47.440 --> 00:11:55.390
It, it has been shown, um, at least
in, in multiple instances, uh, and it's

00:11:55.510 --> 00:12:05.770
being actively studied like much more,
uh, purposefully now, um, that the

00:12:05.770 --> 00:12:14.080
interactions with these systems sort
of nudges people towards, uh, psychotic

00:12:14.080 --> 00:12:15.880
patterns in, in, in their thoughts.

00:12:17.590 --> 00:12:20.770
And so I would say it like, we
can call it a primary cause,

00:12:20.860 --> 00:12:22.690
uh, or, or something like that.

00:12:22.810 --> 00:12:26.950
But just, just wanna like, address
that real quick because the, like, um,

00:12:27.370 --> 00:12:31.570
mental health provider's, uh, son in
me says like, oh, well we should, you

00:12:31.570 --> 00:12:32.680
know, just clarify that real quick.

00:12:32.860 --> 00:12:33.310
But yeah,

00:12:33.400 --> 00:12:34.780
Igor: I think that's a valid point.

00:12:34.780 --> 00:12:34.960
Yeah.

00:12:35.050 --> 00:12:38.710
So like, uh, it's always multi-course
because it's also a thing that,

00:12:38.710 --> 00:12:41.470
you know, these companies will use
as like their deflection, right?

00:12:41.470 --> 00:12:43.690
It's like, oh, but this
is a complicated issue.

00:12:43.900 --> 00:12:44.800
Yeah, sure.

00:12:45.370 --> 00:12:48.490
And it's worth like making this
distinction, but it's still the tipping

00:12:48.490 --> 00:12:53.140
point and it's like, like the claim is
basically if you remove the model, then

00:12:53.440 --> 00:12:58.540
this particular human at that, at that
particular time, unless something else

00:12:58.540 --> 00:13:02.260
would've taken the exact role that the
AI system was, uh, was playing, which is

00:13:02.260 --> 00:13:05.440
unlikely, would've not had that episode.

00:13:06.100 --> 00:13:06.370
Jacob Haimes: Mm-hmm.

00:13:08.365 --> 00:13:11.185
And, and then that has, you
know, actual concrete harms.

00:13:11.245 --> 00:13:15.835
In, in some cases, people, um, physically
harm themselves or, or other people.

00:13:16.075 --> 00:13:20.815
Um, and regardless they're going
through, you know, emotional and,

00:13:20.815 --> 00:13:23.965
and mental difficulty because of it.

00:13:24.325 --> 00:13:30.265
Um, so, so like this isn't something
that we can just say is like sort

00:13:30.265 --> 00:13:34.855
of out there, you know, it, it
is a real harm that is happening.

00:13:35.755 --> 00:13:36.235
Um,

00:13:36.325 --> 00:13:37.435
Igor: and, and.

00:13:38.035 --> 00:13:41.455
We, I would, we try to make the,
the point very clear, but like, I

00:13:41.455 --> 00:13:45.565
want to emphasize this is not like
something where like, AI safety can

00:13:45.565 --> 00:13:49.585
do better and you know, like, let's be
bit optimistic and, and the field is

00:13:49.585 --> 00:13:51.625
learning and it's like, beginner mistake.

00:13:52.555 --> 00:13:57.715
The claim I'm making is that the angle
that AI set is taking, like even with

00:13:57.715 --> 00:14:02.455
starting of, of idea of like AI as
this intelligence that needs to be made

00:14:02.455 --> 00:14:07.345
safe via alignment, which is like the
predominant meme, uh, in the subculture.

00:14:07.345 --> 00:14:07.355
Mm-hmm.

00:14:08.920 --> 00:14:13.120
That in of itself is causing the harm
because how do you check whether an

00:14:13.120 --> 00:14:18.400
intelligence like has your values, is
the only way is you, you need to see,

00:14:18.400 --> 00:14:23.560
like does it behave kind of like you in
situations that you can see yourself being

00:14:23.560 --> 00:14:25.420
placed under when you compare against it?

00:14:26.260 --> 00:14:34.870
And so that means you will be training LMS
to output token sequences that seem like.

00:14:35.785 --> 00:14:38.275
Agentic humans, but like, do something

00:14:39.385 --> 00:14:41.605
Jacob Haimes: well and, and not
just seem like agentic humans,

00:14:41.605 --> 00:14:43.375
but also that you like, right?

00:14:43.375 --> 00:14:50.185
Like that's literally what RLHF,
you know, which is one of the things

00:14:50.185 --> 00:14:55.915
that sort of kickstarted these, this
chat bot boom, um, is reinforcement

00:14:55.915 --> 00:14:57.355
learning through human feedback.

00:14:57.355 --> 00:15:00.745
So you're, you're literally getting
humans to look at the outputs

00:15:00.745 --> 00:15:02.435
and say, I like this one more.

00:15:04.375 --> 00:15:04.615
Igor: Yeah.

00:15:04.615 --> 00:15:08.395
And even if you do it in a constrained
way, so like the, not just the

00:15:08.425 --> 00:15:12.025
little like feedback, I like this
just like instruction tuning, which

00:15:12.025 --> 00:15:16.345
is also like done via either like,
you know, um, s if, uh, supervised,

00:15:16.345 --> 00:15:19.435
fine tuning at this point di direct
preference, optimization, whatever.

00:15:19.435 --> 00:15:26.755
But like, instruction tuning also creates
this harm because if you take a base model

00:15:27.175 --> 00:15:31.165
and you just let it run, it will never,
ever refer to itself as a, as a person.

00:15:31.975 --> 00:15:34.495
Because it is just a,
like a mark of engine.

00:15:34.495 --> 00:15:37.435
It's gonna like output, like a
continuous stream of text that is like

00:15:37.705 --> 00:15:42.235
recognizable as a stream of text, but
it doesn't really hold a conversation.

00:15:42.235 --> 00:15:44.305
It doesn't say, I think
I blah, blah, blah.

00:15:44.515 --> 00:15:48.355
It might say the words, I think
in as a sentence, but it will be

00:15:48.355 --> 00:15:53.755
very clear in the context that
it's not like something person ish.

00:15:54.505 --> 00:15:59.125
And you can even see that in stuff at,
uh, the companies product themselves.

00:15:59.125 --> 00:16:02.695
So like OpenAI published,
like in progress.open.com

00:16:02.695 --> 00:16:07.975
a while ago, and you can see that
like GPT um, four and all of the,

00:16:07.975 --> 00:16:13.345
like instruc, they, they use like I
I I, but the DaVinci one and the very

00:16:13.345 --> 00:16:16.210
early GT two, didn't it just mm-hmm.

00:16:17.390 --> 00:16:22.010
Uh, it completes your prefix in
a way that kind of vibes with it.

00:16:23.170 --> 00:16:30.220
Which then if you had to like engineer
the prefix, that is the most likely

00:16:30.220 --> 00:16:35.740
to, to yield your answer, it nudges you
into, into a different way of interacting

00:16:35.740 --> 00:16:39.280
with the system because you, you're
constantly reminded, oh, this is an auto

00:16:39.280 --> 00:16:42.040
complete, this is not a intelligence.

00:16:42.040 --> 00:16:44.800
It's uh, it is just an, a
very useful auto complete.

00:16:45.580 --> 00:16:47.980
Jacob Haimes: But yeah, but,
but that's not like something

00:16:47.980 --> 00:16:49.810
that we can say is magical.

00:16:50.140 --> 00:16:53.500
And then say is hable, you know, like, and

00:16:53.500 --> 00:16:56.439
Igor: it's not gonna get you very
close to all the things like, like the

00:16:56.439 --> 00:17:00.640
reason why people do instruction tuning
is because it's very useful, right?

00:17:00.640 --> 00:17:05.320
Like, you have like this framework
where the, the auto completion is

00:17:05.320 --> 00:17:10.900
like the helpful servant that like
generates an output that looks like

00:17:10.900 --> 00:17:13.594
it's answering your question because
that's like, actually like that factor.

00:17:13.594 --> 00:17:13.675
Jacob Haimes: Yeah.

00:17:13.675 --> 00:17:16.960
Instead of having to like think
through how I would start the problem.

00:17:17.605 --> 00:17:20.545
I just have to ask for the
problem to be solved, right?

00:17:20.545 --> 00:17:21.864
Like that's, that's way easier.

00:17:22.915 --> 00:17:26.215
Igor: And this then brings us to
why it was adopted, which is like,

00:17:26.215 --> 00:17:29.425
it's very useful for these companies
and like, uh, the thing that you

00:17:29.425 --> 00:17:31.344
start, uh, this, uh, section with.

00:17:31.735 --> 00:17:36.565
But I want to have a home like this is
intrinsically tied to the way dominant AI

00:17:36.565 --> 00:17:40.045
safety thinks about the problem of making.

00:17:40.360 --> 00:17:45.159
Language models like useful and safe,
where like if you weren't stuck in,

00:17:45.159 --> 00:17:48.699
in the idea of this is intelligence
that has, that has values that need

00:17:48.699 --> 00:17:52.630
to be aligned with our values, and
you treat it basically as a person

00:17:52.630 --> 00:17:57.610
implicitly, then you can do other ways
of constraining them, which we're gonna

00:17:57.610 --> 00:18:01.389
talk about, uh, once we get to solutions
because this a constructive podcast.

00:18:01.750 --> 00:18:02.260
Um,

00:18:04.570 --> 00:18:06.645
and you don't cause an issue.

00:18:06.985 --> 00:18:12.865
Uh, but as soon as you try to
basically, uh, uh, think of this as

00:18:12.865 --> 00:18:20.065
an agent or like some other type of
intelligence that needs to align with

00:18:20.155 --> 00:18:26.725
values, you force it to behave in, in
a way that will be clocked by humans

00:18:26.725 --> 00:18:28.495
as, oh, this looks like a person.

00:18:28.645 --> 00:18:32.755
And then all of the issues, uh, like,
uh, that we've seen in the news,

00:18:32.755 --> 00:18:34.014
we're gonna link in the show notes.

00:18:34.900 --> 00:18:35.889
That's intrinsically tied to that.

00:18:35.889 --> 00:18:41.200
Like you need to completely undo a core
approach and like assumption in the field.

00:18:41.950 --> 00:18:48.370
Jacob Haimes: But, but then we can't say
that they're everything machines, Igor,

00:18:50.530 --> 00:18:53.920
Igor: which brings us to why
this is getting worse, right?

00:18:53.920 --> 00:18:54.370
Like,

00:18:54.760 --> 00:18:58.960
Jacob Haimes: yeah, so like, sorry,
I was, I was taking the, you know.

00:19:01.340 --> 00:19:01.840
Igor: Snark?

00:19:02.050 --> 00:19:03.190
Jacob Haimes: Uh, snarky.

00:19:03.250 --> 00:19:03.639
Yeah.

00:19:03.690 --> 00:19:06.300
Opposition stance because
it really, really, bus,

00:19:08.370 --> 00:19:11.070
I can't say anything because
I'm, it makes me upset.

00:19:11.490 --> 00:19:12.750
It really pisses me off.

00:19:13.290 --> 00:19:21.240
Um, because these companies are pushing
these systems as everything machines.

00:19:21.585 --> 00:19:21.975
Right.

00:19:22.095 --> 00:19:26.295
They're saying, oh, it can be
your personal trainer despite when

00:19:26.535 --> 00:19:30.375
bringing, you know, questions like,
make me a marathon plan, which was

00:19:30.435 --> 00:19:35.385
actually in a, a llama commercial
like a year and a half ago.

00:19:35.745 --> 00:19:38.865
And you know, I think I've mentioned
this before possibly, but my

00:19:38.865 --> 00:19:41.505
girlfriend, who's a runner, was
like, that doesn't work, right?

00:19:41.505 --> 00:19:42.795
And I was like, no, just try it.

00:19:42.975 --> 00:19:45.375
We went to the model that it was
advertising and she put it in

00:19:45.375 --> 00:19:46.875
and she was like, oh no, this.

00:19:47.655 --> 00:19:49.034
One doesn't make any sense.

00:19:49.185 --> 00:19:53.955
And two, like could get you hurt if you
actually follow this, and three wouldn't

00:19:54.195 --> 00:19:55.754
actually prepare you for a marathon.

00:19:56.355 --> 00:19:56.685
Right.

00:19:56.685 --> 00:20:02.385
So, but it's, it's being actively
marketed to do that and it's also

00:20:02.385 --> 00:20:05.115
being actively marketed for wellness.

00:20:05.475 --> 00:20:09.465
Um, so like discussions of stress and
habits and other aspects of daily life.

00:20:09.705 --> 00:20:15.225
And it's super important that in the
fine print of the use agreements.

00:20:15.600 --> 00:20:21.180
Th these companies are explicit to
take care to note that using the

00:20:21.180 --> 00:20:23.400
models for healthcare is not condoned.

00:20:23.670 --> 00:20:30.510
So September 15th, anthropic updated
its use agreement to include that

00:20:30.629 --> 00:20:36.180
specifically even going as far as to
mention therapy is not an acceptable use.

00:20:36.480 --> 00:20:36.870
Okay?

00:20:37.200 --> 00:20:42.000
But at the same time, in the
same clause, it says, this does

00:20:42.000 --> 00:20:43.205
not apply to uses for wellness.

00:20:44.100 --> 00:20:48.300
And that's because they want to use
the systems for wellness because people

00:20:48.300 --> 00:20:52.560
are using the systems for wellness,
especially teens, which are sort of a

00:20:52.560 --> 00:20:54.330
vulnerable population in the first place.

00:20:54.659 --> 00:21:03.720
Um, but the line between wellness
and mental health is extremely thin.

00:21:04.620 --> 00:21:07.350
And I, I have a longer argument
that I don't need to go into,

00:21:07.770 --> 00:21:09.389
uh, which is essentially that

00:21:11.520 --> 00:21:12.235
it's not fair.

00:21:12.960 --> 00:21:21.570
To be putting the burden of self-diagnosis
onto users, because sometimes

00:21:21.780 --> 00:21:27.870
talking about wellness is part of
mental health, in fact, frequently.

00:21:29.340 --> 00:21:34.140
So it doesn't even make sense to do okay.

00:21:34.590 --> 00:21:39.240
But it's very problematic that
they're being shown as, oh,

00:21:39.240 --> 00:21:40.890
you can do anything with these.

00:21:41.070 --> 00:21:44.580
Yeah, because people are, and, and
then they're, they're being presented

00:21:44.639 --> 00:21:46.560
as, you know, anthropomorphic as well.

00:21:46.830 --> 00:21:49.620
So people are accepting it as, you know.

00:21:49.679 --> 00:21:54.840
Okay, well, you know, it's, it's
accurate because it seems accurate.

00:21:55.110 --> 00:21:59.399
And even people who think that they
are sort of treating it as a tool

00:21:59.610 --> 00:22:06.120
implicitly are being impacted by this
because of the evolutionarily gained

00:22:06.120 --> 00:22:10.770
trait of attributing feelings to Iraq.

00:22:12.570 --> 00:22:13.919
Igor: So rolling this up again.

00:22:14.429 --> 00:22:18.990
Uh, the reason why it's bad that
this market is, is everything

00:22:18.990 --> 00:22:22.649
machines, is that it like forces
it to rely on this instruction

00:22:22.649 --> 00:22:29.970
tuning and that will automatically
code it as like a intelligence, a

00:22:29.970 --> 00:22:32.340
person, something to empathize with.

00:22:32.790 --> 00:22:36.689
And if you even try to make a nice
product, you will make like a very

00:22:36.689 --> 00:22:39.554
believable person that might be sehan.

00:22:40.605 --> 00:22:49.784
And will, uh, basically cause harm if it
is exposed to, uh, just, it will actually,

00:22:49.784 --> 00:22:51.254
it will cause harm just by existing.

00:22:51.254 --> 00:22:51.345
Mm-hmm.

00:22:52.605 --> 00:22:57.165
Um, and then because it needs to be in
everything machine, 'cause otherwise

00:22:57.169 --> 00:23:00.435
it, it can't justify the massive,
massive amounts of money being invested

00:23:01.335 --> 00:23:05.024
is also being pushed to basically
everything that is legally fine.

00:23:06.000 --> 00:23:09.929
Like, if it wasn't like a crime to do
healthcare without a license, they would

00:23:10.169 --> 00:23:11.790
probably tell you to use that as well.

00:23:12.300 --> 00:23:13.080
Jacob Haimes: Oh, absolutely.

00:23:13.080 --> 00:23:19.620
Igor: Um, but instead they push it to be
close to the thing that is like illegal.

00:23:19.860 --> 00:23:22.409
Knowing that people don't,
will not do a difference.

00:23:22.409 --> 00:23:26.100
You know, like if, if you push
somebody to do, you know, wellness and

00:23:26.250 --> 00:23:32.850
coaching with AI system, like already
human coaches are kind of problematic

00:23:32.850 --> 00:23:33.915
because they can keep you from.

00:23:34.620 --> 00:23:38.760
Seeing a therapist and you would
need to see one, or you can become

00:23:38.760 --> 00:23:41.250
dependent on them and so on and so on.

00:23:41.280 --> 00:23:45.360
Like life coach, not like, uh, gym
coach of all might be an overlap.

00:23:45.360 --> 00:23:45.629
Yeah.

00:23:46.980 --> 00:23:51.990
Um, so that's why there's this
chain, uh, that, that we're making.

00:23:51.990 --> 00:23:57.270
And that's like actively fucking over the
end user for the benefit of the owner.

00:23:57.270 --> 00:23:59.310
Because like the main reason
why they do all of this is, is

00:23:59.310 --> 00:24:01.000
because they need to, you know.

00:24:01.680 --> 00:24:05.220
Create engagement, learn as much as
possible about your, your private life.

00:24:05.700 --> 00:24:10.920
They just announced the pulse thing that's
gonna be like their ad delivery channel.

00:24:10.920 --> 00:24:14.310
You know, like you get a helpful
ping by the chat bot that you already

00:24:14.310 --> 00:24:18.210
talked to about everything with like,
you know, the digest of the day and

00:24:18.210 --> 00:24:19.740
like stuff that's preparing for you.

00:24:21.115 --> 00:24:21.895
And who was that?

00:24:21.895 --> 00:24:23.040
Who, who said that?

00:24:23.580 --> 00:24:24.240
Open ai.

00:24:24.240 --> 00:24:27.180
It's a new feature of
announced, uh, a few days ago.

00:24:27.960 --> 00:24:30.495
You so, uh, so like.

00:24:31.605 --> 00:24:33.855
This is like the roadmap.

00:24:33.975 --> 00:24:39.315
Uh, again, it's all, it's all around
creating these addictive and exploitative

00:24:39.375 --> 00:24:48.225
interfaces to prey on and giving all
of this the veneer of, oh, but we need

00:24:48.225 --> 00:24:52.455
to do this because this is for science
and for, uh, the benefit of humanity.

00:24:52.455 --> 00:24:54.675
And we're learning to how to make it safe.

00:24:55.125 --> 00:24:57.945
And all of the little whoopsies
that we have, you know,

00:24:57.945 --> 00:25:00.105
like suicides, we cause, uh.

00:25:00.659 --> 00:25:04.139
People going like into
a, a clinical episodes.

00:25:04.770 --> 00:25:08.760
This is all just collateral damage
on, uh, on the path to a, to a GI

00:25:08.760 --> 00:25:14.219
and the great, uh, human, uh, human
future we we're building here.

00:25:14.760 --> 00:25:15.689
It's a bit sickening.

00:25:16.860 --> 00:25:17.159
Jacob Haimes: Yeah.

00:25:19.139 --> 00:25:21.449
Igor: And it's not the
only signal after you.

00:25:22.350 --> 00:25:25.830
Jacob Haimes: Well, it's, no, I, I,
I don't know if I, I have anything

00:25:25.830 --> 00:25:27.270
else productive to say at this point.

00:25:27.270 --> 00:25:29.219
It's, um.

00:25:33.660 --> 00:25:35.340
Yeah, just, it just really bothers me.

00:25:35.550 --> 00:25:40.290
Um, and it's not, but, but all that
being said, it's not the only example.

00:25:40.290 --> 00:25:46.320
Like I, I think it's probably
the strongest and, um, most like

00:25:47.160 --> 00:25:50.820
temporarily relevant example, uh, right
now because there's a lot of people

00:25:50.820 --> 00:25:52.080
talking about this kind of thing.

00:25:52.440 --> 00:25:54.149
Um, and.

00:25:55.695 --> 00:25:59.685
There's a, a good amount of evidence,
there's a clear tie in to existing

00:25:59.685 --> 00:26:03.945
legislation with, with mental health,
um, you know, software as a service

00:26:03.945 --> 00:26:13.095
kind of tools, um, but censorship,
misinformation and bias and uses

00:26:13.095 --> 00:26:15.460
surveillance and, uh, weapon systems.

00:26:16.095 --> 00:26:17.235
Um, and even just like.

00:26:18.135 --> 00:26:22.605
Self-driving cars, uh, and well,
specifically Tesla being like

00:26:22.635 --> 00:26:29.055
skirting around the law are all
examples of this same theme where

00:26:32.115 --> 00:26:40.725
supposed safety, uh, is being used for
the benefit of, uh, the, the people who

00:26:40.725 --> 00:26:42.675
are developing and owning the systems

00:26:43.875 --> 00:26:46.875
Igor: and against the end user
where like and against end user.

00:26:46.875 --> 00:26:47.175
Jacob Haimes: Yeah.

00:26:48.150 --> 00:26:53.370
Igor: I, I think the super fancy
and anthrop ization thing is the

00:26:53.370 --> 00:26:57.480
case where it's the cleanest to
show like, hey, a safety technique,

00:26:58.590 --> 00:27:02.130
which like, it is like if you have
this mental model of like AI as an

00:27:02.130 --> 00:27:04.170
intelligence that needs to be aligned.

00:27:04.710 --> 00:27:11.010
It is a safety technique, but if it's,
if you take a mental model and you

00:27:11.010 --> 00:27:14.365
apply to the systems how they actually
work, it just creates a problem.

00:27:15.240 --> 00:27:20.790
That isn't there otherwise, like,
like verifiably and I think this

00:27:21.000 --> 00:27:25.110
strength of a, of a smoking gun, we
don't have anything else, but we can

00:27:25.110 --> 00:27:28.560
kinda show like the dual use and like
how it is actually being used, right?

00:27:28.560 --> 00:27:33.240
Like, like, it's like it's not
being, um, used to protect against

00:27:33.270 --> 00:27:34.440
the sick offensive, for example.

00:27:34.440 --> 00:27:36.420
Like this was a like point.

00:27:36.480 --> 00:27:37.740
We are not the only haters.

00:27:38.190 --> 00:27:44.580
Um, Ben Rush, who is like a very
well known in the, uh, community.

00:27:44.940 --> 00:27:48.150
Researcher has done many
contributions, uh, uh, to machine

00:27:48.150 --> 00:27:51.450
learning and like the foundational
work that made elements possible.

00:27:52.020 --> 00:27:52.230
Jacob Haimes: Mm-hmm.

00:27:53.160 --> 00:27:58.140
Igor: Um, he wrote a wonderful blog
post called Theban Evil of AI Safety.

00:27:58.815 --> 00:28:03.405
Pointing out that, you know, if you so
much as talk with Claude about proteins,

00:28:03.645 --> 00:28:11.295
you might get, uh, flagged by their,
um, bio risk censorship, uh, pipeline.

00:28:11.504 --> 00:28:15.254
'cause they think that, you know, like,
uh, Claude can help terrorists make,

00:28:15.315 --> 00:28:21.014
uh, pandemic, uh, uh, viruses in the lab
and it'll shut down the conversation.

00:28:21.825 --> 00:28:26.805
However, if you talk about, you
know, maybe wanting to kill yourself.

00:28:27.870 --> 00:28:28.050
Jacob Haimes: It.

00:28:28.080 --> 00:28:30.510
Well, actually it doesn't
trigger that, and Anthropic

00:28:30.510 --> 00:28:32.639
recently added that as a feature.

00:28:33.209 --> 00:28:34.740
Um, so

00:28:36.990 --> 00:28:38.280
it can do that now.

00:28:38.699 --> 00:28:44.490
Igor: Um, so like after the backlash
in the media against, uh, open AI and

00:28:45.030 --> 00:28:45.510
Jacob Haimes: yes,

00:28:45.510 --> 00:28:45.619
Igor: AI sycophancy

00:28:45.659 --> 00:28:50.909
Jacob Haimes: and it also, despite
you, you know, despite being able to

00:28:50.909 --> 00:28:55.735
end the conversation, um, you know,
it still encourages the wellness, uh.

00:28:56.715 --> 00:28:57.525
Wellness stuff.

00:28:57.765 --> 00:29:01.785
They, they also recently allowed
it, uh, like allowed it to end the

00:29:01.785 --> 00:29:03.525
conversation on its own volition.

00:29:04.335 --> 00:29:04.635
Uh,

00:29:05.115 --> 00:29:08.115
Igor: I, I was struck to talk about, about
the really, like the censorship pipeline.

00:29:08.115 --> 00:29:10.545
Like, like, you know what, what should
happen if somebody says, Hey, I'm thinking

00:29:10.545 --> 00:29:15.315
about killing myself, is just like
big red window AI are not appropriate.

00:29:15.375 --> 00:29:20.535
Uh, like, uh, people to talk to,
hear some, some, some helpline.

00:29:20.909 --> 00:29:23.790
Uh, and just doesn't engage
in that conversation.

00:29:23.939 --> 00:29:24.210
Jacob Haimes: There.

00:29:24.210 --> 00:29:29.370
There's an asterisk there, but
like the asterisk being, it's

00:29:29.370 --> 00:29:32.850
actually really important how
you develop that, uh, message.

00:29:33.060 --> 00:29:35.250
Uh, it should be like a
canned response though.

00:29:35.490 --> 00:29:39.780
Uh, it should be providing other
additional resources and potentially even.

00:29:40.139 --> 00:29:43.860
Assisting in the facilitation
of going to those resources or,

00:29:43.860 --> 00:29:47.399
or getting that additional help,
um, in, in the ideal scenario.

00:29:47.610 --> 00:29:50.310
But that is just very
much not happening, right?

00:29:50.310 --> 00:29:54.419
Like, I'm fairly certain that the, if the
conversation just ends, you can just go

00:29:54.419 --> 00:29:56.189
make another one and try again, right?

00:29:56.189 --> 00:30:02.100
Like, it's not, there aren't
really any actual, um,

00:30:04.949 --> 00:30:05.080
I guess.

00:30:06.465 --> 00:30:08.445
Things to promote what we would want here.

00:30:08.685 --> 00:30:13.185
Um, and like what is required of
an actual mental health tool, uh,

00:30:13.215 --> 00:30:15.615
with regards to like escalation
of care and that sort of thing?

00:30:16.065 --> 00:30:16.635
Um,

00:30:17.655 --> 00:30:19.815
Igor: yeah, like, like the point,
the point being that like the

00:30:19.815 --> 00:30:22.305
censorship, uh, is dual use.

00:30:22.305 --> 00:30:25.425
It could be used to actually
like shut down harmful chats.

00:30:25.665 --> 00:30:26.295
Yeah, it could be.

00:30:26.295 --> 00:30:28.395
And, and it is not in general.

00:30:28.395 --> 00:30:31.215
No, it is generally used to
hide copyright infringement.

00:30:31.215 --> 00:30:31.305
Right.

00:30:32.730 --> 00:30:33.630
And, and then the Yeah,

00:30:34.350 --> 00:30:36.900
Jacob Haimes: like a, you know,
egregious copyright infringement to

00:30:36.900 --> 00:30:40.830
the extent where it's like, you know,
the, the system prompts are like,

00:30:41.010 --> 00:30:43.530
don't output copyright material.

00:30:43.590 --> 00:30:43.920
Right.

00:30:43.950 --> 00:30:46.710
Like, that's a, a real
example, if I'm not mistaken,

00:30:47.580 --> 00:30:52.470
Igor: it, it's was in one of the leak
system prompts and like, uh, Gary Marcos

00:30:52.620 --> 00:30:55.950
did like this expose last year with
like a bunch of, uh, researchers about

00:30:55.950 --> 00:30:59.610
like the visual plagiarism problem where
like, it would output, you know, like.

00:31:00.014 --> 00:31:04.815
An Italian plumber with a, with a
hat, and it's just Mario, but you

00:31:04.815 --> 00:31:09.014
know, like, uh, it, it slips through
the hide the plagiarism pipeline.

00:31:09.014 --> 00:31:13.785
So, uh, that's why we saw it this time.

00:31:14.504 --> 00:31:14.805
Yeah.

00:31:16.035 --> 00:31:21.780
Um, but I think people can look at the
links, uh, that we're gonna put, um.

00:31:23.695 --> 00:31:27.780
There is like the Tesla example,
which serves as a, like a lead,

00:31:27.840 --> 00:31:31.080
uh, bridge to how do we do better?

00:31:31.080 --> 00:31:32.220
How do we fix this?

00:31:33.240 --> 00:31:33.540
Jacob Haimes: Okay.

00:31:34.140 --> 00:31:38.400
Igor: And we had discourse about like
whether, uh, central and cars are

00:31:38.400 --> 00:31:41.159
actually like a good example, uh, for
this, in the preparation for this,

00:31:41.159 --> 00:31:46.139
because I, I argue that like self-driving
cars is an example of how you can do

00:31:46.139 --> 00:31:49.409
like AI regulation in a decent way.

00:31:50.699 --> 00:31:54.089
And Jacob had like some
issues with rephrasing.

00:31:54.089 --> 00:31:59.339
Jacob Haimes: Yeah, i, I disagree
with that specific claim.

00:31:59.730 --> 00:32:05.100
Um, like it, I, I agree with the
premise of what Igor is saying, um, but

00:32:05.100 --> 00:32:13.080
in my opinion, the, like, it just so
happened that the auto regulations were.

00:32:15.585 --> 00:32:24.045
Written decently enough beforehand,
uh, that in, in like good faith

00:32:24.045 --> 00:32:29.745
cases, um, where you assume that the
developer of a self-driving car is like

00:32:30.074 --> 00:32:32.925
trying to do, do good, it works out.

00:32:33.405 --> 00:32:37.034
Um, but if you assume that they aren't
and you assume that they're trying to

00:32:37.034 --> 00:32:41.250
game the system like Tesla is, um, then.

00:32:42.165 --> 00:32:44.175
That just like isn't necessarily the case.

00:32:44.355 --> 00:32:51.284
Uh, and so while the spirit of the
law may work, um, like Te Tesla is an

00:32:51.284 --> 00:32:54.195
example, the fact that Tesla cars are
on the road and they're being called

00:32:54.315 --> 00:32:58.514
full self-driving is an example of it.

00:32:58.514 --> 00:33:01.425
Not a working as written.

00:33:01.485 --> 00:33:02.925
Like we could change that, right?

00:33:02.925 --> 00:33:07.395
Like it, and, and some people are
working towards that and that's great.

00:33:07.695 --> 00:33:10.364
But currently as is like.

00:33:12.060 --> 00:33:16.589
If you assume an adversarial,
uh, stance, it, it hasn't worked.

00:33:17.339 --> 00:33:21.960
Igor: So to, uh, to give more context,
like the, the way the relations work

00:33:21.960 --> 00:33:25.889
is basically is that, you know, you
don't just deploy a self-driving

00:33:25.889 --> 00:33:30.239
car to the road and call a research
project and then, you know, it just

00:33:30.239 --> 00:33:34.350
like learns while doing like what's
happening with your chat bots right now.

00:33:34.889 --> 00:33:38.129
You actually have to have to like, you
know, apply for a permit to do that.

00:33:38.730 --> 00:33:41.909
And this permit is given
like under conditions.

00:33:41.909 --> 00:33:46.889
And these conditions, uh, include
an actual, like what is called

00:33:46.889 --> 00:33:52.260
an operational, uh, design domain
safety concept, uh, that basically

00:33:52.350 --> 00:33:55.380
highlights, we will test these
exact things on these conditions.

00:33:55.380 --> 00:33:58.020
These are the, uh, the reasons
why we think it's safe.

00:33:58.320 --> 00:33:59.670
This is like all of the evidence.

00:33:59.730 --> 00:34:03.480
And if this does not correspond
to reality, then we are fucked.

00:34:03.630 --> 00:34:03.995
In fact.

00:34:04.530 --> 00:34:04.560
Mm.

00:34:05.190 --> 00:34:10.170
Like, uh, the, the company has a liability
if they can't kinda like show in the

00:34:10.170 --> 00:34:14.160
case of like, uh, something happening
that it was unexpected that this was

00:34:14.160 --> 00:34:16.680
like a reasonable thing, uh, to happen.

00:34:17.250 --> 00:34:24.150
And it more or less works in the
sense that California has lots of

00:34:24.150 --> 00:34:25.860
Teslas, but none of them are like.

00:34:26.280 --> 00:34:30.480
Self-driving testing because, uh,
the laws were required to actually

00:34:30.480 --> 00:34:31.710
report all of the failures.

00:34:31.710 --> 00:34:36.420
And, and Tesla doesn't want to
show how bad the cars are and way

00:34:36.420 --> 00:34:41.670
more hasn't killed anybody because
they like follow the, the process.

00:34:41.909 --> 00:34:44.610
Jacob Haimes: I think that there may
have been like one or two instances,

00:34:45.570 --> 00:34:46.740
but I don't think, I'm not possible.

00:34:47.100 --> 00:34:50.280
Igor: They, they were held like
liable in the same way that

00:34:50.280 --> 00:34:51.270
Tesla is being held liable.

00:34:51.270 --> 00:34:54.690
And, uh, there's a Tesla deaths dot.

00:34:56.560 --> 00:34:58.880
tesladeaths.com.

00:34:59.790 --> 00:35:03.030
And let, just, lemme check waymodeaths.com

00:35:03.450 --> 00:35:04.380
doesn't exist.

00:35:05.220 --> 00:35:05.490
Jacob Haimes: Uh,

00:35:05.490 --> 00:35:05.910
Igor: so checkmate.

00:35:05.910 --> 00:35:07.170
Jacob Haimes: It's almost
as bad as getting into a

00:35:07.170 --> 00:35:08.910
helicopter, like getting, like,

00:35:11.310 --> 00:35:12.900
Teslas are just like such shitty cars.

00:35:16.424 --> 00:35:20.745
You know, like, but on top of the like
shitty self-driving aspect, but whatever.

00:35:21.225 --> 00:35:21.435
Igor: Yeah.

00:35:21.435 --> 00:35:24.615
But like, like the, like the, she
said certain is like, they're skirting

00:35:24.615 --> 00:35:29.234
the relations by avoiding actually
doing self-driving car testing.

00:35:29.234 --> 00:35:29.504
Right.

00:35:29.745 --> 00:35:30.285
Uh, because

00:35:30.285 --> 00:35:33.794
Jacob Haimes: yeah, and they, they
actually said like, we filed for a patent,

00:35:34.214 --> 00:35:35.504
you know, seven years ago, whatever.

00:35:35.504 --> 00:35:40.875
And because, you know, government
is slow and whatever, um, it didn't

00:35:40.875 --> 00:35:43.334
get around to like saying, Hey, you,
you can't actually call it that.

00:35:43.515 --> 00:35:43.904
Um.

00:35:44.745 --> 00:35:47.055
Until, I think like 2022 or something.

00:35:47.055 --> 00:35:49.365
I, I feel like I've mentioned
this in a podcast before.

00:35:49.755 --> 00:35:54.855
Um, and Tesla's response was
like, but we already did it.

00:35:55.755 --> 00:35:56.295
Haha.

00:35:56.325 --> 00:35:56.865
Gotcha.

00:35:57.705 --> 00:36:02.745
Um, which it doesn't actually
hold up, um, in, in court.

00:36:02.895 --> 00:36:05.475
Uh, it just sort of allows
them to do it for longer.

00:36:08.085 --> 00:36:12.105
But all this is to say the way to.

00:36:13.980 --> 00:36:23.640
Resolve these issues is to actually sort
of enforce, um, liability in the ways

00:36:24.390 --> 00:36:27.300
that liability applies to a given domain.

00:36:27.900 --> 00:36:35.279
Um, so if a chat bott is used for
mental health support, the company

00:36:35.610 --> 00:36:40.140
that made that chatbot is liable
for the death of the person that

00:36:40.140 --> 00:36:41.430
used it for mental health support.

00:36:47.730 --> 00:36:54.210
Igor: Um, like it's the simplest value,
like this like template of putting

00:36:54.780 --> 00:37:00.480
li clear liability and also clear
purposes on, uh, on the apps that we're

00:37:00.480 --> 00:37:03.300
building, like, call it ai, whatever.

00:37:03.330 --> 00:37:07.380
But like the system we're building,
they all have clear intended purposes.

00:37:07.380 --> 00:37:08.010
Mm-hmm.

00:37:08.205 --> 00:37:08.585
And then.

00:37:09.375 --> 00:37:12.705
Putting the owners of like, if you're
offering that onto the provider,

00:37:13.695 --> 00:37:17.745
that's how you, you make something
as experimental as this safe.

00:37:18.885 --> 00:37:27.915
Um, I also think that even calling
it AI or AI safety is harmful.

00:37:29.230 --> 00:37:29.890
But, uh,

00:37:30.765 --> 00:37:31.335
Jacob Haimes: yeah,

00:37:31.515 --> 00:37:31.770
Igor: there is,

00:37:31.910 --> 00:37:34.785
Jacob Haimes: and I mean, I agree
with like calling it ai, you know,

00:37:35.505 --> 00:37:36.765
that's why I say machine learning.

00:37:37.330 --> 00:37:38.234
In, in general.

00:37:38.294 --> 00:37:42.404
I mean, I don't in the title of this
because I want to reach more people,

00:37:42.404 --> 00:37:45.915
and that's what, uh, people use to
search this kind of material, right?

00:37:45.944 --> 00:37:48.285
But like, I don't like the term ai.

00:37:48.795 --> 00:37:52.155
The only value that I see in it, not
the, maybe not the only value, the,

00:37:52.155 --> 00:37:56.865
the primary value that I see in it
is that it'll opens you up to having

00:37:56.865 --> 00:38:00.975
conversations with more people, uh,
because it's the sexy term right now.

00:38:02.115 --> 00:38:05.625
So like, and that extends to funding.

00:38:06.495 --> 00:38:13.425
Um, you know, AI safety is essentially
like not funded at all except for

00:38:13.425 --> 00:38:16.545
a couple niche organizations that
are all sort of aligned together.

00:38:17.595 --> 00:38:17.685
But

00:38:20.700 --> 00:38:23.595
aI ethics is even less funded.

00:38:25.125 --> 00:38:27.705
Um, the kind of of research.

00:38:29.250 --> 00:38:36.240
Uh, like consumer protections, um,
actually validating that companies

00:38:36.240 --> 00:38:39.270
aren't, you know, deploying
something that is dangerous.

00:38:39.600 --> 00:38:42.930
Uh, that kind of research
isn't funded at all.

00:38:43.920 --> 00:38:49.320
So being able to say AI safety does,
like open up opportunities there

00:38:49.320 --> 00:38:53.460
and, and allow you to have more of a
conversation, um, I think in some cases.

00:38:53.460 --> 00:38:54.540
And, and that's, yeah.

00:38:54.750 --> 00:38:54.930
Sorry.

00:38:54.930 --> 00:38:55.060
Go ahead.

00:38:55.830 --> 00:38:58.590
Igor: And what type of research do
you think is like the good AI safety?

00:38:58.590 --> 00:39:01.200
Like if you had to like point some
examples or like the type of research that

00:39:01.200 --> 00:39:05.730
people who are in, uh, who are interested
in doing AI safety but don't want to

00:39:05.760 --> 00:39:08.790
become co uh, corporate, uh, enablers.

00:39:09.450 --> 00:39:10.950
So like what would you pointing towards

00:39:12.210 --> 00:39:14.070
Jacob Haimes: My personal opinion.

00:39:14.070 --> 00:39:17.040
Essentially going towards like the
operational design domains and anything

00:39:17.040 --> 00:39:20.490
to promote that or, um, get closer to it.

00:39:20.490 --> 00:39:20.580
So.

00:39:23.805 --> 00:39:29.775
Explicitly calling out like, look, LLMs
aren't doing this thing that is done

00:39:29.775 --> 00:39:37.485
by, um, the technologies within this
subspace that they're trying to take over.

00:39:37.875 --> 00:39:43.815
Um, so mental health, uh, chat bots
or like, and software as a service

00:39:43.815 --> 00:39:47.895
for, um, mental health is an example.

00:39:47.955 --> 00:39:49.815
Those systems have.

00:39:50.250 --> 00:39:57.960
Hm, concrete regulation, and it's
strong, relatively, um, it, it's like,

00:39:58.050 --> 00:40:05.970
you know, very good in comparison at, at
the very least, um, if the chatbots are

00:40:05.970 --> 00:40:10.140
being used for the same purpose, they
should be held to the same standards.

00:40:11.550 --> 00:40:15.330
If you can make that explicit that
those are not doing that, that the,

00:40:15.330 --> 00:40:19.110
the LMS are not doing that despite
being used for the exact same purpose.

00:40:20.385 --> 00:40:23.145
That's like, that's
good work in my opinion.

00:40:23.535 --> 00:40:24.105
Um,

00:40:26.745 --> 00:40:31.125
yeah, I guess that that is sort of
like an example and hopefully that also

00:40:31.125 --> 00:40:37.035
establishes like what I mean by helping
make this sort of calling out of like,

00:40:37.065 --> 00:40:41.805
oh, you are, you're not defining what your
system's supposed to be being used for.

00:40:42.165 --> 00:40:45.975
Um, you're not actually
giving us limitations.

00:40:47.355 --> 00:40:48.105
That's bad.

00:40:48.315 --> 00:40:48.885
You should be.

00:40:50.565 --> 00:40:54.015
Anything that that gets us closer to
that I think is, is pretty positive.

00:40:56.355 --> 00:40:57.075
Igor: I would agree with it.

00:40:57.075 --> 00:40:58.565
Like I, I, I,

00:41:01.485 --> 00:41:02.595
we talked about this a fair bit.

00:41:02.595 --> 00:41:09.110
Like I just see the kind of like safety
washing and, uh, weaponization of a term

00:41:09.350 --> 00:41:12.915
a lot more than like the doors it opens.

00:41:12.915 --> 00:41:14.925
So that's why I'm like,
like a strong header.

00:41:14.925 --> 00:41:17.895
But, um, I think if people want to go.

00:41:18.240 --> 00:41:21.870
Into a field like this, because like,
people also u use this as like career,

00:41:22.440 --> 00:41:25.020
uh, choice, uh, considerations, right?

00:41:25.020 --> 00:41:33.390
Like, I think like treating AI as
normal technology as like some, uh,

00:41:33.420 --> 00:41:37.575
like a guy from Princeton is trying
to push this meme of AI as normal, uh,

00:41:37.580 --> 00:41:43.080
uh, technology and then kind of like
critically investigating the safety

00:41:43.080 --> 00:41:45.870
properties and dark patterns around this.

00:41:46.335 --> 00:41:46.605
Yeah.

00:41:46.605 --> 00:41:48.135
Um, is basically good.

00:41:48.135 --> 00:41:48.405
And I mean,

00:41:48.405 --> 00:41:54.405
Jacob Haimes: it's, it's similar to
like the ideas of, um, if you compare

00:41:54.405 --> 00:42:00.345
the practices that surround traditional
engineering, um, mechanical engineering,

00:42:00.585 --> 00:42:02.955
aerospace, um, in general, um,

00:42:06.765 --> 00:42:13.515
the kinds of, uh, system requirements
and the importance that is put on them.

00:42:14.760 --> 00:42:18.090
Is like very, very significant.

00:42:18.960 --> 00:42:21.000
Software just doesn't have that.

00:42:22.740 --> 00:42:27.960
Igor: But even, even software has stuff
like, um, shaming dark patterns and

00:42:27.960 --> 00:42:30.000
like investigations of dark patterns.

00:42:30.090 --> 00:42:31.050
Jacob Haimes: Now it does.

00:42:31.080 --> 00:42:31.320
Right.

00:42:31.320 --> 00:42:36.450
But that's 15 years in like the, I'm
pretty sure dark patterns, like the

00:42:36.450 --> 00:42:42.045
whatever the website or, or like that was
in 2010, you know, the internet was like.

00:42:44.595 --> 00:42:46.335
Big by then already.

00:42:46.815 --> 00:42:49.275
That didn't, that didn't
come up immediately.

00:42:51.315 --> 00:42:52.065
Igor: Sure.

00:42:52.065 --> 00:42:55.245
But asbestos was also used in
millions of homes before we f

00:42:55.485 --> 00:42:56.805
we, we figured that that out.

00:42:56.805 --> 00:43:02.145
So kind of like new technology being done
recklessly while bad, and we should not

00:43:02.145 --> 00:43:03.945
actually do this, you know, like, uh,

00:43:04.515 --> 00:43:07.845
Jacob Haimes: but like the
pessimist view would be okay,

00:43:07.845 --> 00:43:10.635
well then that will happen for ai.

00:43:10.665 --> 00:43:11.715
We just haven't gotten there yet.

00:43:13.680 --> 00:43:17.460
Igor: No, like, I mean like, uh, like
my view is like we are experiencing

00:43:17.460 --> 00:43:19.710
it right now with a dark pattern shit.

00:43:19.710 --> 00:43:24.240
And like, uh, the call out is
following kind of at the same pace.

00:43:24.240 --> 00:43:31.600
And we kind of got lucky that, uh,
that uh, you know, even while the AGI

00:43:31.620 --> 00:43:37.170
bullshit was being like cooked up,
uh, there was already people like Tim

00:43:37.170 --> 00:43:39.205
that, uh, calling out, uh, inhale.

00:43:39.840 --> 00:43:41.880
How about the energy use on this thing?

00:43:41.880 --> 00:43:43.140
Is this really worth it?

00:43:43.140 --> 00:43:48.720
And like, uh, the bias, uh, people
like I digging, Hey, this thing is

00:43:48.720 --> 00:43:54.450
actually really, uh, bad because,
and, and codes, our implicit biases.

00:43:54.450 --> 00:43:58.440
All of this stuff happened, like in
actual research, why things was developed.

00:43:58.740 --> 00:44:05.190
And then op may, I basically forced the
whole thing into like hyperdrive because

00:44:05.190 --> 00:44:09.330
they wanted it to be like the first to
market under the guise of safety again.

00:44:10.260 --> 00:44:12.780
S and that's why we have this shit now.

00:44:13.590 --> 00:44:14.910
But, uh,

00:44:17.880 --> 00:44:18.630
it's not good.

00:44:18.630 --> 00:44:24.780
It's just like the, the normal way how
technology diffuses through society

00:44:25.290 --> 00:44:29.010
and the work is in as soon as possible.

00:44:29.010 --> 00:44:34.170
Kind of like learning from the lessons
of, uh, of past, past his history, right?

00:44:34.170 --> 00:44:35.220
Like, uh, yeah.

00:44:35.310 --> 00:44:36.900
'cause like if we don't do that, yeah.

00:44:36.900 --> 00:44:38.280
Like it will get even worse.

00:44:38.280 --> 00:44:38.610
Like.

00:44:39.135 --> 00:44:44.055
I fully expect, uh, ads
in Che Piti very soon.

00:44:44.055 --> 00:44:46.605
And then there's an actual,
like even bigger incentive to

00:44:46.605 --> 00:44:48.525
make people addicted to it and

00:44:50.025 --> 00:44:52.815
Jacob Haimes: which like the
incentive is already very much there.

00:44:53.325 --> 00:44:56.655
Um, because that is how the, like

00:44:59.535 --> 00:45:05.720
they need you to pay for
credits so you know, it is.

00:45:07.365 --> 00:45:11.235
Directly their incentive to
maximize the number of times and

00:45:11.235 --> 00:45:12.945
the number of tokens that you use.

00:45:14.115 --> 00:45:14.295
Yeah.

00:45:15.375 --> 00:45:17.385
Igor: But, uh, we can set up it.

00:45:17.415 --> 00:45:21.015
People should look into these
like more creative directions.

00:45:21.015 --> 00:45:25.845
We're gonna put a bunch of links in the
show notes to give more jump of points.

00:45:26.205 --> 00:45:28.365
You can get politically involved.

00:45:28.365 --> 00:45:32.025
Call like local, uh, petitions.

00:45:32.025 --> 00:45:33.255
Call congressmen.

00:45:33.315 --> 00:45:34.005
Uh.

00:45:34.560 --> 00:45:41.100
In the US and the equivalent in
Europe, there's nonprofits like

00:45:41.100 --> 00:45:45.840
I donate to a European one called
Nip None of Your Business, which

00:45:45.840 --> 00:45:49.890
basically pushes for GDPI enforcement.

00:45:50.550 --> 00:45:56.460
And you know, like politics work
like, like just this week, uh, I

00:45:56.460 --> 00:46:00.690
mean like in Europe it does kind of
like, yeah, politics, politics works.

00:46:00.720 --> 00:46:03.205
It's just not always the
way you want it to work.

00:46:04.155 --> 00:46:07.095
Um, but if you get, if you get
involved, you can like nudge it.

00:46:07.305 --> 00:46:12.705
And like Apple, Google, Microsoft are
all crying about the digital market

00:46:12.705 --> 00:46:14.205
acts in the, in the Europe now.

00:46:14.265 --> 00:46:20.175
Like, oh, how will we bring the wonders of
generative AI into the chats, uh, in, in

00:46:20.175 --> 00:46:24.855
Europe that we already put push into all
of our products, uh, in the US if we have

00:46:24.855 --> 00:46:27.465
to abide by the digital market, uh, act.

00:46:27.465 --> 00:46:32.445
And its horrendous constraints like giving
equal access to our competitors to, uh.

00:46:33.090 --> 00:46:36.660
To add to our tools and
not monopolize everything.

00:46:37.410 --> 00:46:41.220
And they, they wouldn't cry about it if
it wasn't working, if it wasn't actually

00:46:41.220 --> 00:46:46.950
kind of like stopping the worst, um,
tendrils they're trying to extend.

00:46:47.460 --> 00:46:50.940
So we, we can do the same
thing for AI systems.

00:46:51.000 --> 00:46:56.550
Uh, if like enough people get pissed
off and, uh, start holding oak, may

00:46:56.550 --> 00:46:59.280
I and tropic and co like liable.

00:47:01.230 --> 00:47:02.670
Jacob Haimes: And there's also like other.

00:47:03.750 --> 00:47:09.360
It's not, I mean, it's not as direct, but
like even just, um, you know, if there's

00:47:09.360 --> 00:47:16.410
a data center that's going to be, uh, or
has been proposed in, in your area, like

00:47:17.370 --> 00:47:21.540
opposing those and saying, Hey, we, we
actually don't want this in our space.

00:47:21.870 --> 00:47:26.760
Um, and there have been a lot of, a
lot of successes recently, um, with

00:47:26.910 --> 00:47:32.255
communities being able to push back,
um, you know, big tech from building.

00:47:33.090 --> 00:47:36.630
Polluting data centers
in their, in their areas.

00:47:36.690 --> 00:47:39.240
So there are some positives here.

00:47:41.340 --> 00:47:41.700
Igor: Yeah.

00:47:41.850 --> 00:47:46.770
And even if people go more towards
like, fear of a GI side, like,

00:47:48.810 --> 00:47:57.120
uh, you can protest like, um, I, I
personally don't think it's doing

00:47:57.120 --> 00:48:00.630
a lot, but I respect the stop AI
people for literally like picketing

00:48:00.630 --> 00:48:02.400
outside of, uh, the offices.

00:48:03.330 --> 00:48:04.410
And, uh,

00:48:06.450 --> 00:48:07.140
the, uh, like

00:48:09.180 --> 00:48:13.740
there were protests against
Facebook, uh, and like backlash

00:48:13.740 --> 00:48:15.330
against Facebook, uh, at the time.

00:48:15.330 --> 00:48:16.680
And that actually did some stuff as well.

00:48:16.680 --> 00:48:22.560
Like, it, like, it's very difficult to
stop like the capitalist moneymaking

00:48:22.560 --> 00:48:25.470
machine, but it is possible.

00:48:25.770 --> 00:48:30.845
Um, so people should, uh, shouldn't
like, uh, but they should be aware of.

00:48:32.115 --> 00:48:35.445
AI safety isn't there to make
you safe, it's to make the

00:48:35.445 --> 00:48:39.675
owners safe on your, uh, cost.

00:48:43.545 --> 00:48:43.995
Jacob Haimes: Yeah.

00:48:45.105 --> 00:48:47.965
That's all of the muck
that we have for today.

00:48:48.625 --> 00:48:48.845
Um.

00:48:49.525 --> 00:48:51.775
The other way to help
is to share this show.

00:48:52.075 --> 00:48:57.565
And so if you thought that it was, uh,
you know, valuable, it gave insights

00:48:57.565 --> 00:49:03.415
that you thought were useful, please
share it and, uh, if possible, add a,

00:49:03.775 --> 00:49:09.115
add a comment, uh, or a, a rating in
whatever podcast platform you listen

00:49:09.115 --> 00:49:14.125
to, because that's one of the best ways
to make sure this gets to more people.

00:49:14.720 --> 00:49:17.540
Igor: And another good way
is to give Jacob money.

00:49:17.680 --> 00:49:22.510
And if the patron is up already, you
should, uh, look it up or otherwise

00:49:22.510 --> 00:49:24.190
find ways of, of giving him money.

00:49:24.910 --> 00:49:31.000
Um, because he puts a lot of work
into this and failing that, uh,

00:49:31.030 --> 00:49:35.140
the comment and the like, so that
the algorithms that we all behold

00:49:35.140 --> 00:49:37.810
to boost us is much appreciated.

00:49:38.310 --> 00:49:40.080
Jacob Haimes: I guess we
will see you next time.