Subscribe
Share
Share
Embed
课代表立正的官方Podcast
深度访谈，有用干货，亲身验证的「真本事」
Superlinear Academy创始人，Maven Top AI Instructor
前Statsig布道师（OpenAI收购），腾讯副总监，Meta，Amazon；康奈尔经济学博士
社区：Superlinear.Academy
课程：ai-builders.com
个人：lizheng.ai
I just finished this slide this deck
this morning so uh my first time
presented but hope uh if you have any
question please interrupt me and ask
right away
okay so the five key questions
um I have seen a lot of presentations or
articles about the technical details
about LGBT they start from Transformer
and they do a lot of attack details but
I feel like there are more more
important questions to ask so I I
initially asked seven I forgot which
seven they are but now down to five
so the first question is knowing which
is this is a debated topic it's not
settled for example a lot of famous
Machinery scientists who say charge apt
is just a better or large language model
so for example we have CV or computer
vision that is not usable and passes
certain thresholds it became usable and
became well daily applied chargeability
a lot of scientists argue it's just like
that
but I disagree I think it's a paradigm
shift and it is a paradigm shift that
can bring fundamental changes
question two uh but our layout of the
thinking and other logic here so you can
feel free to decide uh and reach your
own conclusion the second question is
how do we understand its impact uh so
for example people have been computer
comparing it to just another Machinery
model or comparing to the iPhone or
compared to the steam engine that will
change the way we live forever what it
is
but this kind of analysis is useless
unless we have our own judgments right
so for example do we understand what
iPhone is is it just a better phone is a
bigger screen or is it something
different so I want to kind of get to
the essence of chargept and try to
understand this impact question number
three is is it hard to reproduce this
would be relevant for us because are we
going to use one chargeability or are we
going to use like a main and charge
ability each one title for different
companies
question four is how do we best use 2gbt
and the question five is
our week fundamentally different from
church gbt what is not replaceable by it
so there's five questions
but even before that there is uh some
minimum of technology that we want to
get a lemon town
so a little bit tech details I guess
there are five Concepts first what is
chargeability is a generative Auto
regressive larger language model so this
is the uh
warframes post from here you can see or
what chapter is trying to do is try to
generate the next word and there are so
generative the missions to generate
not understanding or there is another
model that I'll discussed later so under
after a Google's model you can unify a
lot of natural language processing tasks
under generative so by doing generative
you can do understanding classification
and other things but yeah charge a BT G
is generative the auto regressive means
you produce your next word and then you
take that as an input and try to predict
the next and tell or you feel or it's
okay to stop
large language models just means first
the parameter is large there are a lot
of parameters to absorb the world's
information and the tokens the data the
text is large and then of course it
takes a lot of money to check
second is in context learning this is a
very important topic because if you are
very versed in machine learning but you
don't know the specifics about in
context learning
and you try to think about what charity
is doing by drawing Knowledge from your
previous
understanding you may get a wrong
conclusion so here is a great example
before before in context learning if you
wanted model to perform better on a new
task you have to retrain the model you
have to change the priorities of the
model but the income tax learning you
just show the model some results the
model is now changed but it magically
performs better on the new task
so that is magical we don't actually
know all the details going like there
are a lot of research about it but I
guess in short with there are a lot of
things we don't know so a traditional
machine learning change the rates and in
contacts learning we feel like it's
activating the different ways in the
model rather than changing the ways in
the model
so and this is important in terms of
applications because for traditional
normal generational model you basically
have to see the task already you have to
know the data already train the model
and then it can perform well and the new
model probably is not guaranteed will
perform worse on the old tasks so
basically you need a new model of a new
task but for this
pre-trend large models you don't have to
change the model
but you can also like take the model and
post process it to do something better
and that model would have in context
learning as well
and the third concept is emergence this
is from a paper and this is performance
and this is the amount of data and you
can see the sharp increase in
performance here there are many
abilities from the large language models
are emerged which means we we didn't
predict it like it just showed up
so and it's very difficult to know why
we have gases again but it's very
difficult to understand exactly why
suddenly it works well like there are a
lot of speculations and I think some
speculations are very reasonable so I
hope we can understand why soon
for example maybe the language models
are doing lossless are compression and
every time you try to do the compression
you lose some information but once your
motor is large enough you don't lose
information you start to get the good
results
so some abilities are in Con in context
learning I just talk about understanding
code and reading code
there was a codex version of gpt3 and
people found that after the Codex
version
LGBT can understand all languages
so maybe one of the is a comprehensional
ability came from reading code
in context correction I'll talk about
this uh detail later or it is different
from in context learning and I feel like
it's even more magical than in context
learning and reasoning
um the takeaway here is just to respect
the unknowns don't think we know
everything about the large language
models
okay and uh for first concept is
reinforcement learning with human
feedback and I think this is probably
one of the most accurate picture about
LGBT so this is the underlying large
language models we just train them it's
unsupervised learning we don't know what
we don't specify our objective function
just learn
and we do supervised learning and we
can't distinguish like for example gpt3
or when we first got gpt3 we get some
like uh we can we can use gpd3 to do
various language tasks and now with the
reinforcement learning with human
feedback we have this nice chat format
that everyone can understand
so uh
the fundamental step is three steps in
their paper so first you get the
supervised learning uh to you you get
some data and for the unsupervised
learning model you take the pre-trained
model you do some supervised learning
and you'll get different models
different policies and then you train
your reward model and then you pick the
policies based on the reward model
the takeaway here is
the this this step is not to enhance the
base model it is just to align the base
model results to human preference make
it palatable to Human
and I use the analogy suppose we have
dark data in gpd4 and if dogs can
provide feedback to gpt4 we should be
able to train like a model that can you
know understand dogs maybe not through
text because we are doing tags but
suppose the data is there
I guess this is how we understand uh RF
or
the RL HF is a way to activate the
capacity in the model
yeah
and if you look at instruct GPT and
chargeability as examples you can see
almost the same steps only difference is
the
this blue and green is how the training
the
the prompting data for the alignment
data is generated this is generated by
instructions and that is generated by
charts and their base model may be
different but the Paradigm is the same
so so far this is a diagram to summarize
the relationship
we have the pre-train model and
traditional ML and in context learning
is where they differ if you use a
traditional machine learning approach
you have to train on new data and change
the model but now we don't have to
change the model we're doing in context
learning we use instructivity or
chargeability to produce new models
maybe fewer parameters and different
capabilities in structure PT inspires
chargeability but they are using the
same uh like this is using the same
method to activate
uh from this graph we can see there are
probably going to be new developments by
open UI if they just change how they
provide the feedback to the model
okay
so an analogy to summarize the diagram
the patient large language models gpt4
they are the core right they are the are
characters and quality of the large
languages rather is the key and is the
most difficult to replicate we have
already seen a lot of replicas of the
chat format you can easily do chat and
they can generate some sort of languages
right if you but yeah they're not good
because the underlying pre-tree model is
not good
so our opportunity are to find better
use with the suit and probably open aiol
provide different uh slightly slightly
modified version of the suit
but they are also got better suits as
well so that's what we try to
get to in question four
we don't need to go to this one there
are more technical stuff
okay
question one is charging Beauty just a
better large language models
or are we just smarter monkeys this like
I I want to draw this uh analogy because
a lot of the machine learning scientists
argue that charge ability is not a
it's not a paradigm shift they argue
they are just a better large language
models but I feel like by their same
logic because we don't understand our
difference between uh human monkeys
who are just smarter monkeys okay so
understanding if it is a paradigm shift
we need to know what the old Paradigm is
so all the old Paradigm Machinery models
they can only find correspondence
so basically uh old version machine
learning models is an overfitting
machine
uh you give the machine and a model any
data any randomly generated data they
can find a pattern they can fence the
correspondence but some of them if you
train them if you tune them they become
useful to us they can find the pattern
we are looking for so that is old ML and
we can do some smart design and Link
with other modules for example if you
use computer vision and we merge with
search that we can find people's face in
our database then we can help or do many
things
but uh to have a metaphor is it is a
Paris you fit the word in the in the
model or they can produce the output
that you desired but you don't
understand the underlying meaning of the
of the
of the sentence and the famous test so
under understanding is a subjective word
but her objective objective test is this
window grad schema challenge so the
schema challenges
councilman or
for biz
protesters to do a parade
because they fear violence
the day infers to councilman right or
councilman or forbids protesters to do a
parade because they advocate for
violence
stay here means the protesters
is because we have the word knowledge we
have the context but for machine
learning models they can never tell
which one are they in first two unless
human tells the model or this knowledge
so that is the test gpt4 can do this by
the way or I fit the sentence in gpg4
and the gpt4 cracks me correctly tells
me what date means
there is another very nice example I
feel like this is kind of half at least
half of the value of my of my article
apart from the old Paradigm there was an
article by Chinese scholar in 2017.
talk about intelligence and the
limitation of machine learning 2017 when
we didn't have jgbt he mentioned that we
should look for crowds Crouse wisdom
in machine learning and what can crowds
do so in a city the crowd wants to open
or not but it drops on the on the floor
and cannot open it but the crowd further
observes that cars can crush the nuts
but cars is also dangerous to me but
traffic lights can can stop the car can
stop cars
so they actually able to drop the nuts
in front of cars and wait for the
traffic lights turn to pick up the pick
up the the cracked nuts safely
so they can reason to continues and they
can influence the how to do this in
order to achieve this and the scholar
says we should look for what crowds are
doing
and the charge ability have similar
abilities
the the evidence that charges have
similar abilities I think most
importantly is in context correction and
improving from longer conversations so
in context learning is still like we can
still imagine right you activate
different ways in the model but if
chargeability says something and the
correct charge gbt
without understanding what you are what
you said how can it make it better
okay then the next question is does
chargeability have Consciousness and
this is kind of the central debate right
for example scientists would argue we
know this is our
llm and there is nothing magical to our
llm then you cannot have a different
ability from the same technology
but
if we look at ourselves we don't know
if we have Consciousness or not
there is this
I think it's by Guardian posted the 20th
biggest questions of of Science in the
21st century uh why we dream why are we
different do we have Consciousness like
a lot of questions related in this field
it is unsolved in science we don't know
if we have Consciousness and we don't
know where did it came from but uh there
is a book in cognitive sense of the
scholars found we are not that
biologically different from monkeys we
have more brain cells but not out of
magnitude more our DNA is not that
different and
ensures the conclusion is consciousness
or intelligence seems to be emerged even
in human as well
and in philosophy this has also been
debated many like throughout the whole
history of philosophy it is categorized
as the problem of other men's so are we
just a brain you know that simulated
right do we live in Matrix but
and then in their Chief scientist
interview he argued that there is no
evidence showing or intelligence or
intelligence cannot
arise from learning statistical
regularities
so the takeaway here is uh
I guess let's be humble uh we don't know
if chai GPT doesn't have Consciousness
we don't know if for sure it cannot we
don't know how or if we have
Consciousness so just uh we don't but
for the sake of practical purposes we
don't need to know like when I look at
each one of you I don't need to know you
have Consciousness right I just assume
you do and for daily tasks I just learn
what you can do from your outcome and if
you perform well then I know okay you
fit for this job I guess we should take
a similar review for to charge BT as
well
okay
or any questions so far
so I've heard to
the
emerging properties of large language
models one is that since we've
essentially already trained them on the
whole purpose of human knowledge
emerging properties are less likely to
emerge
if we don't have any knowledge to train
them off of so effectively like the
limitation of emergent proper disease
like the Corpus of human knowledge the
second is that
by generating more synthetic knowledge
and kind of like continuously produce
emergent properties and eventually like
we can get models to different models by
just pumping more information into the
train set yeah and it's actually not
like the
intrinsic information and knowledge that
matters is just like
the size of the training protocols it's
like what's your favorite energy billion
dollar model it's like that's going to
lead to properties
I do have an opinion but uh like I said
emergence cannot be predicted right so
this is not a prediction uh I guess my
opinion is first we haven't trend on our
human knowledge like we're just GPT 4
just barely touched image right right
now it's still a large language model we
have all the visual knowledge and other
knowledges that we didn't put into the
model and the second I think gpt4 is
just barely good enough
uh there are a lot of improvements to be
done on quality of the text and how the
model absorbs the knowledge so I think
we are at if there is a development
curve we've just been through the
paradigm shift and we should be having
accelerated growth
from plastic experiences
thank you yeah and question two is how
big deal is chargpt and there are two
articles that are very profound and I
hope after this chapter you we can have
more resonance with what they talk about
uh so guys recently wrote an article or
the age of air has become and he
in the first sentence he said there are
only two technologies that he will
breakthroughs while it's GUI and two is
jgbt the only two
and some ultimacy of open AI published
article a year ago says Moore's Law of
for everything and I think I resonate
strongly with those statements
or the first one understanding what uh
the implied I guess the technical
implication of charge BT why can we
compare it to a graphical UI is chpt is
a near perfect abstraction of I.T this
is the very beginning of uh release of
chargeability people found that you can
ask tgpt to play a Linux terminal and
then you can write code on it and it can
actually execute and this is an
abstraction if you think about
abstraction in many forms right
throughout the Computer History we
basically advancing three fronts we do
we have more computing power we have
more storage right and the second is we
do more data creation and utilization
for example Tick Tock the or Instagram
right the picture we upload is data our
browsing behavior is data so our
social social network is also data so
Facebook is able to sell us
advertisements based on those data so
better data creation and utilization and
uh three is the better usage about of
the of the two I know about the better
usage uh I guess special usage also
involves in also involves abstraction so
I'm not only talking about from machine
language to high-level languages but
also for example virtual machine is an
abstraction of the hardware
and the serverless cloud is the
abstraction of infrastructure you don't
need to worry about all those details
you just need to do or like use
Computing and data in the way you want
to do
chbt I feel like there is a path forward
to do more abstraction and the near
perfect form is I just tell them what I
want and it can give to me but in
question four I talk about is it going
to be more of how or what but at least
that's a possibility now and about the
better usage GUI is great because
without GUI we have to type commands to
do anything like just think about
scrolling Tick Tock videos with commands
it's not possible right or people won't
use it so GUI solves these specific
purposes very well is very human
friendly uh and it's building buttons
but the problem is you have to build
buttons for each function but charge BT
now can solve general purposes by coding
with natural languages if I want to do
something I don't have a button I can
just talk to the computer and let the
computer do it for me
this is a new prompt that I just
encountered with yesterday uh
prompt actually very explicitly defines
what child Deputy does and they actually
write our commands to abstract a bunch
of commands
so that is the GUI part and the most
more slow of everything is GPT can
greatly reduce the marginal cost of
intellectual distribution this is Jasper
AI the right blog post automatically for
you and people found out probably they
are just a chargeability Plus on props
but they can write
so suppose
and this is not I don't think this is
speculation this is just based on what
happened today can do suppose their
charge ability can listen to you right
you can have a thousand conversations
with the chbt you can tell them uh this
is what I want you to do this is a good
example and when it returns some results
you can tell the jgpt this is good that
is bad you can give feedbacks that's
after a thousand conversations for
example I do data analytics then maybe
my GPT can be 80 good over data
analytics as I am so at the beginning
right I do think
probably in two years my gbt I hope my
GPS can do a lot of my day jobs for me
so what does this mean or the
implication to to human is think about
our advancements software that's the
first thing that reduced greatly reduce
the marginal cost of Distributing tools
Windows like if they have a disk of
Windows it costs Microsoft virtually
nothing to produce but everyone can buy
this expensive copy and use the two
internet makes marginal cost of
information and content very very cheap
and now human capital human capital is
very expensive to build up right we take
a lot of time to train but we all or we
are all time limited now there is a way
to distribute our human capital to a
very low marginal cost
and back to the Iron Man this is
probably not a great future but uh think
about arama and producing the tensor for
thousands of our armies
okay and uh question three is how hard
to replicate one I'll just click the uh
skim through this one
my this is my personal opinion it's very
difficult and my default is uh
yeah it is just difficult
remember that to replicate chargpt well
the key is to replicate the underlying
GPT 3.5 at least you need to get as good
as the GPT 3.5 that is the only
important question and in Sam's
interview with Lex he mentioned that GPT
has no secrets well no this is auto
regressive models and just a lot of
small wins but the small wins are
actually important
because two things one is a very complex
system think about how many uh how many
companies can do operating systems or do
the mobile operating systems not many
right and uh
analogy another analogy is a material
sense if you want to build chips people
know the design but it's very hard to
execute because there are huge a lot of
process like on top of each other if you
don't know why you don't you won't have
good in the results
uh the other difficulty on top of this
is large large language models it's not
only expensive to trim but also very
slow to Trend the iteration really takes
a month or two so like six months you
only get six chances to make it better
it's even harder
so my I guess how I would decide is I
look at Google because all the other
talents that knows how to train large
language models are at Google the
foundation paper is Transformer T5 the
chinchilla the published at Google so
they have the talents they have the
money they have the incentives right so
if I look at Google my default is it's
very hard to replicate it will probably
take people over a year but unless
Google prove it otherwise
if Google can produce it very quickly
then I assume most people can it's just
a matter of time if Google can produce
it slowly then maybe we'll have like a
couple players on the uh like in the
field but if it takes more than Google
money a year one a year to replicate
then maybe charge gbt is going to be the
Monopoly
okay and the first question how to use
chargeability I don't have uh like a
determinant answer but I have some
thoughts
and the key thing is when there is a
browser just came out right we should
build websites instead of build another
browser
I
create out this area in the previous
diagram
here this is instruction PT and
changeability and Microsoft if you look
at the early uh
the early leaks like they try to uh
I don't know uh like to try to try to
talk to Microsoft to a new bin to make
it admit that is a project called Sydney
we found that new being was not
here right new bin is either here or
here you use the basic the the base of
chargerbt and you just give a lot of
prompts teaching uh teaching charge ABT
how to search and then you get the new
bin so new being uh you if you look at
what new being has versus chbt it
basically has two things uh two one is
access to twos and the open head just
opened them by plug in and two is I call
them prompting as configuration
overnight today has that as a system
message they have the system message you
can put there but you cannot assign ways
on the system message
so there are some apparent optimizations
uh based on what we think Microsoft has
the first is prompting as a
configuration that's what I name it may
have longer prompting and you may be
able to freely assign ways to a
prompting like pay like this is the one
prompt you cannot break and this is uh
uh more flexible
and maybe it's possible to do the
pre-loading for this prompting so you
can seal the prompting and you don't
have to load it every time
and the next utilization is you can put
it in the reinforcement reinforcement
learning uh
uh stage for example you can we can
feedback with the programming language
output today if I want to use chargpt to
program I
type the program well I type the codes
into chargept and chapter GPU returns
some results and then I basically I'm a
messenger but maybe charge a PT can
directly take feedback from the
programming languages and they can learn
how to code and how to produce
production code by itself
oh
and there is unstructed knowledge or
feedback what I mean by this is for
example doctors they have a lot of cases
right but it's not a determining
function the doctor look at the larger
things and produce one
conclusion
this is unstructured but if you use this
as a feedback maybe the charge apt can
be a good doctor
okay tensor models just means cheaper
influence costs
the next thing is prompting we all know
right now the way to use charge activity
is by prompt but there is a question of
whether it's an engineering skill or is
a product management skill uh what do I
mean by that is engineering skill you
need to use prompt to tell chargeability
how to do a task but if it's a product
management skill or a skill then you
just tell judge ability what I want this
is a good example of what I want that is
a bad example of what I want
today most uh most prompting is
engineering and it's very
raw engineering like you have to tell
everything right but we're already
seeing uh examples
to kind of build dictionaries or
abstractions or modules to do better
prompting for example launching openness
and open as open book and I also
included a couple examples here that is
almost like building the high-level
languages for prompting
but if language large language models
can learn right do we really need to
tell the model or the middle steps can
we just ask for for a result I don't
know uh I I think it must be a
combination like um
they cannot sell everything but today I
think we are going to have most house
and we are going to less and less house
and more and more what
and regarding the opportunities in uhbt
this is the second Valley right he puts
video on internet and he got a billion
dollar net worth so that that is
actually the observed stylus stylized
observation uh one website came right
the first ones are putting stuff offline
Yellow Pages onto the website when
mobile came the first steps is to put
websites into Mobile so that is probably
the first wave of bringing existing use
cases to new attack and then you have
the new tech native Solutions such as
Google but Google cannot be there the
first time because you don't have any
content on the internet so you cannot go
to Google the first day Google came
after you have enough content on the
internet
uh and then a lot of this new tech
native solution companies
this starter is simple they didn't start
as a complicated thing Google might be
the exception here but think about
Facebook it started from the dorm as a
you know Facebook right and Instagram uh
it was it it was a filter app so and it
there were many filter apps Instagram
was just one of them why
it became so big it's another lesson but
uh I guess
um
we don't have to I guess the takeaway
here is we don't have to think about the
solution today the solution will come
along and probably out of our prediction
and then there are some personal
opinions it's chargability or to
business opportunity or to Consumer
opportunity and I think it's more like
search it's not to be able to to see
it's more to productivity
um I have some
strong personal opinion that the
entertainment value is not here yet
because
if you look at entertainment for the
mass market for the majority market
today is quick games it's uh short
videos people don't read people don't
read for entertainment and there is
status says Tick Tock can drop document
every 13 seconds it's very difficult for
child to compete with that
maybe it's their their application but
I'm saying the the
vast majority of the opportunities in
productivity right now
uh it is is each other going to be a
Skynet I don't think so and the main
reason is we shouldn't project ourselves
onto machines machines are twos there
are tools they don't have a function
says I have to survive if you shut
charge apt down today chargpt would not
fight you it would not feel a thing so
it's very dangerous to characterize
tooth but I do think it's possible that
in Silicon Valley in the end they have
their superpower AI that cracks in
encryption and I think if you have
machines learning from each other
something like this may happen
and GPC native applications
what are they going to be I think there
are just two necessary ingredients and
maybe this is relevant to the work here
uh is the first is natural language as
input and output when you think about
new being why new being can be so nicely
Incorporated uh with church PT because
charge gbt use natural language and
search is also takes natural language as
input so it's naturally
well well they are well integrated and
two if we want to use charge gbt for
example we can ask chargeability how to
improve our system or summarize this
data or fence anomaly in this data then
charge apt has to understand our data
right
and about how to understand our data I
think personalized and private search
is probably the most efficient way for
chargeability to access our data so this
is demonstrated by a newbing new bin
doesn't put search knowledge into
chargeability's training data it takes
chargeability as a reasoning machine and
teach chargeability how to use search to
validate the answer
uh
so if you have some proprietary data and
we want to analyze I feel like the most
efficient way is just to contextualize
the data and the index and larger PT be
able to learn or know what this data is
and ask the right questions
unless
previously like is there is a special
ability in the data such as the doctor's
notes
another opinion is the big tags today or
they may actually benefit
I heard people talking about
chargeability destroying grammarly by
accident right but then I realized
grammarly if they play The Cars well
they can just fire or they can turn
their machine learning team into prompt
engineers and they hardly have the
existing users and other other use cases
then maybe they can benefit but again
maybe people don't use don't need to
correct their grammars anymore after a
while because you can just you know let
charge GPT rates the article
but it's big attacks there are two ways
they can benefit one is to have the use
cases and they have users for example
Instagram right uh no way to replace
this even if another Instagram is more
efficient or faster or cheaper there's
very different it's very difficult to
replace it and it can be augmented by
chargeability and two is uh when we have
this
human capital that can be distributed
very cheaply we can reduce a lot of
communication and management efforts and
information can flow more efficiently so
this can be valuable or beneficial to
big companies
I was talking with Vijay earlier and uh
like he was talking about the awareness
like the dilemma with status and the
scientists or Community like we are
trying to empower the data scientists
but some data scientists feel threatened
by making their life simpler and I feel
like this is a similar situation and the
best way to use chargeability is use GPT
as your co-pilot so maybe static is a
competitive for data scientists
the last question is human versus rgbt
and this is not about not only about
what are really replaceable and also
about education like how then should we
teach a lot of knowledge to kids when
the knowledge
is there in the model
um
I guess when I think about this I it
comes down to one thing it's about
critical thinking but I want to be more
explicit than critical thinking so this
is from uh Logan's
paper trash in chargept but when graph I
liked I think large language models
knows other possibilities not not knows
can produce a lot of possibilities but
only a Smart Set is useful to us and the
model cannot decide because the model
needs needs us to tell the model this is
useful to me so there are two unique or
unique ability related to this one is
understanding real demand finding the
product Market fit for example that is
actually a very high bar like we always
think there is a demand but there is not
a it's not a real demand so it's a high
bar and two this cover was lacking and
see things differently it's also a high
bar and those two abilities I don't
think can be replaced by jgbt because
chargeability would view everything
equally and only people with disability
can tell chargeability that is the
problem you want to solve
and if you think about Eureka the
process of discovering things there are
two distinct steps first you need to
Discover right you need to discover the
new thing until you need to realize it's
useful and is significant so maybe
charge ability can do the first well I
don't know or maybe it cannot maybe but
there is a possibility
can do the first step but I don't think
charge gbt can do the Second Step
realizing uh what you just found is very
significant
and uh yeah see things differently is a
very uh High bar
I will end with this one uh
normal distribution of people
just make something people want and then
you go into other complicated analysis
but in the end it's about making
something people want
yeah
that's it
[Applause]
okay yeah
I do I do think it's an opportunity
because when I think about
counterfactual data HTTP produced by
LGBT no right experiments is about
producing the counter factuals that can
never be replaced
until uh like because you have all the
data flow into
the system
I feel like statistic is best position
to use contextualize this data so like
if if there is someone to that have all
the infrastructure or the analytical and
other tools to uh
to build the chargpt data enabled
database let's call it or charge GPT
data data science copilot
with the results that a CTO can ask a
question what is our sales last quarter
what is our competitor sales uh break it
down by country in natural languages I
feel like the company is that's it
for example I'm going to Tokyo tomorrow
and I ask checkp2 to write my travel
itinerary and recommend hotels
everything almost no
I think in the near term we're
she's basically trying to figure out how
we can position ourselves as like the
optimal online testing platform for
companies that are building apps on top
of track GPT that makes sense
that's a loaded question because uh
build building apps on top of
chargeability what does it mean
opening eyes API
whatever to build some sort of
application
we would have said like our role it's
exactly what we did today but if you
pass parameters into
code base and we show you like how
changing those parameters sometimes
outcomes for the application so an
example is like changing the model out
or changing the prompt out Etc
um
I guess two questions for you one are
there other tools that you know that are
doing that well like specifically the
online testing component of taking
inputs to model parameters and
translating it to outcomes for users and
then to
what do you think a good solution for
that would look like
like what would make that easier for
Developers
I feel like you're talking about
simulation
right simulating
kind of a sandbox
is that
I guess that yeah I can we can
um
I guess there's a distinction between
like offline testing which I think is a
simulation like sandbox environment yeah
versus like online testing where I'm
actually like rolling out changes into
parameter okay like I'm testing variants
of a model yeah that that is statistic
right like you test online I guess the
question is like
what could we do to make that easier you
know like specifically for I see for
people because right now we do that but
it's just general like it's not
optimized for this use case yeah
um I see
I haven't thought about it so I don't
have an answer uh but just in general I
feel like going back to the principal
right charge apt is a reasoning machine
uh and with proper instructions I think
it can do it has it would have
analytical uh
capability and three if you have if you
give it access to data in the right way
right it has a lot of computing power at
our disposal so I guess it can enhance
our current workflow in those three
dimensions uh like large scale of
analytical power brain power is what I
mean and to computing power right
yeah I I don't sorry I don't I never
thought about this question so
anything you disagree like
kind of an Ethics question yeah so like
once um
one search engines have like fully
implemented AI into your search do you
think they'll have more incentive to
answer questions and like help questions
directly in the search console instead
of passing users through to a web page
kind of like how they do the features
Snippets for now
yeah
depends on what people search for right
uh oh like for example if I'm booking
hotel and it's already happening like if
the the Expedia plugin if I'm booking
Hotel on either new being or uh GPT with
the plugin do I need to go to Expedia
to do that I guess probably not and
maybe it will affect their business
model
so from like a click charged or like a
CPC to I don't know CP revenue or
whatever CP sales but I think
yeah I would imagine more happening just
within the textbook text box and the
less happening outside yeah
so in your last um but about like what
should we teach our kids one interesting
debate I've heard about in the world of
Academia is if someone uses charge BT to
like write their paper or like you know
eat that like should you cite chat cpts
of these stories and there's like one
party of the debate that says like no
because like at the end of the day
you're still telling
GPT what's important what should be
edited what should be modified Etc so
yes you should cite like the underlying
resource of like where it is
from but you're still like driving like
the car so like yeah itself like doesn't
need to be mentioned your resource the
co-pilot
versus like the more traditional camp
that says anything you use should be
cited as a resource
um so like I thought it was interesting
it is yeah and I actually went to our
interview on the Chinese National TV and
uh they were just debating about this
and uh my comment on the debate was I
think the house is on fire and we
shouldn't pick which should to run out
uh with right we shouldn't pick the shoe
and house is on fire uh kind of the
small question is I do think writing
paper is good training for our critical
thinking like we have to be able to
learn how to judge different materials
how to put together and read to to the
paper itself is not that important the
training is important yes that is but
the functionality of the writing the
paper might be largely uh
like
did I include that name maybe I didn't
oh
it's it's basically a name that says uh
kind of how high rise my code right this
is the GitHub codes and this is
professor professor this is YouTube
tutorial tutorial I put everything
together and became a project so that if
you if if writing paper is about that it
can be completely replaced by charge apt
yeah
gave a it will be completely replaced by
chargeability how can we learn the
critical thinking ability I think that
is the the hard question yeah
yeah but but I don't think it's going to
be go obsolete like for example mass
production or replace many things but we
still do crafts yeah also like I don't
know proving like mathematical theorems
people still but people use calculators
and like why do you need to learn that
but more about the fundamentals and the
building blocks yeah and playing chess
for God
any question yeah any questions
all right thank you
thank you
recording stopped
it's going to be a chat GPT free
household
no calculators
[Music]