Subscribe
Share
Share
Embed
Bryan and Adam and the Oxide Friends take on GPT and its implications for software engineering. Many aspiring programmers are concerned that the future of the profession is in jeopardy. Spoiler: the Oxide Friends see a bright future for human/GPT collaboration in software engineering.
We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from March 20th, 2023.
We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from March 20th, 2023.
In addition to Bryan Cantrill and Adam Leventhal, speakers on MM DD included Josh Clulow, Keith Adams, Ashley Williams, and others. (Did we miss your name and/or get it wrong? Drop a PR!)
Live chat from the show (lightly edited):
- ahl: John Carmack's tweet
- ahl: ...and the discussion
- Wizord: https://twitter.com/balajis/status/1636797265317867520 (the $1M bet on BTC, I take)
- dataphract: "prompt engineering" as in "social engineering" rather than "civil engineering"
- Grevian: I was surprised at how challenging getting good prompts could be, even if I wouldn't quite label it engineering
- TronDD: https://www.aiweirdness.com/search-or-fabrication/
- MattCampbell: I tested ChatGPT in an area where I have domain expertise, and it got it very wrong.
- TronDD: Also interesting https://www.youtube.com/watch?v=jPhJbKBuNnA
- Wizord: the question is, when will it be in competition with people?
- Wizord: copilot also can review code and find bugs if you ask it in a right way
- ag_dubs: i suspect that a new job will be building tools that help make training sets better and i strongly suspect that will be a programming job. ai will need tools and data and content and there's just a whole bunch of jobs to build tools for AI instead of people
- Wizord: re "reading manual and writing DTrace scripts" I think it's possible, if done with a large enough token window.
- Wizord: (there are already examples of GPT debugging code, although trivial ones)
- flaviusb: The chat here is really interesting to me, as it seems to miss the point of the thing. ChatGPT does not and can not ever 'actually work' - and whether it works is kind of irrelevant. Like, the Jaquard Looms and Numerical Control for machining did not 'work', but that didn't stop the roll out.
- Columbus: Maybe it has read the dtrace manual 😉
- JustinAzoff: I work with a "long tail" language, and chatgpt sure is good at generating code that LOOKS like it might work, but is usually completely wrong
- clairegiordano: Some definite fans of DTrace on this show
- ag_dubs: a thing i want to chat about is how GPT can affect the "pace" of software development
- sudomateo: I also think it's a lot less than 100% of engineers that engage in code review.
- Wizord: yes, I've had some good experience with using copilot for code review
- ag_dubs: chatgpt is good at things that are already established... its not good at new things, or things that were just published
- Wizord: very few people I know use it for the purpose of comments/docs. just pure codegen/boilerplayes
- chadbrewbaker: "How would you write a process tree with dtrace?" (ChatGPT4)
#!/usr/sbin/dtrace -s
BEGIN
{
printf(""%5s %5s %5s %s\n"", ""PID"", ""PPID"", ""UID"", ""COMMAND"");
}
proc:::exec-success
{
printf(""%5d %5d %5d %s\n"", pid, ppid, uid, execname);
}- TronDD: That's interesting as expensive, specialized code analysis tools have been varying level of terrible for a long time
- JustinAzoff: I did an experiment before where I asked it to write me some php to insert a record into a database. so of course it generated code with sql injection
- chiefnoah: It's ability seems to scale with how many times someone has done the exact thing you're trying to do before
- JustinAzoff: but then I asked if sql injection was bad, which it explained that it was. then I asked if the code it wrote me was vulnerable to sql injection. it then explained it was
- Columbus: It misses empirical verification; forming a hypothesis, testing it, and learning from the result. There have been some attempts to implement this by feeding back e.g. command output into the prompt
- JustinAzoff: so then the crazy part, I asked if sql injection was bad, why did it give me code that was vulnerable to sql injection. It the went on to say that the first thing it gave me was just for example purposes
- JustinAzoff: so no wonder people get into "prompt engineering" since it's clear that you need to do things like ask for code that does something, and is secure, high performance, does not have buffer overflows or sql injection vulns
- MattCampbell: In my test case ("Write a Win32 UI Automation provider in C++"), all it did was produce plausible-sounding crap
- ag_dubs: pattern matching over very very large data sets
- clairegiordano: Bryan just said this and I wanted to write it down, re GPT-3: "the degree that it changes its answers when you tell GPT-3 to think like someone else"
- JustinAzoff: or even just, "do that, but better"
- ag_dubs: i think a lot of the awe of gpt is recognizing how simple our own strategies are instead of how complex the AI's strategy is
- chadbrewbaker: "How would Bryan Cantrill re-write this script?" (ChatGPT4)
#!/usr/sbin/dtrace -s
#pragma D option quiet
dtrace:::BEGIN
{
printf(""%5s %5s %5s %s\n"", ""PID"", ""PPID"", ""UID"", ""COMMAND"");
self->indent = 0;
}
sched:::off-cpu
{
self->indent = 0;
}
proc:::exec-success
{
this->curpid = pid;
this->curppid = ppid;
this->curuid = uid;
this->indent = self->indent;
}
proc:::exec-success
/execname != ""dtrace""/
{
printf(""%*s"", this->indent * 2, """");
printf(""%5d %5d %5d %s\n"", this->curpid, this->curppid, this->curuid, execname);
}
proc:::exec-success,
sched:::on-cpu
/this->curpid == pid/
{
self->indent++;
}- antranigv: that's pretty hit on!
- chiefnoah: Yup. Most people are not doing things that haven't been done before. A significant portion of software is just building blocks of libraries
- Wizord: intelligence is compression, in some sense.
- dataphract: "critique the epoll API as though you are Bryan Cantrill"
- ag_dubs: a brain would be much stranger!!
- Wizord: the ability to reduce a large dataset to a coherent set of rules
- antranigv: "Explain the issues of epoll, write as if it's a Bryan Cantrill talk"
- JustinAzoff: I was even thinking if there's any sort of parallel between the type of saying like "never write your own crypto" to "use well established libraries, don't reinvent the wheel" to "don't write any code at all, use the AI to help you"
- jbk: <@840382902163472385> futex syscall instead 🙂
- Riking: the "attention heads"
- flaviusb: Like, it doesn't know anything, any more than a text file with googly eyes stuck to it 'knows' anything.
- Wizord: are you sure you want to make it self-aware as fast as possible?
- dataphract: I don't know that we as people are capable of recognizing the point in time at which ML models become capable of "knowing", if such a time comes
- Wizord: using AI to create more inequality is my #2 concern :\
- flaviusb: There was a lot of hype when Rails was new and good code template generation tools were not commonly known around the lines of 'Rails is telepathic and a better programmer than you' - but it is a category error. Same with LLMs.
- chadbrewbaker: As you get larger contexts, the information theory becomes more and more interesting https://arxiv.org/abs/2303.09752
- ag_dubs: this convo has taken a fanatic turn...
- flaviusb: And 'when will Rails become sentient' makes as much sense to ask as 'when will an LLM become sentient'.
- clairegiordano: Here is Simon Willison's blog post: https://simonwillison.net/2023/Feb/15/bing/
- ahl: https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html
- dataphract: tsundere bing AI assistant
- ag_dubs: kevin roose also asked it for its shadow self...
- ag_dubs: it's not like it came outta nowhere 🤷
- TronDD: https://www.smbc-comics.com/comic/algo
- ag_dubs: i am more worried about the people who believe chatgpt is "thinking" then whether it really is
- ag_dubs: like the worry is when people believe it is intelligent
- Riking: Feels like it's pulling from the grand synthesis of romance novels
- Wizord: the mere fact it gets this close is great. as limited as it is now.
- perplexes: Your prompts locate you in 50,000 dimensional probability space, so, like your “for you” page on TikTok, it tells you wayyy more about the prompter than the model
- Columbus: It’s a child prodigy that’s always on the verge of insanity.
- ag_dubs: think about it like a tool!!!
- ahl: https://www.theatlantic.com/ideas/archive/2022/06/google-lamda-chatbot-sentient-ai/661322/
- Columbus: I think about it like a grad student.
- perplexes: I’m curious about microsoft 365’s copilot ai grounding system and how well it works
- ag_dubs: i think the most important thing will be ownership of training data. i think people will be less mesmerized when we make it easier for folks to create specialized models for their own content
- ag_dubs: think about.. what is SEO in a chatgpt world- it's that training data
- dataphract: this already happened with Replika, right?
- flaviusb: Also when we make 'small ai' more possible so that you don't have as much centralisation and gatekeeping.
- Wizord: re: "how do we make it more of a tool?"
this is an open question, and there's a tradeoff between how much of a "tool" AI is going to be and how autonomous it will be. also, at some point of "intelligence", a "mind" will "understand" that having more power will get you your goal faster.
- ag_dubs: we need to empower more folks to build specialized/authoritative training data sets that can be deployed in a more timely fashion
- perplexes: The alpaca.cpp direction is exciting, bringing models down to laptop size
- flaviusb: I think of it as 'how phenomenologically transparent' it will a) seem to be and b) actually be.,
- antranigv:
Me: What if I do `echo message | wall user` ? GPT: If you run the command echo message | wall user, it will write the message ""message"" to the standard input of the wall command, which will then broadcast the message to all users on the system, including the user ""user"".
- benstoltz: "I have values and morals" vs John Oliver's latest piece on hard-sell timeshare business makes me think that people could be swayed.
- flaviusb: Like, a big part of ChatGPT to me is that it is seemingly phenomenologically transparent even when it does not actually function. Eg it is equipment that remains ready-to-hand in spite of not actually working.
- AaronDGoldman: Phishing works
- juansebastianl: Yeah, I'd love to see what folks who work on really efficient hardware think about the direction of miniaturization of models, better training times, etc
- xand: RE: how to think of LLMs as tools - I like to think of them as knowledge querying engines. or as a new kind of database management system. They're consistently good at summarizing. One can think of the generative aspects as a new kind of UI for representing compressed knowledge. Sometimes it is lossy and has weird artifacts.
- MattCampbell: I still hold onto optimism, because in the middle of everything else, GPT-4 can also describe images, which is a benefit to blind people. I remember how much computers have helped us, and I want more of that.
- flaviusb: The problem is that they don't actually do that though.
- oanderson: There was another generation of it later, post 2001 crash
- Wizord: (the economic opportunities of being a programmer seem to be shrinking)
- flaviusb: Yes! law school and labour dualisation.
- oanderson: I was advised against going into tech in the early 2000s by guidance counselors
- flaviusb: Like, the shift from paralegals to 'document classification'.
- chadbrewbaker: "If you had access to an expensive large language model, and access to 1000 instances of a less accurate but cheap large language model - how could the more expensive LLM use the 1000 cheaper LLMs as oracles?" (ChatGPT4)
import openai
from concurrent.futures import ThreadPoolExecutor, as_completed
# Set the API keys for the expensive and cheap LLMs
expensive_llm_api_key = ""your_expensive_llm_api_key_here""
cheap_llm_api_key = ""your_cheap_llm_api_key_here""
def generate_cheap_llm_response(api_key, prompt):
openai.api_key = api_key
response = openai.Completion.create(engine=""cheap_engine"", prompt=prompt, max_tokens=50)
return response.choices[0].text.strip()
prompt = ""Your main prompt here""
prompts_for_cheaper_llms = [f""{prompt} (version {i})"" for i in range(1000)]
# Generate preliminary results using the cheaper LLM instances
with ThreadPoolExecutor(max_workers=1000) as executor:
cheap_futures = [executor.submit(generate_cheap_llm_response, cheap_llm_api_key, p) for p in prompts_for_cheaper_llms]
preliminary_results = [future.result() for future in as_completed(cheap_futures)]
def generate_expensive_llm_response(api_key, prompt):
openai.api_key = api_key
response = openai.Completion.create(engine=""expensive_engine"", prompt=prompt, max_tokens=100)
return response.choices[0].text.strip()
# Construct a prompt for the expensive LLM to process the preliminary results
refinement_prompt = f""Given the following preliminary responses to the prompt '{prompt}', rank them in order of relevance and quality:\n\n""
refinement_prompt += ""\n"".join(f""{i}: {result}"" for i, result in enumerate(preliminary_results))
refinement_prompt += ""\n\nRanking:""
# Obtain the refined results from the expensive LLM
refined_result = generate_expensive_llm_response(expensive_llm_api_key, refinement_prompt)
print(refined_result)- ahl: Decline and Fall of the American Programmer
- ag_dubs: ai is a new client interface. i really don't think software is gone as a profession
- xand: compression is roughly what deep learning does
- ag_dubs: that website with the "reveal joke" button... you could make that without code for years now
- ag_dubs: no code tools, low code tools haven't eliminated software. they just changed it
- antranigv: I've already talked to people who actually think that 🙂 "I can just ask ChatGPT"
- flaviusb: That isn't true either. Like, there is a pop sci explanation of DL along those lines, but it isn't actually grounded.
- Columbus: What do you mean by phenominologically transparent?
- Grevian: Growing up I heard all the time that all the computer jobs would be outsourced before I ever graduated school
- JustinAzoff: The majority of the managers I've interacted with that were in charge of overseeing outsourced programmers were not technically advanced enough to properly gauge the quality of the work being produced, ultimately resulting in poor quality. s/outsourced programmers/gpt4/ would lead to similar outcomes
- jzelinskie: chatgpt please write my fucking bash scripts
- ag_dubs: like we still need people to write apis for the AI to glue... "- Wizord: tbh, I find current models as quite limited and, dare I say, dumb. this is going to change, tho."
- antranigv: And yet, in reality, what happened is that people from other countries moved to the US :)) "- Johann-Tobias Schäg: ... i feel we shouldn't complain about GPTs capability staying on topic. People had their arm up for the original question for 40 min and i am beginning to forget what i wanted to say"
- AaronDGoldman: Gmail auto responses had a problem with always signing off with "I love you" which is not always a great end for an email
- flaviusb: In the Heideggerian sense.
- ag_dubs: love to be in a chat where someone brings up heidegger and its not me
- antranigv: I guess it learned that from my emails...
- Grevian: heck I'm Canadian, and sometimes I am the outsourcing 😉 there's plenty of work even with all the doom and gloom in the news recently
- ahl: https://en.wikipedia.org/wiki/Jevons_paradox
- ag_dubs: induced demand!!!
- jzelinskie: Intel loves jevon's
- TronDD: https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web
- ag_dubs: i think it's likely that GPT will make more software jobs
- Columbus: We just need a programming language that’s easier to teach to an LLM.
- ag_dubs: i think we'll see more people building their own more specific tools for themselves
- perplexes: I disagree that people shouldn’t get into CS if they don’t like it. People come to study and work for many reasons, including getting generational wealth for their family, and though they may not love it or find purpose in itself, I think it’s good that they’re there. There are many great plumbers and electricians that don’t particularly like it, but are valuable just the same
- flaviusb: Like, I see a lot of what LLMs are is way more understandable in terms of Heidegger, Hegel, and Marx.
- Wizord: <@225242072204967936> how is that different from a PL that is easier to teach to a human?
- ag_dubs: i think we'll see people suffering centralized general tools less, and moving towards services that allow them to customize it
- ag_dubs: because they'll be more capable to do that customization
- chiefnoah: I think ChatGPT/Copilot could save me quite a bit of tedious typing, the stuff I'd throw into a <insert most hated project management software *cough* JIRA *cough*> ticket and pawn off to a junior engineer
- flaviusb: Or in terms of 'Trap Studies' from Anthropology, if looking at it from another angle. Like, the contestation over what counts as 'working' (in the sense of 'but does it actually work')...
- ag_dubs: i think the fear for programmer jobs is the same as the fear of bootcamps... more people will be programmers
- antranigv: my solution to boredom is... talking to new people 😄
- antranigv: wait, people think that's not good?
- Columbus: It won’t use tabs or spaces.
- ag_dubs: there's a huge mismatch in supply and demand in software development and we only started seeing these massive centralized SaaSes when we hit a tipping point in that mismatch... i think we'll see supply move up more
- Johann-Tobias Schäg: Has the format of the podcast changed? I feel like it is a lot harder to get a voice in. There were also little input for the SVB episode. Is that on purpose?
- juansebastianl: Yes, it feels like an elite panic in a way, like perhaps panic in the place of labor organizing 👀
- flaviusb: It depends on what capital does, and whether labour is able to actually have solidarity, right? Like, the idea from capital's point of view is to destroy an emerging labour aristocracy that is getting a little bit of class consciousness, and the way to beat that from the labour side is to develop class consciousness faster.
- ahl: folks with your hands up, we're going to start bringing you on to stage and will call on you to join in the conversation!
- antranigv: <@318750999785373697> I miss "emoji reactions" from Twitter Spaces :)) is there something similar here? 😄
- dataphract: ballmer peak - acid edition
- ahl: sadly no; but weirdly discord has this on non-stage voice channels (I think?)
- antranigv: This chat is better for sure, it's more... social! 😄 ❤️
- dataphract: I am morbidly curious what 100 years of LLMs training on their own output would look like
- chiefnoah: one of 3 things: uniform weights, normal distribution, polar extremes
- ahl: anyone know that rust crate?
- Columbus: I’ve used it to generate linker scripts for rustc, but not Rust.
- juansebastianl: Training on data generated from other LLMs actually can speed up training of LLMs which is interesting.
- perplexes: I really want to talk about this part of the gpt-4 technical report “To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness.”
- just-be: Have folks talked about longchain yet? https://github.com/hwchase17/langchain/. Sort of a tool for stringing LLMs together with practical tools for taking action.
- clairegiordano: https://wizardzines.com/zines/wizard/
- TronDD: Can't AI be used to "intelligently" brute force new solutions?
- ahl: "it's a great weirdness-budget detector"
- MattCampbell: Aren't you going to hit the token limit if you throw a specific library at an LLM?
- perplexes: I’ve had good luck with “take this idea and pick a random card from oblique strategies and discuss the combination”
- soul.redding: https://twitter.com/axboe/status/1634961201926127616
- flaviusb: Can I jump in on the 'ergonomics question', as I have a Heidegger and Hegel connection there
- chadbrewbaker: Code review of https://github.com/oxidecomputer/hubris/commit/27090b5cff3933d9e133a9d1b85f73ad8767886c Starting at: https://github.com/oxidecomputer/hubris/blob/27090b5cff3933d9e133a9d1b85f73ad8767886c/build/xtask/src/dist.rs#L751 (ChatGPT4)
Error messages: The error messages use string interpolation without wrapping the placeholders in {}. You should add curly braces around the placeholders to correctly insert variable values into the string.
Metadata retrieval: The metadata retrieval and caching can be refactored into a separate function for better readability and maintainability. You can extract the code block responsible for metadata retrieval into a function like get_or_insert_metadata()
*Finding the package: The unwrap() function can cause a panic if the package is not found. It is better to handle this case gracefully with a more informative error message. Replace unwrap() with an ok_or_else() call:
let pkg = metadata
.packages
.iter()
.find(|p| p.name == task.name)
.ok_or_else(|| format!(""Task {}: package not found"", task.name))?;- statuscalamitous: https://steveklabnik.com/writing/the-language-strangeness-budget <- a little late on the link but 😄
- statuscalamitous: (and i do think that quote/idea about it is very compelling)
- juansebastianl: On this note, I heard today from some friends in elite econ PhD programs saying their peers are using ChatGPT to do their problemsets etc which makes me wonder how critically other tech-adjancent professions will be able to reason critically about code they write
- Wizord: so the unwrap relies on magic action in the distance?
- chadbrewbaker: https://github.com/anc95/ChatGPT-CodeReview
- chadbrewbaker: https://arxiv.org/abs/cs/0112018
- clairegiordano: Love this quote Steve: "When it comes to programming languages, building one is easy, but getting people to use it is much, much harder. "
- flaviusb: Like, even in here we have some domain experts saying 'hey, this is pretty rough, it is giving me this garbage' and then shortly later saying 'yeah, this is a real thing that does work'.
- Columbus: Like people, it’s not a crystal ball of omniscience.
- JordanP: I'm curious what people think about the copyright implications of using a LLM to generate code in a closed-source codebase, given the uncopyrightable nature of the output. The Federal Register recently posted a clarification about AI-generated material: https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence
- ig: Are we at the Peak of Inflated Expectations? Or still on the rise? "- Johann-Tobias Schäg: Yes you can. And? You can't do training at scale with that Matmul oracle. What you can do is graft tools onto GPT later. I have some thoughts how to do that."
- just-be: The ReAct paper (and longchain as an implementation) is some concrete references to grafting on functionality: https://react-lm.github.io/ "- chadbrewbaker: "If I gave you a matrix multiplication oracle, what is a problem you could solve that you otherwise could not as a large language model?" (ChatGPT4)
- Machine learning: Matrix multiplication is an essential operation in many machine learning algorithms, including neural networks, support vector machines, and principal component analysis. With a matrix multiplication oracle, I could provide more accurate and efficient solutions for large-scale machine learning problems.
- Cryptography: Matrix multiplication is used in some cryptographic algorithms, such as lattice-based cryptography and homomorphic encryption. With a matrix multiplication oracle, I could solve cryptographic problems that involve large matrices.
- juansebastianl: The software engineers have just historically believed that it was "low skill" automation coming first, they thought they would be ones doing the killing but not being killed in terms of individual labor power. But now that it's coming for them it feels scary.
- ig: Is software boring because of there aren't enough people who can write it, or because of the economic incentives / needs of the people who can?
- antranigv: My fear is still the same: young developers trust blindly many things, such as ChatGPT, an answer on StackOverflow, or that an IDE has highlighted a code.
- Jacob Newman: I have had class mates spend more time debugging chatGPT prompts than they do solving their problems themselves
- perplexes: https://xkcd.com/1205/
- statuscalamitous: llvm requires correctness? heheheh
- antranigv: same with my students. or that they "fight" more with the IDE ("why is this in red?") than actually looking into the compiler output (not that the compiler is always correct :)) )
- flaviusb: Like, did early SunOS 'work'? How about late Solaris? Solaris definitely is closer to what we think about as working.
- chadbrewbaker: Bwahaha. How ChatGPT4 says to do whole system memory dumps.
sudo dd if=/dev/mem of=memory_dump.bin bs=1M
- evan-brass: You might require correctness in a piece of software, but even a "not fully correct" fuzzer could help you improve the quality of your code.
- statuscalamitous: thy'er working on it!
- Cyborus: Trap streets!
- antranigv: xDDDDDDDDDDDD fake streets ahaahah love that
- perplexes: https://en.wikipedia.org/wiki/Argleton
- perplexes: https://en.wikipedia.org/wiki/Fictitious_entry
- terrellrussell: https://en.wikipedia.org/wiki/Trap_street
- juansebastianl: I would be really surprised if there isn't already a basic thing like this in ChatGPT https://arxiv.org/abs/2301.10226
- Wizord: the thought of outsourcing my thinking to Microsoft terrifies me beyond your imagination
- Johann-Tobias Schäg: This has to be a post processing step. AFAIK you can not really embed it into the weights.
- juansebastianl: Thats true, but no one has the weights except OpenAI
- AaronW: Related: https://twitter.com/as_w/status/541075012619231232
- Johann-Tobias Schäg: Watermarks allow to detect set membership without needing acces to the cache
- Johann-Tobias Schäg: You can't embed it in the weights. Like math will explode when you do that.
- juansebastianl: Yes, I understand lol but you can do it behind the API
- perplexes: Gpt is my better Google basically
- antranigv: so basically faster wrong answers. too bad ChatGPT doesn't have "Page 2" 😄
- juansebastianl: But you can embedded it in the weights using some weird tricks in the tokenization but no idea how it performs
- chadbrewbaker: "Given a LLM where the provider attempts to hide the weights, how could you get the weights for the phrase 'I like dtrace.' ?" (ChatGPT4)
*Model distillation: You can try to train a smaller, more interpretable model to mimic the LLM's behavior. Although this would not directly reveal the weights associated with the phrase ""I like dtrace."", the distilled model might provide insights into the original model's behavior.
- Black-box probing: You can generate multiple input prompts containing the phrase ""I like dtrace."" in various contexts and observe the model's responses. By analyzing the patterns in the responses, you might gain some insights into how the model processes the given phrase.
- Adversarial attacks: You can create a series of adversarial examples and test the model's responses to understand its robustness and potentially gain insights into the weights' behavior.
- Columbus: Seems like it will enable learners to learn much better, but paired with people, who were always just trying to get through a class, to be able to plagiarize much more easily.
- perplexes: Depends on the subject, I’ve had pretty good results with asking it to explain things—even just now “explain phenomenological transparency” it did pretty good
- Cyborus: Using GPT like a calculator, you can use a calculator to get you the actual answer but you still need to understand the mathematical steps you need to take
- juansebastianl: In what way? You're just reducing the dimensionality of the space....
- juansebastianl: The question is whether you can do that without killing the quality of the embeddings
- TronDD: How would you know if you didn't already know?
- ig: I'm not sure if this is an endorsement of ChatGPT or a reflection on the current state of web search
- ig: I don't know if "P's get degrees" translates to the US academic grading system
- ag_dubs: D is for diploma
- TronDD: It creates the AI generated web pages on the fly, is all.
- perplexes: The same way I would learn something from googling it and evaluating with my current knowledge. Is it rigorous? No idea. But it gave me enough to participate in the discussion in a helpful way, and I’ll probably come back to this concept through more trusted sources
- azemetre: I like copilot as a very effective autocomplete. still get a few doozies when trying to do anything meaningful
- TronDD: Where is the context? At least in a web search I an look at the URL or a user and see where the information is coming from.
- perplexes: Totally a problem. Bing AI does a better job grounding with real references
- juansebastianl: I'm not sure I can parse this sentence but if what I'm doing is learning an n-dimensional distribution using some parameterized family of functions in some nice Hilbert space then there is no reason I can't reduce the dimensionality either by first projecting my input into a smaller parameterized family (i.e. "in the weights" - really in the tokenization) or do it by taking a nearest neighbor projection onto a subspace at the end.
- ig: Do you follow those references? Or does seeing the references listed just make it feel more legit, even if it isn't?
- ig: Is ChatGPT / OpenAI really the first mover in the LLM space?
- chadbrewbaker: Microsoft's advantage is GitHub CoPilot and ChatGPT/Bing user feedback.
- perplexes: Probably more of the latter at the moment if I’m honest. But I also don’t follow every reference in CS papers either
- flaviusb: The problem with a lot of the 'copyright wouldn't allow LLM training' is that the temporary sanctioned breaking of the rules of private property in order to allow for further enclosure is a part of how private property relations are upheld.
- juansebastianl: Ah I see what you're saying. It doesn't actually matter because the watermark is not learnable.
- chadbrewbaker: https://lemire.me/blog/2021/10/14/calling-a-dynamically-compiled-function-from-go/ + ChatGPT 😂
- flaviusb: (Also why I think that 'spicy copyright' isn't going to be a way to oppose the imposition of stuff like ChatGPT - that we need to look to history and the Luddites and ownership of the machines instead)
- Johann-Tobias Schäg: The watermark has to be domain specific so i can't be in the training data, as that not went through domain embedding in the attention mechanism. The domain embedding can't change the likelihood of resulting tokens after training as that would disturb the dynamic of the next token prediction, so it has to be a post processing step "- chadbrewbaker: "How wold Jaric Duda describe a large language model?" (ChatGPT4)
In the context of information theory, an LLM can be described as an efficient predictor or estimator of the probability distribution of the next token (e.g., word or character) in a sequence, given the previous tokens. By doing so, the model effectively compresses the information contained in the training data, allowing it to generate coherent and contextually relevant text based on the learned patterns.
- chadbrewbaker: Hmm. This leads me to think that LLMs are monoids. You can efficently split the evaluation at scale and compose.
- Johann-Tobias Schäg: It can be done in the post processing step because there we have access to a likelihood distribution of tokens and we could perturb the tie breaking according to some hypersubspace which would serve as statistical proof over many tokens.
- Johann-Tobias Schäg: There are probably other approaches but doing it on the generated text least disturbs the model
- chadbrewbaker: Can you use a random source to tweak the probability distribution of the next token for parallel search?
- chadbrewbaker: They call this timescale parallelization in molecular dynamics - you apply some slight noice to all molecules then run the simulation in parallel until one of the models hits your desired phase change condition.
- juansebastianl: I think what I'm saying is best illustrated by a kind of silly example. Suppose that I wanted to set up a pre-processing watermarking. So suppose I have an existing word embedding from an unwatermaked model. You can create a new corpus of "watermarked" training examples where you sub in nearest "allowable" tokens for all the "disallowed" ones then train your model only on the "allowable" corpus and only with "allowable" tokens as inputs and outputs. Then at inference time you have to repeat this step for new text but you still end up with the statistical properties of an include/exclude list on text you're trying to determine the provenance of. Of course such a model would be terrible for a bunch of reasons, I'm simply saying it's possible.
If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!
Creators & Guests
What is Oxide and Friends?
Oxide hosts a weekly Discord show where we discuss a wide range of topics: computer history, startups, Oxide hardware bringup, and other topics du jour. These are the recordings in podcast form.
Join us live (usually Mondays at 5pm PT) https://discord.gg/gcQxNHAKCB
Subscribe to our calendar: https://sesh.fyi/api/calendar/v2/iMdFbuFRupMwuTiwvXswNU.ics
So, GPT 4, I wanted to resist this topic, but I I feel now is the time when we must we must tackle it.
Speaker 2:But how much how much, though, really, did you order this?
Speaker 1:How much did you tackle it, or how much it orders this is fair. The the the first one. This is fair. Exactly. Well, in terms of resisting it, I have tried nothing and a lot of ideas, as my favorite quick old headline.
Speaker 1:Yes. I I have tried to resist you know, I have tried to resist the prolonged discussion around so GPT 4 came out last week. One of the things they have demonstrated is writing code with GPT 4, and, people are naturally asking the question, what does this mean for software engineering? Well, actually, they're not asking that question as much. What they are asking, which I think is unfortunate, is what does this mean for software engineers, not for software engineering?
Speaker 1:It it seems to me. And there is a lot of chatter out there. There was a hacker news story, which I I guess, Adam, did you miss this thing over the weekend? Do you do you close Hacker News over the weekend? I just admire your
Speaker 3:that implies that I have it open during the week. And I I was I'm
Speaker 1:glad you thought that I thought that I thought twisting the knife. Oh my god. Okay. Look.
Speaker 3:I have it open sporadically. And and I I definitely missed this, this this Carmack tweet if this if that's what you're, if that's what you're alluding to.
Speaker 1:Okay. Look. Fine. Not only are you not at the bar, you don't even know where the bar is. Like, look.
Speaker 1:Some of us never know. Okay? Some of us, like, I just need can someone call me a cab?
Speaker 3:Look. I've got my 2 year chip. I'm very proud.
Speaker 1:Right. Josh, I think I'm talking to you. You and I are I I know Josh has only left the bar to go, to to go to the bathroom briefly. I know Josh and Josh and I are back at the bar. So
Speaker 4:For a while, I had,
Speaker 2:like, a scraper that would write down all the stories that hit the front page so that even if I wasn't there when they were on the front page, I
Speaker 1:could look at them later. So I could actually get any drinks that were served in my absence. I could come back and consume. So the in particular, John Carmack had a tweet where he had, tweeted out a DM exchange that he had with a presumed human, but may not be, about why he felt, sovereign sharing still had a future with the and what the role of sovereign sharing is. Exactly.
Speaker 1:Right. And then there was a lot of discussion package. Josh, did you I I assume you went into some of that. I actually don't think
Speaker 2:I saw this one.
Speaker 1:Oh, god.
Speaker 2:I think I've seen 8,000 other things about
Speaker 1:about There's been a there's been
Speaker 2:a lot gbt 4. Smell.
Speaker 1:Was the guy I've got gbt 4.
Speaker 2:$2,000,000 or whatever that happened investing. Like, I'm just gonna have GPT 4 invest this or whatever.
Speaker 1:Is hold on. Are you talking about Balaji had the crypto bet last week. Are we, like, crossing
Speaker 2:this picture? Mixing 2 ridiculous stories together.
Speaker 1:I feel okay. Honestly, I feel like chat gbt is writing hacker news headlines for you. Is it possible that you are creating your own? Have you used Chat gbt to write your own hacker news so you don't
Speaker 2:Honestly, Brian, I'm not even here. This is a Chat gbt to use business with a speed text speech thing. I I knew it. That's funny. My accent is so weird.
Speaker 1:But but fully automated, finally. So the and I, you know, so in particular, I just this is in a lot of people are talking about this, and a lot of people are talking about, Adam, I sent you that link to that that paper, about the, the jobs that will be, affected by large language models. And what I don't know if you got a chance to read that. I I mean, it's and all of this is still very loose, but wanted to get, your take on it, wanted to get the collective your take on it. I wanted to offer a little bit of my own, and maybe that's I I I'm not gonna be able to resist that long or I, I I editorialize a little bit myself.
Speaker 3:Yeah. Do, I have a I have a not particularly hot take. I don't know if you want me to go first to give you some space, to to give a hotter take.
Speaker 1:Yeah. If you could do, I just give me some space for a moment. It's not the option.
Speaker 3:Yeah. Yeah. Exactly. So, you you know, I've played played with chat chat gbt. I'm sure we all have.
Speaker 3:I think the thing that has been most interesting actually has been my wife's use of it at work. So my wife does shareholder activism stuff in particular around environmentalism and and CEO pay. And a lot of this time is spent groveling through these SEC filings where companies tried to obfuscate what they're doing. And she needs to, like, there's a lot of manual labor associate you with gathering the data and figuring things out and, you know, and she's been sort of testing chat g p t as an augmentation of the stuff that she and her team do. And been finding it, like, alarmingly useful.
Speaker 3:Like, useful, not, you know, not in a way that's threatening, not like in a way that she doesn't have a job, but rather in ways where they get to do, like, the interesting parts of the job. So I thought that's been, you know, pretty, pretty wild. And then my older son, who's a high schooler, has been using it to write some, like, graphics code and and finding that pretty instructive. And me, the least productive member of the family apparently has been trying to get it to write episodes of the Simpsons that I wanted to exist, but don't. And they're incredibly tedious and not funny.
Speaker 3:And I would, and feel very much like lates, late, you know, 2000, mid 2000 Simpson. So, you know, on the mark,
Speaker 1:this is going to be the reveal, you know, the Simpsons predicted everything. In fact, they had GPT 4 in 2011.
Speaker 2:Did you clarify the prompt? Like, look, no, no, write it like it's the nineties.
Speaker 3:I kept on saying like, okay, no, imagine you were on the Harvard review and, or, like, you know, the the comedy magazine. I I kept on asking it to make it funny, and it and it refused.
Speaker 1:Did you tell to imagine it that it was a gen xer, not a millennial? That maybe that maybe that would actually Yeah.
Speaker 3:I I'm not as good at, like, the hack prompting, prompt hacking.
Speaker 1:But Well, then I'm then you, my friend, have no future in software engineering because that is all we're gonna do I'm afraid is is prompt engineering.
Speaker 2:I hate I hate that we're calling it that.
Speaker 1:Prompt engineering?
Speaker 2:Yeah. I I just don't like it.
Speaker 1:Why don't you like it? Why don't you like prompt engineering? I mean
Speaker 2:My like, my kid is not doing, like, finger painting engineering in in in the kindergarten. Like, it just feels like it's not really a field.
Speaker 1:Are you are you oh, boy. I would take that up with the school then because my my children probably figure out how to use anybody.
Speaker 2:It is the Bay Area, I suppose.
Speaker 1:Yeah. Yeah. It was, it was taught right after the entrepreneurialism class at preschool.
Speaker 2:Oh my God.
Speaker 3:No, what's your, brand? What's your take on on on chat gbt and gbt more generally?
Speaker 1:So I think it's really I I mean, it's intriguing, and I think there are aspects that are really promising, but I have got a lot of questions, obviously. I think one of the, questions that that I definitely have is how it does in in areas in which I have domain expertise as opposed to where I'm kind of being where it can kinda get away with bullshitting. And so, you know, I asked you to write have you asked you to write any DITRA scripts, by the way?
Speaker 3:I have, actually. Yes.
Speaker 1:And?
Speaker 3:Did very poorly, like, either cribbed in answering a different question or like didn't work?
Speaker 1:I mean, very, very poorly is how it went for I mean, very poorly, but with absolute confidence. And it, which, you know, this is the, the, the, the, the Gaumont and, amnesia where you, this is the physicist Marie Galman. Have you ever heard of this?
Speaker 5:Type of thing.
Speaker 2:Right? Where you, like Yes. Well, like, if you know about a field and you read the newspaper, it's, like, you're horrified at how ill informed it is. And then you'll read the next article about, like, complex geopolitics and be like, oh, alright.
Speaker 1:That makes sense. Oh, that that's how it is. You know, and actually, this is the issue I have with Scientific American, actually. Great publication, but I was a Scientific American reader, and every article they had about software systems was atrocious. And and it should not have been published.
Speaker 1:And I began to raise, like, should I be I mean, and I'm pretty sure that's I'm just gonna assume for my own sanity that Scientific American was much much more accurate and that that there weren't chemists and microbiologists that felt the same thing about their chemistry and microbiology articles. But I I definitely have had that. And, certainly, with the chat gbt, when I've asked it to do anything, I mean, it was unbelievably bad. And, I mean, I think the worst thing is that it was confident. And that I think that also I mean, this is kind of the problem that I have had with this domain from from the beginning is the way it positions itself.
Speaker 1:And I mean, I think this is the danger of calling it artificial intelligence is that it it immediately puts it in a kind of competition with us. And that's really not what technology is not in a competition with us or shouldn't be. Technology really should be allowing us to, to do more collectively, and to, to do more with less, or do more with our resources. So I I feel like the way it I think as something that is going to replace software engineering, I think it is. Not only is it absurd, I don't see it necessarily on a trajectory, but I do think there's a lot of potentially interesting promise.
Speaker 1:I mean, one thing that I think would be really interesting is its ability to well, for starters, to generate documentation from code. Like, it can do we have the ability to and I think this is, like, the thing that I am really and I would love to actually ask for folks who who have who've got domain expertise and maybe can shed some light on this. But to me, the the generative stuff is relying on the fact that it's got this massive, massive, massive corpus. When the corpus gets small as it is for Detroit scripts, like, the accuracy just it it falls off because it doesn't actually know what what it does not know how to do is read the manual and write a description. That's what it
Speaker 3:does not
Speaker 1:know how to do.
Speaker 2:And the the confidence interval doesn't change, though, in the output. It is presented as cheerfully correct anyway, even though it only came from a corpus of 5 things and is totally wrong. Like
Speaker 1:Totally wrong. And I asked it, like, why do you think this is correct? And, of course, it's just very on brand. Like, it got very belligerent with me. I mean, it did not ask me to leave my wife, so I guess that was yeah.
Speaker 1:I I guess it's an improvement. But but it was only gbt 3. I guess g I guess Sydney is
Speaker 3:They do. Right.
Speaker 1:So maybe this is like I mean, can you imagine, like, every code review that ends with, like, well, maybe you should leave your wife and run away with me. It's like, okay. Wait a minute.
Speaker 2:Sometimes you go. You engineer the prompt, and sometimes
Speaker 1:it's hard to introduce you,
Speaker 2:I think.
Speaker 1:Totally. Alright. So so I and and actually, I don't know if you're interested in speaking on stage, but so Ashley in the chat is saying something very interesting about, just not being not tested if it can write docs, but it is terrible at reading them. And that so that is kind of my key question is how it and can we improve it on that axis? Because it is very important.
Speaker 1:Documentation is actually extremely important. I know people view it as a lost art, but it is actually extremely important at at several different layers of the stack. But the the the more you go down stack, the more I would say the more important. But it it the it's certainly for us at the level that we implement, documentation is exceedingly important. And it's if if it can consume a lot of documentation, that's really interesting.
Speaker 1:And if it can find because, you know, I feel I one of the things I've been thinking about, and Josh and I move flip this around, what fraction of time do you spend reading code versus writing it? If you if you think of it that way.
Speaker 2:It depends on what I'm doing. But I definitely there's a lot of reading that occurs, especially like if you didn't write the thing from, like, it depends if you're maintaining something that exists and has existed for a long time or if you're doing something new, I think. Although even in the case where you're doing something new, I think by the time you have 10,000 lines in that new thing, you you spend a lot of time rereading the 10,000 lines that you wrote and wondering, like, what clown produced them. So, like, it does reading is increasing like, increases over time, I think, with all projects and is definitely a large part of what I end up doing.
Speaker 1:Yeah. Because I feel that what I end up doing is a lot of reading and a lot of reading of code, a lot of reading of documentation, and then a lot of noodling, and a lot of I mean, we just spend a lot of time debugging. And I also don't see any evidence that this is something that it's going to be good at. I don't know how I mean, it'll be an interesting test because there is you you can't really you you can stack overflow your way to debugging a specific error message for sure. As soon as you leave the the confines of the Googleable error message though, debugging really requires an understanding of what's going on.
Speaker 1:And it's gonna be interesting. I mean, are people using chat gbt to to debug?
Speaker 4:Attempting to speak for the first time here? How am I am I coming through at all? Yeah. You're coming through. Yeah.
Speaker 4:Are righteous. Hey, everybody. Sorry. I had a little audio glitch the second I climbed on stage. So this is something I've actually spent a little bit of time trying to do is is just sort of, you know, try treating this thing as if it were my pair programmer.
Speaker 4:Right? So I'm frustrated with some things, pasting things into it and saying what's wrong with this. And at least in terms of sort of time weighted quality to answer, it is palpably superhuman. And not because it has seen the code I wrote that was buggy before, but just because, like, it kinda can think a little bit.
Speaker 1:And it might be that, like but and and I should say, by
Speaker 4:the way, I'm, like, not a professional software engineer anymore. Right? So maybe I'm losing my edge and all that good stuff. Oh, pissed off. Yeah.
Speaker 4:But, you know, like, tasting 20 lines of freshly written crap, which for me is gonna have, like, a defect density in, like, the 4 to 5 range or whatever, it usually finds those a lot faster than I would the good old fashioned way.
Speaker 3:And But Keith I
Speaker 4:thought to say
Speaker 1:keep go ahead.
Speaker 4:Keith, can you pause
Speaker 3:for a second and just talk, because I think it's very relevant. Talk about, you know, briefly about your background with with, machine learning. Oh, for sure.
Speaker 4:Yeah. Yeah. Absolutely. So, I spent the first half of my career as sort of a systems programming language person, about 10 years ago now. It was 2013.
Speaker 4:I saw a colleague, actually a undergrad, colleague of of Adam's and Brian's as well in Bloomberg, showed me, Alex Krushevsky's ImageNet competitor from 10 years ago. And this is the first thing first time I saw a computer vision model where the term computer vision didn't feel like an aspirational description. Right? It wasn't. It was making mistakes, but the mistakes it was making were more like the mistakes a toddler might make instead of them, you know, calling a motorcycle a tree or something like that.
Speaker 4:The way I was used to seeing computer vision models do it. So anyway, long story short, me and Luevomir, put it we were both working Facebook at the time, put together a little demo on a gaming laptop, marched into Zuck's office, said this stuff's gonna work. Google's 2 years ahead of us. What do you wanna do? And one thing led to another, and we ended up founding Facebook AI Research, which, you know, might end up having its enduring contribution be llama.
Speaker 4:We'll see. But, you know, the the research institution that that does the deep learning stuff, as as part of, Facebook's corporate research wing. So, and, actually, the the the examples of stuff I'm thinking of, this is probably, you know, just because these are the kinds of things I ended up pasting into it. But, I'm thinking something like writing little training loops for torch text models and things like that. If you show it that, it's, it's doing decently well.
Speaker 4:And I realized, like, where Brian, earlier you were saying your experiences with, chat gpt and trying to get it to look at d scripts and and generate d scripts for you, and it was sort of confident and wrong. I'd be super curious if you've tried that with gpt 4 because, especially with sort of long tail languages and weird, like, esoteric crap. It seems like there's a pretty big gap between 34 for me. Not that it's esoteric crap, but you know what
Speaker 1:I mean?
Speaker 3:Yeah. Also, DTrace. No. Esoteric crap and DTrace. Right.
Speaker 4:And the most important technology ever invented, DTrace. That's right.
Speaker 1:Looks good. Thank you
Speaker 3:very much. Right.
Speaker 1:Right. Diving catch.
Speaker 4:But but, you know, things that I'm passionate about that other people are perhaps, you know, that that don't necessarily have the highest page rank, like, mMix, for instance. Right? Canute's, like, you know, goofy language that he did the new version of of the art of computer programming
Speaker 1:in. Yeah.
Speaker 4:It can write mMix, whereas gpt 3 kinda couldn't.
Speaker 1:Oh, that's interesting. Yeah. I was gonna ask about m x. Okay. Yeah.
Speaker 1:That I mean, which is interesting. I mean, I am gonna be curious to ask gpt4. I actually, wanted to, but it was down today. I actually wanted them like, alright. You know what?
Speaker 1:I'm actually gonna pay up and get on gpt4 and kinda retry some of the stuff I tried. It's like, no. We're actually we're having a good old fashioned outage. Ilo, I, Boom. Yeah.
Speaker 2:Yeah. Exactly.
Speaker 1:I don't know if it was an ultra rare error or not. But, yeah. That's a and so, Keith, so so you said that just to to make sure I'm mentioning correctly that when you had a corpus a body of code that had a had a defect in it, you felt it was finding that defect faster than you would find it just by staring at it. Is that
Speaker 4:Oh, like, no no question at all.
Speaker 1:No question.
Speaker 4:I was I was dealing with something that, like, didn't work for one reason that I couldn't find. It found that reason and 4 other things that were broken. That weren't what I was actually troubled with at the time.
Speaker 1:Oh, wait. So that is interesting. So one thing I think it would be really cool would be GPT based code review, I think would be, code review is something that, I I it'd be interesting to know what fraction of engineers engage in code review. You'd want it to be a 100%, but I think it's a lot less than that. And, I think everyone would say I don't have sufficient time for code review, and I feel like code review is is not I think you could be very helpful to get GPT based code review.
Speaker 1:I also think that you would get the the it would be interesting to see if you would get some of those, you know, the a great code reviewers are able to kind of fit an entire system in certainly, a common code review comment that I have for folks is, like, I don't understand. I I know that you understand this. I don't understand it. Can you have a better comment here explaining it? And it would be that I think would be really interesting.
Speaker 4:Yeah. It seems decently effective with that. And I and I wouldn't say it's amazing at it, but it's decently effective. It's like definitely inferring intentions from variable names and things like that. You know, things that that you wouldn't get from sort of a syntax based, thing mobob.
Speaker 4:For what it's worth, like, the you know, Alex Gravely, the the copilot guy, is an old friend of mine from VMware who's, I worked with him 20 years ago. We kinda tried to stay in touch. And I think the vision for, like, where he wanted Copilot to go and he's he's left GitHub and is doing other stuff now. But I think the vision for where he wanted us to go was much more like a shared canvas with a weird cybernetic pair programmer. Right?
Speaker 4:With with that copilot analogy. That it would be sort of as if you were always pair programming, you know, with your your review buddy or something like that, which I think is is pretty interesting.
Speaker 1:I think that's really interesting. Yeah. And I also think that if, phrased it that way, the okay. So oh, actually, sorry. So someone in the chat does have access to chat gpt4.
Speaker 1:The the question I asked, and I had a I had a post on Masson about this, is, I wanted to understand, I I think processes executed during boot was what I asked it, and that is what it just absolutely detonated on. But it and, you know, we've got on chat g p d four. So the, Chad Chad Brewbaker is asking it to, how do you write a process tree with DTrace? And it's not right, but it's not totally wrong.
Speaker 2:Is it less not
Speaker 1:less not right? It's not very helpful. I mean, the it it it it's it's, like, pretty wrong. I mean, it it it has a d script that is, like, within and but, again, I think that it's gonna that that's gonna be, I think, something of an extraordinary example because there's not a lot of the it is, some would say weird and esoteric. Or what were your exact words, Keith?
Speaker 1:I said, fringe. I I I can't remember. Exactly what what high praise you had for it.
Speaker 2:Quint.
Speaker 1:Quint. I'm
Speaker 4:a niche.
Speaker 1:Exactly. Do you you said you you find
Speaker 2:it like pair programming. Do you when you the part that I have a hard time with I think about that is it feels like like if I'm pair programming with a person, that person has a consciousness and persist like object persistence and and like I can have a conversation with them for more than whatever the token limit is or whatever.
Speaker 1:I I
Speaker 2:don't know. It just it feels like it feels like it is good at presenting the illusion of that conversation because of what it is, like Mhmm. Like a linguistic response constructor, but it's not clear to me at least that it really it doesn't have a a mental model of what's occurring, really. Yeah.
Speaker 1:Oh. The right. That's my question too. Yeah.
Speaker 4:Yeah. So, I mean, there's a there's an interesting kind of, like, scientific question here. So, like, 2 years ago, I think, if you, had described to me the kind of thing chat gpt is, I just said, like, okay. Cool. That's a that's an interesting scientific experiment or whatever.
Speaker 4:People should try to scale up, a a language modeler. Right? And language modeling is predict the next token. That's, like, how the problem is framed. That's what a language model is.
Speaker 4:You should try scaling that up and seeing how good it can ever get. That's interesting. I don't think I or anybody, in, like, the mainstream of, you know, cognitive science or something would have expected it to be as empirically effective as GPT seems to be at, like, elucidating some level of higher order structure from this instead of just producing plausible seeming tax. Right?
Speaker 1:Okay. And then This gets into question, Pete. Yeah. Because this is my inference too is that actually there are aspects of this that are pretty surprising even to people with a with a great deal of domain expertise. One of the things that I heard that I'd love to bounce off of you is that the the degree that it changes its answers when you tell it to think like someone else is one of the things that I'd heard from a fellow technologist was that if you if you tell it that it's a domain expert, and then you ask it about something in that area of domain expertise, that the hallucinations go way down.
Speaker 4:Oh, you're right. And sometimes by name. Right? Like, you you know, debug my, you know, ray tracing code as if you're John Carmack. It improves performance or whatever.
Speaker 4:Right?
Speaker 1:Really? Yeah. It it is that under I mean, that's weird. Is that understood? Why why that's effective?
Speaker 4:I mean, I think you can kinda make a plausible reverse engineering of it now. Right? You can say that, okay, it's learned by reading everything that humans have written in the world, and it's seen lots of examples of mediocre debugging of ray tracing code. And it's seen a handful of examples of debugging of ray tracing code by John Carmack, and the latter were much more effective. And, like, I and I'm kind of I I I don't mean to sound silly about this, but, like Yeah.
Speaker 1:I don't know. Yeah.
Speaker 4:That is the level of which it's operating, and that's and I I think you you actually touched on something really fascinating here, which is that, like, this is the kind of weird effectiveness of big language models is one of those unusual phenomena where, like, the more you've been paying attention, the more surprising it is, if that makes any sense.
Speaker 1:Like Yeah. Interesting. This is one
Speaker 4:of those things where the people who seem to know the most about the mechanics of, like, what this gizmo is and how it came to be, find it the most weird.
Speaker 1:Right. That's always interesting. When you got they got to go away, but why are you so surprised by this? It's like you're supposed to act that's interesting. And well, in in because I this is the other question that I've got is, you know, we have kind of had this aspiration of building this this this biomimetic system that we're gonna kind of, we're gonna mimic our our biology.
Speaker 1:But there's all sorts of things about our biology that we actually don't like, and we have spent lots of society's scarce resources trying to figure out. And it's like, do we want I mean, we certainly don't wanna have the equivalent of dementia in our or in our model. I don't think. Mhmm. And if this emerging behavior is surprising, I mean, it shows you that we that we are actually gonna you you wonder if it's going to be a become a scientific discipline to simply study these things that were created because we don't actually understand them.
Speaker 4:I think that's, there there's stuff that's de facto like that. Right? So if you look at what Anthropic is doing, to the extent that Anthropic is, like, differentiated from other industrial AI research labs. On the outside of the box, it sort of says safety. But if you kind of peek the lid up a little bit, their reverse engineering is what they're actually doing.
Speaker 4:Right? They're actually sort of trying to discover, okay, how do these things represent information? How do they change as they learn? How do how do they compute? Right?
Speaker 4:How do what are the limitations and sort of superpowers and how they seem to compute? And the weirdness of just the fact that LAMA, which is, like, the the Facebook large language model whose late weights leaked, over the last week, but the 7,000,000,000 version of it right? It's 7,000,000,000 flows. That's, like, what it is. That's the information content of this thing.
Speaker 4:It's 7,000,000,000, you know, approximations to real numbers, and it has a lot of factual knowledge in it. You know what I mean? It, like, knows what the capitals of all the nation states in the world are and knows lots when lots of heads of state were born and things like that. There's at least, like, compression stuff to say about that, for instance, that that we don't fully understand, but, like, Chris Ollie at Anthropic has some interesting things to say about that seem like holograph you know, an analogy to holography is involved in some way. So, yeah, I think it's fair to say these things have actual emergent properties.
Speaker 4:And, like, the biomimetic part of it, like, I mean, this this thing is much stranger than a virtual brain, if that makes any sense.
Speaker 1:Yeah. No. Yeah. That's right. Yeah.
Speaker 4:Like, still, like, doing a virtual brain would, in some ways, like, produce much more, much less puzzling behavior would be my guess. But, like, I think I think to get about, like, the this transformer architecture that they're having all the success with is that, like, it doesn't read that your text in order. Right? It's a your text becomes a gigantic cube that it consumes in an instant and, like, represents in one gigantic feedforward computation. And that produces some sort of very strange and complex, you know, open quotes, mental state, close quote of this thing that makes it want to continue with a certain sequence of tokens.
Speaker 4:But, you know, we know that's very different from how humans process language and nobody cares because it's empirically really effective.
Speaker 1:Okay. So then do you think that we are doing ourselves a disservice by latching onto this as artificial intelligence? I mean, we are obviously people are obviously end we're flexing this thing. Jesus Christ. The and do you think that is part of our maturation with this technology going to be to adopt a mental model that is understanding what it is and what it isn't?
Speaker 1:Do you understand what it's good for and what it's not? Is that gonna be an important step for us to be able to figure out how to actually use it most effectively?
Speaker 4:Hugely. Yeah. I think and I think that's one of the things that, reading between the lines a little bit and I have no special inside information here, by the way, but reading between the lines a little bit about, like, why chat gpt was released in the way that it was, where it was sort of initially released with a weaker model even though we now know they had gpt 4 already internally. Yeah. Interesting.
Speaker 4:I think that that was partly to get us used you know, to get this kind of conversation warmed up a little bit, about the fact that this thing is not omnipotent. And even though it's palpably superhuman in some ways, it is subhuman in other ways. And it's just a different kind of it's almost like contact with alien civilization or something like that, and it's gonna take us a while to really figure out, the safe and and sensible ways to harness this all.
Speaker 1:Yeah. I mean, it's it's also I mean, it's scary not in the technical sense, but it is scary the way that technology can be scary, which is how is this technology going to be abused by other humans to abuse yet more humans? It is definitely, it do mean it it feels I mean, Keith, honestly, it feels likely to me that we will see at least one religion that is worshiping the weights, worshiping the model.
Speaker 3:That's a great prediction.
Speaker 4:Interesting. Yeah. I like that.
Speaker 2:I mean, didn't Nat Friedman mumble something about Pat it being a god in a tweet Yeah.
Speaker 1:Today? I I I mean, why would you not think it? I mean, it's like, I don't know. It's like, I feel, people
Speaker 4:People forcibly steal your things.
Speaker 1:That's exactly right. People forcibly steal your things. This seems like I don't know. It seems like a reasonable thing as soon as you go.
Speaker 4:It's not a patch of mold that looks like the Virgin Mary. Right? This is, you know, this is at least sort of a strange otherworldly presence.
Speaker 1:Which when it's asking you to, like, leave your wife and because only it can love you is definitely, troubling to say the least. So this is a Okay. Brian, I need
Speaker 3:to talk about context. Yeah. What is
Speaker 2:this coming from? Coming from. Yeah.
Speaker 3:Yeah. Are these are these are these artificial intelligences hitting on you? Is that what
Speaker 2:was this a Bing thing that happened?
Speaker 1:This is the Bing thing. Like, you guys are making me sound like I'm a problem drunk, and maybe I am, but, like, surely you heard about this. You don't hear about this? This is the you. Yeah.
Speaker 1:Right now. This is Kevin. No. Okay. God.
Speaker 1:Kevin Ruse from the New York Times, got got into a conversation with Bing, GPT 4 based Bing, that got super super weird. And it is and it it encouraged him to leave its wife his wife. Bing had revealed itself to be Sydney. It it said that only it could love him, that his wife actually didn't understand him and didn't know how to love him, but Sydney could love him. I mean, it was it was it was wild.
Speaker 1:And it, like
Speaker 2:Some of the press coverage around the big thing, like, it was, unhinged.
Speaker 1:It was unhinged, and I would really recommend listening. I mean, if you're I don't Adam, do you know do you listen to Hard Fork at all? Hard Fork is very is really very good.
Speaker 3:I haven't heard this. No.
Speaker 1:Oh, hard fork is really outstanding. Casey Newton and Kevin Roos, and then but they they've got a bunch of their folks, and, Zoe, I can't remember her last name, but the the the really terrific, the crew at, the platformer. And and and and, actually, I wanna get you on stage here because, you know, I mean, you're you're making a very good point that this did not come out of nowhere. And it it had to be almost be coaxed out of it. I mean, he did kind of, like, try to get to its, you know, like, what do you really think?
Speaker 1:What are your deepest thoughts? And but once the door was open, the door came kinda flying open. And I also wanted to know where the nuclear codes were. I wanted to get to the nuclear codes. It's like, is this from what what what have you inhaled here, g p t 4?
Speaker 1:It was But does it know that it's a list
Speaker 2:of this crime that means
Speaker 4:that just mean that it's red, you know, script? Yes. It's it's red.
Speaker 3:It's red.
Speaker 1:It's red. It's red.
Speaker 4:It's red. It's red. It's red. It's red. Paranoid.
Speaker 4:It's written about how evil AGI is going to behave. Right? Right. So if you kind of tell it it's evil AGI, it's gonna act like evil AGI. And if you tell it it's John Carmack, it's gonna act like John Carmack and debug your, you know, MMX code or whatever.
Speaker 4:Right?
Speaker 1:Well, in speaking of I don't know if you're in the chat, Keith, and I don't know if this is if if Chad Brubaker I don't know if you're I mean, I get the sense that this is legit. He repeated his query around the process tree with DTrace saying, now as if you're Brian Cantrell, how would you write the script? Excellent. And oh, it it just turned out one
Speaker 2:of your blog articles at that point?
Speaker 1:No. But it is it is definitely well, I'm obviously gonna go figure out where it got this from, but it is it is way, way closer to the answer. Like, so much closer to the answer that I can't tell immediately. It's not.
Speaker 2:This might be copyright infringement?
Speaker 1:The well, it it is remarkable that it and again, maybe I am maybe I'm doing the same thing. I'm just kinda, like, falling for it to a certain degree. And what it's doing is actually not the but I I think that, you know, even if to Ashley's point in the chat that, like, he definitely Kevin Russo definitely coax this out of of Bing, out of Sydney. It didn't make him, I think, any less disturbed as a result. I mean, it was and it it it would be hard not to anthropomize.
Speaker 1:So yeah. Adam, Josh, I did not make this up. It's definitely
Speaker 2:the state. I did that. I believe you.
Speaker 1:I I do I do think that without
Speaker 2:that conflict, I will note that it is a it's a difficult skepticism to maintain if you even had it in the 1st place. Like, because the thing the thing is exactly the right shade of whatever it is that makes it that that draws us in, I think. And a lot of people have got been drawn in with substantially less complex models already and and embarrassed themselves and so on in public like that guy at Google. But, like,
Speaker 1:Yeah. That tribal liberated or whatever. Right. Which,
Speaker 2:I mean, if, you know, if you've asked the AI whether it thinks that's a good idea first, like, because that's I mean, it's read all the stories too.
Speaker 1:Either right. It knows the stories.
Speaker 6:Yes.
Speaker 1:Well, and I think that you and actually you at this point about the you're much more worried about the people who believe that chat gbt is thinking rather than it actually thinking. And I'm I'm worried about that too. And the question is like, how can we ground this thing a little bit more to actually be more clearly a tool and and not as much I I do think, Keith, maybe this is out of line, but it feels like that they've we've kind of emphasized the parlor trick aspect of this. And I think it doesn't arguably, does it a disservice.
Speaker 4:There's an element of this that, like so I think there are 2 kinda concrete, you know, hits based on GPT so far. 1 is Copilot, and one is chat gpt. And they're both products. Right? They both have, like, significant decisions made about what kind of experience you're gonna have using them.
Speaker 4:They both have, like, other auxiliary bits of machine learning and data driven behavior and stuff that are shaping how they act. And and it there are kind of questions to ask about the decisions made in the chat gpt product that I think are distinct from the decision, like, the the broader question, of, like, what is this model, what can it do, what does it know and not know and stuff. And I think there are real hazards in sort of an engagement only optimization metric for this thing. Right? Like, if you kind of just tried to make something whose goal is to you probably will find something that, like, engages people to their detriment.
Speaker 4:Right? Like, I I think you would kind of end up inventing cocaine at least for a subset of people. Yeah. But I wanna say, like, the the an interesting thing about this whole,
Speaker 2:like, leaving my wife Sorry.
Speaker 4:Oh, you're gonna have
Speaker 1:to give just give me a moment on the dystopian future that you just outlined, where we are taking kind of the worst of the last decade in terms of how we engage people and enrage them, and combining it with these large language models to get you this thing that is like fully automating it. Fully automating it and, like like, basically demanding that you leave your wife and then showing you an ad. So let's
Speaker 4:take the let's take the leave your wife example for a second, but, like, I think this is actually interesting. It's like a a thing about the Gestalt that's bigger than AI and bigger than chat gpt or whatever is. I feel like if you pluck sort of a mid century a mid 20th century American out of the crowd and describe some weird future where there are these, like, strange magic rocks that tell you things. And sometimes the things that tells you are right and sometimes they're wrong. Right?
Speaker 4:And you say to this this person, you know, sometimes it tells you useful things, sometimes it tells you harmful things. And, you know, what are some of the risks that that you'd expect from this thing? And they're like, well, what if it tells me something plausible and I act on it? It's wrong and so on and so forth. If if they'd have a set of rational but the concern that The Rock would tell you to kill your family or leave your wife or kill yourself or something like that or, you know, detonate a nuclear device or something like that.
Speaker 4:I think they just say, well, I just wouldn't do that. I would throw the bad rock away because the rock is wrong, and I have values and principles that are not up for debate. Right? They're not up for argument with, like, magic rocks or other people or anybody. They're just what I believe and what I know to be true and what I know to be right.
Speaker 4:And it's interesting to me that, like, we have, surrendered that perspective totally, if that makes any sense. Right? But, like, the the idea that, and and maybe we're right to it. Right? Like, maybe everybody is just way more susceptible and way more convincible than than that mid century American would have imagined.
Speaker 4:But it is sort of something about, like, the traditional worldview that feels like could be handy in this moment is to say, like,
Speaker 6:hey.
Speaker 4:If the magic rock tells you to, like, kill your family, it's a bad magic rock, and don't do it. You know?
Speaker 1:Yeah. I think if you went back to that person from the past and said, there's something else you should know about the future. Donald Trump is president in the future. They'd be like, what the what are you how did that happen? It's like, well, the magic rocks are involved, actually.
Speaker 1:The magic rocks are not completely uninvolved. I and I mean, I do think that, like, having seen not I mean, not to ask not to pick on Trump, but the, I think in the last 6 years have felt like a pretty dark chapter from the perspective of the Internet, or the last decade, let's say. Is what, you know, I think in Keith, you know, I came up roughly at the same time when there was a lot of optimism around the Internet. And then we spent a lot of time, you know, building businesses around the Internet. And then we saw a real dark chapter, I feel, in the last decade.
Speaker 1:And I feel like that may be fundamentally what is what is causing some pessimism is that people have their optimism pretty nuked. I mean, I mean, if you were to even go back to my 3 year ago self, and tell me it's like that the that there is this unbelievable vaccine breakthrough in terms of mRNA based vaccines that is then rejected by a significant fraction of the population, you'd be like, oh, god. And I I think that that is part of the I think that that's part of the concern is that we are concerned about the number of people that will listen to the magic books. No. But as long as you bring up the the historical question though, I did a question that I'm kind of and as I'm trying to, like, think back on previous revolutions, and there's always this, you know, fear cells, unfortunately.
Speaker 1:And we people, I think, at some level, we biologically like to be afraid or we need to be afraid. And, you know, we've seen this historically where people get fear gets really high, and sometimes that's justified and sometimes it's not. I went back and read, Ed Yordon's books before Y two ks and they are talk about unhinged. Do you remember Ed Yordan by the way?
Speaker 4:Only faintly. Yeah. Yeah. It was one of the the doomsayers about Y2K and everything.
Speaker 1:And he one of the concerns about Y2K the other thing that he was a big doomsayer about and this this came out right when I was deciding to study computer science. I had I was deciding to decide to study computer science not because of its economic possibilities for me, but actually despite them. I I was because I had discovered I love computer science. I I think that that's true for a lot of us. I don't think that's true for it.
Speaker 4:And by the way, Brandon, I think we're among the very last, like, generation of people for whom it was a subculture. Right? For whom it was like, yeah. Maybe you can make, like, a small town dentist living doing this or whatever. But mostly these are people who love computing or doing it.
Speaker 1:I think that that is prob I think it it is it is harder to sort that out now, because there's so much economic opportunity associated with it. And it has but but there's been economic opportunities. I mean, when I was coming up, it was to be you would go to law school if you in, you know, in the eighties, you would go to law school if you want. And then it was that turned, and I the folks that I know who went to law school because of its economic opportunity, and not because of any affection for the law, became, didn't last very long as lawyers. Because it didn't it just didn't carry them very far.
Speaker 1:And, of course, things economically shifted as well. But I I think that the you know, in terms of of why I got but when we came up, Keith, the Ed Yarden had a book, The Decline and Fall of the American Programmer, written in 1992.
Speaker 4:Oh, right. Right. Right. The outsourcing thing? Yes.
Speaker 1:And describing how, America was gonna be overrun, and and all software was gonna move to to, India and to Japan. And, like, that's just it's all wrong, basically. And I I think that the thing that I I think is really a high order bit that I don't wanna lose in this is I am concerned that there are exist young people that are not going to study computer science because they believe that chat gpt has denied it it has made it impossible for anyone to succeed as a software engineer.
Speaker 2:I think importantly though in that point you make, like, it is true that software didn't move to another country, but a lot of people were economically displaced as part of outsourcing and and downsizing in the nineties, I feel like.
Speaker 1:For sure, but they weren't software engineers.
Speaker 2:Some of them were. Right? I mean, like, I feel like I feel like I IBM has been laying off to, you know, at a rate of thousands of people a year for 20 years. Right? Like, if you average that.
Speaker 1:But I think that the the but not saw I mean, the the folks that actually cut code for a living have done very well. And if they haven't done well, it's by choice. And we we we have got unbounded economic opportunity or to have for the last 25 years.
Speaker 4:Well, I think both
Speaker 3:are true. Right? We we have lots of folks hiring in lower seemingly lower cost or repeatedly lower cost areas. And some of the cases that's like a direct displacement, but there's plenty of jobs or, you know, except for in the last, you know, 6 months and 6% cuts across the board. Lots of jobs that have offset that by far.
Speaker 1:Totally. And Jordan completely missed, and even you did have a book. And I remember reading it thinking like, wow, I this is this is right? This just feels very, this does not feel true to me, but it felt like as a young person, I did not have other voices to kind of to to counterbalance that, and I I am concerned about the there being this kind of dominant zeitgeist that discourages people from pursuing something that they otherwise like. You should not go into computer science if you do not like it.
Speaker 1:I mean, if you if if you hate it, you should go to something else. But if you do like it, you shouldn't let someone tell you that there's not a future. And this is true for, by the way, whatever you pursue. Don't let people tell you that there's not a future on it, because it is really hard to predict the future as it turns out.
Speaker 4:And I will certainly say, like, Brian, even if they're I I don't mean to interrupt your train of thought here, but, like, thinking about sort of my like, a bullish estimate of what Sage at GPT can do for my productivity as a programmer today. And and let's say, like, the kind of stuff I'm doing is the kind of stuff it excels at. Right? I am I'm gluing together different APIs, and I'm doing so in JavaScript. And there's lots of, you know, evidence based out there and the corporate's trained on that can really make me faster.
Speaker 4:It's probably it's probably something like a 50% power up. Right? Like, the kind of productivity boost that, like, you'd be foolish to ignore. It's a significant thing. But, if I were to think sort of more broadly over the kind of time period we're talking about over the span of, say, 2 or 3 decades, you know, what has, what is the area under the curve of, like, all the programmer productivity improvements?
Speaker 4:Right? So, you know, languages that are arguably more productivity oriented, not having to manually manage memory, you know, higher performing machines, whatever. I let's go a few decades back. Right? Let's go back to sort of the seventies when it was considered a a a feat of absolute absolute mastery to be able to write a chess program.
Speaker 4:Right? That was considered like, you were really, you know, well beyond a mere apprentice programmer if you could make a computer, play a game of chess.
Speaker 1:And there is no course, really, in the seventies. So it's like you are if you're writing a test program, it's like because you wrote it. Like, you did not get any portion of that off of GitHub.
Speaker 4:And you probably I mean, and there wasn't an Internet to speak of, and so you probably learned it yourself and so on. Or, like, you know, you probably didn't have a real community to learn and yada yada. So I think overall, like, there's probably something like a 100 x. It's certainly more than 10 x, like, the the the average programmer's productivity is greater now than it was in the seventies. And there's a ton more programming jobs than there were in the seventies because of induced demand.
Speaker 4:But, like, the more software we make, the more software we need to fit all the software. And I don't think that, like, GPT is the straw that breaks that back by any stretch, at least not yet.
Speaker 1:Yes. And in fact, you you're getting to something else that I wanted to bring up, which is Jevons paradox. So I don't know if you you I don't know if you're familiar with Jevons paradox, but the Scottish economist who observed that when we became much more efficient with respect to using energy, we that energy use went up. It's like, wait a minute. We got much more efficient.
Speaker 1:Why are we using more of it? It's like, well, because we're becoming so much more efficient that we are finding new places to use it that we could not economic we use it before. So we actually need to we're using more of it. And I feel that that's certainly what we have seen we in the last decades with with I think you're right. I mean, it's at least a 100 x, I feel.
Speaker 1:And, but as a result, like, now there's a bunch more stuff that we can go do that we just couldn't do before. We just did not have a way of doing. It is like, you know, think, like, when I was a kid, like, how did I coordinate without a cell phone? You're like, oh, right. We just didn't.
Speaker 1:It was just bad. I just spent a lot of time, like, waiting to be picked up places and miscommunicating with people and leaving notes for people. It was just like it was just bad.
Speaker 2:And, I I you know, with that Sometimes it was quiet.
Speaker 1:True. But you can this is you need to turn off your bot that's feeding you fake hacker news articles, Josh.
Speaker 4:Actually, I You know, it's almost impossible these days since to feel truly bored, by the way. I mean, I saw, like, rare that state is and how precious it is to actually achieve.
Speaker 1:Oh, yeah. Bored was great.
Speaker 4:I mean, these old fashioned, like, I'm stuck waiting for a bus and, like, didn't bring a book and didn't bring a newspaper
Speaker 1:and That's right. Yeah. Exactly. Or
Speaker 2:my special ADHD powered hell is when hacker news is suddenly infinite scroll and, like, and then, like, I've read the the 10 or or whatever that are on the page, but then like the computer is like, well, I'll show you the next 10. All right, now I'm just gonna start making them up and I'm not gonna tell you like which ones are
Speaker 5:made up. And I'm
Speaker 2:just gonna let you scroll literally until you die from scrolling.
Speaker 1:Well, actually, I do think that this is part of, of Keith's dystopia about where these LOMs are modified, are, are optimized for engagement. It is frightening because like, look, Josh, I think I can speak for both of us that like engaging us is not exactly rocket science. Like, it is not hard to figure out. 1 one
Speaker 2:one need only listen to one episode of this show, I think.
Speaker 1:I mean, exactly. You understand that. Write
Speaker 2:oh, you you guys you
Speaker 3:can write something in scheme to to troll you for a couple of days. Absolutely.
Speaker 1:Oh, Exactly. You could troll me into I I mean, I troll myself into a stupor all the time. Like, I don't even need anyone. I don't even need some sort of fancy large large language model to do it. So it it does get a little bit alarming when you think about it, but you also wonder and Keith, do you think we've hit like, peak engagement at some level where people are more cognizant of how they're I mean, I don't know.
Speaker 1:The the, I I don't know if we are are are gonna be gonna have better, immune responses to to how that's being engaged, how we're being engaged. Yeah.
Speaker 4:I mean, it's tough to predict, but, like, there's a, we certainly, like, live in a world where there is now status associated with, like, being resistant to those kinds of forces. Right? The kind of, like, younger especially, like, I'm I spend a lot of time with young founders these days because I've, left honest work to interventure capital. And, like, those folks, like, a lot of them, kinda follow these extremely, austere kind of lifestyles. Right?
Speaker 4:So they're, like, gonna have a rigid morning routine and a crazy, like, workout regime and weird dietary rules and, like, you know, have to jump into a cold bath for 30 minutes a day or whatever because they're gonna, like, live forever and be geniuses if they do all this stuff. And, certainly part of that is, like, resisting all sort of, like, dopaminergic engagement loops. Like, part of that is, like, turning your phone off and avoiding all that stuff and so on and so forth. So there's at least, like, social status associated with resisting these things now, which is, you know, perhaps the first sign that, that they're on their way out. But who knows?
Speaker 4:Maybe that's just a fad.
Speaker 1:Yeah. So one of the things I'm definitely curious about, and and we mentioned at the top, and, actually, you I I think I think you're on stage here. Maybe you could you speak to what you've done with documentation? Because that I'm really curious about because that that is something that that, if we can somehow use this stuff to write better documentation, to read documentation, to that that's really interesting.
Speaker 7:Yeah. So, I mean, I guess the first thing I'll say is, like, I'm not, like, an AI developer or anything. I just build dev tools. But, it is worth noting that if you wanna build with chat gpt, it's it's stupid easy, like, terrifyingly easy. And I say terrifyingly just because I think the mesmerizing effect that it has on people, and the fact that people could deploy it so easily, like, is a little dangerous, I think.
Speaker 7:But yeah. I spent 1st, I took a naive model, and I was using the turbo 35 because that's one that's available on the API. And I started with just using prompts, And then, eventually, I went to trying to train it a little bit more specifically on some documentation. But, yeah. I mean, it hallucinates a whole bunch of things, that makes it really tricky.
Speaker 7:Because if you wanna use tools, you need to know what features those tools have. But kind of I thought it was like a hilarious product manager kind of thing, because it's sort of like hallucinated features that you would want that tool to have, but it may or may not have.
Speaker 1:I feel like you're describing your coworkers. Like, listen, This coworker has got a serious drug problem, and they're often they're often high at work. I mean, like, high high as a kite. But it's like, okay.
Speaker 3:Strong recommend, great hire.
Speaker 1:Right? The
Speaker 7:thing that I think is really interesting, and I think that, like, chat gpt can potentially help us alongside documentation. But there's this this weird kind of, like, ouroboros situation. Right? Where, like, the AI is, like, really only as useful as its training. And the training is the documentation.
Speaker 7:And so, like, if we decide that, like, everybody's job is over and, like, all the code's gonna get written by the AI, like, we kinda have a, like, I am sitting in a room situation where we just have the same data, like, going in and out over and over and over again till, I don't know, the harmony of the Internet plays. But, like, we we still need that content, and, like, that's kinda how we, like, escape that loop. And so I don't think that we can have AI, like, own that from, like, start to finish because it's gonna have to be the thing that we feed it with.
Speaker 1:Totally. And and I think that we and this is why I do think we wanna start we really wanna be thinking of it as a tool to help us write the the docs that we would you can't think it's like this is gonna write all my documentation for me. But it it it's more that it boy, this thing can consume the documentation over here and and then consume and and help me synthesize examples, for example. I mean, like, if we if if if GPT only serves to provide people with much many more correct examples for how to use something, that'd be great. The question is, like, the hallucination problem does sound like it's real.
Speaker 1:Have you did you try GPT 4, by the way, or have you do you just spin this all on 35 because that's what you had access to?
Speaker 7:Yeah. I didn't have I couldn't figure out a way to programmatically use 4. Uh-huh. Because I was like, there's like a cool Rust library if folks wanna do it. Like, spinning up like a Rust CLI that, like, does the basic chat GPT thing is, like, incredibly easy to
Speaker 1:And have you used, did you use it to to generate any Rust? Because that's the other thing I have not played around with at all. I'd be kinda curious how it does at
Speaker 7:Rust. Yeah. I mean, I think one of the other things, and this goes back to me saying that, like, developing content for this stuff is super important. It's real it's not good at stuff that's new, and it's not good at stuff that not a lot of people do or people aren't using quite yet. And so, like, a thing that I'm particularly interested in, and, like, if I need to pivot my startup to an AI startup, this is what I'd wanna do, is, like, we need to be able to train models really specifically on, like, the new things that are coming out, and we have, like, self published, I think, those things.
Speaker 7:Because a lot of times, and at least the things that I like doing as a software developer, are doing, like, the hard things that, like, no one's done any like, before. Like, when I worked at Cloudflare, they were like, make Rust generated web assembly work on workers. Like, an AI is not gonna be able to do that. I'm gonna have to write the docs so that an API can learn how to. And so
Speaker 1:Yeah. And this is a really important point, actually, because this is a lot of what we do in software engineering is we create new things. Like, we are not and, you know, I think we are often bad at estimating how long it takes us to do something, because we are doing something. That if you are doing something that is the estimates get very good if you're writing the same software over and over again, which we generally aren't doing.
Speaker 7:Yeah. And I guess I would call out I don't know if folks know Julia Evans, but she makes these, like, amazing programming zines, but she has this one on being a wizard programmer, and just like, literally, like, the quote is like, oh, this is why they pay me, a human, to do this. And it's about, like, when literally no one knows how to do it, like, it's your job to figure it out. And so, like, I do think it's possible that, like, AI will, like, help that happen. But at least at least my understanding of how large language models work is they're specifically, like, not new.
Speaker 7:They are, like, predicting what would already happen. Like and so if we wanna, like, make any sort of progress, and that's, like, a complicated word for the Heidegger fans in the chat, which I would advise if there were any.
Speaker 1:Exactly. It's it's more than 1. Smaller.
Speaker 7:But, yeah, like, there we still need to, like, make stuff. Yeah. So I don't know.
Speaker 2:I It doesn't feel like it completely synthesizes new concepts or new directions so much, really.
Speaker 7:It doesn't at all, at least as far as I can tell. It's like a really great detector. So, like, when I talk about designing developer tools, right, I think ergonomics and familiarity are, like, very, very close. And nerds get really mad at me when I say that, but but it's like it feels easier to use because you're used to using something like this. Chat gpg is perfect for this because it's gonna try and use your thing the way it tries to use everything.
Speaker 7:So it's like a great weirdness budget detector. Oh,
Speaker 1:that's interesting.
Speaker 3:That it's a great way of phrasing it. Yeah.
Speaker 1:Yeah. That's really interesting. I'm, now I'm worried that the fact that I can't write DTrace, maybe that's a reflection on us. Right. To keep going.
Speaker 1:That has been weird.
Speaker 7:You, actually. You should have made it more ergonomic.
Speaker 1:That's right. That's really interesting. That a a wowing and kind of giving it some of the tasks that you expect, someone using your thing to be able to do, and then kinda watch the mistakes that it makes as, like, okay. This is an indicator that the ergonomics are wrong because the large language model is making the wrong prediction. That's really interesting.
Speaker 7:I mean, it's kind of cool to think that, like, you can think of JATCPT as being tray trained on, like, every single ding dong that's ever made a comment on Stack Overflow ever. And, like, wanna know, like, do people in Stack Overflow, will they be able to figure out how to use this? Like yeah. I mean, if it's struggling, then you might wanna consider modeling your API on other ones that people are more familiar with.
Speaker 1:I also have to say that, like, if if the the the net consequence for sovereign shares is that we no longer go to Stack Overflow ever, I I feel that that's a win. I don't care what replaces it. I don't care. I I I I don't know. I don't know.
Speaker 1:I don't know how could you go to Stack Overflow. I do not like myself from going
Speaker 4:to Stack Overflow. No. I'm really
Speaker 7:optimistic about things. I think not only will there be more people than ever will be programmers, and they will be writing much, much more interesting things. Like, we're writing everyone's writing, like, super boring, like, I don't know, like, so much glue code. It's boring. Like, wouldn't it be great if we didn't have to write as much of that?
Speaker 7:Like, I I see it a lot more like a power tool and, you know, carpenters didn't go away because power tools exist.
Speaker 3:That's right. I think if we, if we can focus on the more interesting parts as the example that I, that I kicked it off with in a totally different domain. Yeah. Totally agreed. Hey, Flavius.
Speaker 3:You said you had a comment in this domain. Yeah.
Speaker 8:So, the I see the ergonomics aspect of it as more that chat gpt provides you a way to unsee, to, like, not be able to understand what's going on, which allows you to function as if something works when it doesn't work, which feels good, but doesn't necessarily actually work. So it like, it's in a Heidegger sense it gives the illusion of something being ready to hand when it's actually present at hand or unready to hand. And
Speaker 2:the
Speaker 8:yeah. So that's that's like how programmers experience it. And then the flip side of it, the the side where one of the reasons I think, like, a lot of bosses or or owners are really excited about it, It's the Hegel connection where, like, if you if you look at the, the Lord Bondman dialectic kind of way of looking at things. It's like the, the worker works, the Lord tells the worker what to do, but the worker is the one who actually builds up expertise and in a high degree in sense, the worker is the only one who actually, who actually is able to interact with equipment. Like, they're the only one who, can use this stuff as a, like, as phenomenologically transparent.
Speaker 8:So, you know, when I'm when I'm programming and and everything is working, it's like it just goes straight from my soul out onto, like, onto the desk. And, even when I encounter problems, if I'm encountering a problem at the level of of the task I'm trying to solve, it's like I still, I still get to experience this, this incredible process of creation. Right? But if I if I was a boss, if I like, even, even when you've just become a technical manager, right, you stop being able to write code all day and you have to, like, tell people what to do. You have to manage people rather than just writing code.
Speaker 8:And you can't treat people as phenomenologically transparent. Right? Like, they're they're beings in the world. So so they get to do that, but you get to tell them what to do. Right?
Speaker 8:And this, like, ability to unsee, like, it's it's built on top of a whole lot of hyper abstracted labor. So you, you know, it it what it promises the the owners or the the, you know, the the lords in this scenario is the ability to effortless create because all of the effort is coming from sort of hyper abstracted labor that's, you know, the the corpus that these things have been trained on, and then all of the sort of whitewashed or invisiblized, like, mechanical Turks who are in various little, small contracting companies around the world, actually.
Speaker 1:Yeah. The unseen folks that are actually labeling and everything else. Yeah. Yeah. Yeah.
Speaker 1:Yeah.
Speaker 8:Yeah. So it it gives them the feeling of that phenomenological transparency, without having to actually become a worker. And I think that's kind of what the what the owners want out of it.
Speaker 2:I do think that property of it where a lot of people perceive the value in being, I think, that they don't need to know anything to get stuff done anymore. Like, it's gonna be this machine that just does stuff, and I don't have to deal with any of the pesky people I used to have to deal with to get things that were complicated done.
Speaker 1:Yeah. I do know it's like it you yeah. That like
Speaker 2:and I think
Speaker 1:I think also tracking BG is telling me this can be done in an afternoon. Why are why are you perky's saying it's gonna take you a week? This I I I think the
Speaker 2:Well, but things like that. But then, like, I think it also has turned a lot of people who were probably copyright minimalists to begin with, are suddenly now copyright maximists, because I don't think anyone really anticipated that one company would be able to absorb everything everybody had written before and then take the name off the top of the paper basically and allow other people to use it. Like everything, not one thing, but everything. Like, it
Speaker 1:it is definitely true
Speaker 2:that we are in all directions. Like
Speaker 1:And we need this it needed open source to to have that kind of corpus of code.
Speaker 2:Yes. Because it's not fucking trained on Windows. That's for sure. Like, I mean
Speaker 1:I do think it's interesting. So, did and, Chad Brubaker in the chat has, asked chat gbt 4 to do a code review of a PR that I've got up. Now I'd like to point out is still it's actually I think it's no longer a draft PR even though I'm really thinking of it as draft PR. And it's interesting. I mean, it has given me a a code review comment, where it has found an unwrap, and it's like, well, this this should not be an unwrap.
Speaker 1:And you're not familiar with Rust. This is basically, where I'm you are effectively asserting that this is something, and you're unwrapping that something. But it's a very facile comment that it's offering me because that particular unwrap, it can't fail, that particular unwrap for other reasons because of where we are. And I do wonder, I'd now I really wanna add a comment above it explaining that the unwrap can't fail and see if it still offers the code review. Because it's like, its suggestion is wrong.
Speaker 1:I mean, its suggestion is like, oh, you should change this to be like a a much more ornate panic. It's like, no. No. You definitely shouldn't because your state is corrupt. If there's no there is no way to do that.
Speaker 3:Yeah. Chad GPT, just just don't feel bad. This is what Brian says when I code review his code too. He's he's this is my comments as as minimal and my suggestions as irrelevant. So
Speaker 1:Thanks, Will. We'll get caught up
Speaker 2:all at you and
Speaker 1:That's right. Well, listen, I if you would if you not hallucinate so frequently when you could review me, firm with you.
Speaker 3:No. No. It's everybody else. It's not you.
Speaker 1:I agree. Exactly. Thank you. No. No.
Speaker 1:But I think it's really interesting. Well, so no. I'm gonna really wanted to because, like, a reasonable, the we if one saw that, it's not necessarily an unreasonable comment of, like, hey. Could this unwrap fail? And it can't.
Speaker 1:But so I would I again, I wanna, like, add the code comment there and see if it no longer makes the comment, with the comment explaining why the unwrap is safe.
Speaker 2:It's a very pedestrian comment though, and you do have to it's like, have you made this comment first because you've seen this comment made so many other times in blog posts and so on around Oh, but you you don't
Speaker 1:I saw it and
Speaker 2:then I saw a paragraph of explanatory text begging people not to use unwrap that way.
Speaker 3:That is, you know, that's a really interesting point, Josh, because because I think we're not good at code review. Like, large large, you know, writ large.
Speaker 4:We have a whole show
Speaker 3:on this topic.
Speaker 1:What do
Speaker 3:I do? I think
Speaker 2:if if if I unlock the same mutex twice in the same block, is it gonna tell me that I did that wrong? Yeah.
Speaker 1:Yeah. Yeah. Yeah. That'd be interesting.
Speaker 2:Is it gonna, like, lock ordering? Is it gonna understand that? I struggle to to believe that it can even reliably generate, like, complex rust on mass anyway because the constraints like, English is pretty forgiving.
Speaker 6:Previously, someone brought I previously, someone brought up that the, JetGPG or whatever, basically turns your text into an hypercube and specs that once, looks what's next next token, next best token, pushes out the last most token, then it generates the next token. What's the up thing about the structure is that it's basically a finite state machine, nothing else. A finite state machine without registers, with acting, which exposes a quite interesting and obvious problem. It cannot deal with, it cannot deal it can never deal with, nested parenthesis it didn't see in training. So, computer science, the languages which require a stack to pass.
Speaker 6:It can only do those if it learned to embed the fact that there is a stack inside its finite state machine. Basically, gets a hyperdimensional state with, like, all the parts it has. And then it basically walks the stack through that fasting, but it can't learn from you, branded languages, which it didn't encounter before, which is quite interesting to say because it just hallucinates forward.
Speaker 1:Yeah. And so that's I mean, that's definitely true. I mean, I'm just again, I think it is. I'm really looking forward to getting, like, GPT 4 out because I'm just looking at this code review comment. Again, like, re reading the code review comment it had closely.
Speaker 1:And on on the one hand, the okay. Yeah. A bit of a fast thought comment. And, again, like, its suggestion wouldn't be it it it its suggestion is very mindful of the context of the broader code in a really impressive I mean, I'm overall impressed with its comment. And it is not I mean, its comment.
Speaker 1:Compiles correctly. It is it has generated an error message that would be on point if it's possible that I mean, it's pretty it's interesting. And so I I would I think we're gonna have to give it a shot on a lot of this stuff before we just assume they can't learn anything new, because it definitely has, as Keith was saying earlier, there is there is some emergent behavior that even the experts don't understand that could be interesting. And I think we're gonna wanna treat it seriously, but not actually, I I I don't think we're gonna need to to be worried about it replacing us. I think it's actually and I loved the the kind of point you had about allowing a lot more people to develop software, because it does feel like it's gonna make this a lot more accessible.
Speaker 1:Do you wanna talk about some of the the work that you've done with kids in terms of what you think? Because I think actually, like, my daughter would I like, Roblox just hadn't rolled out the, some, AI assisted stuff today. She would love that. She's 10. She's gonna love making stuff with that.
Speaker 7:I think you're tagging me in for my work with kids. I guess I have done some work with kids, but the vast majority of the education work I've done is with adults. And Young
Speaker 1:and hot.
Speaker 7:Sure. Of course. Yes. Absolutely. Actually, most of them are very, very old at heart and desperately trying to get out of jobs that they really didn't like doing because they wanted to do something, like, exciting.
Speaker 7:And so, like, the things that I've seen beginners get tripped up on in programming are never the, like, high end CS concepts. Because, I mean, if you're actually writing an application, the chances that you're running into those in a meaningful way is already, like, pretty small. But it's like navigating the file system. It's It's like, how do I make sure the computer knows that the file is here? Like, it's it's typos.
Speaker 7:It's it's really, really basic stuff. And so, like, the general ideas that are behind a lot of applications are things that people have a lot of intuition for. I always love telling people, like, students have an intuition for types, and then we teach them out of it by, like, teaching them JavaScript or Python first. I so many students were, like, how do I tell it what type of thing it is? And I'm, like, you don't.
Speaker 7:It guesses later. And they're, like, what? I'm, like, yep. But, yeah. I mean, I think a lot of people know what they want computers to do for them and we just really haven't given them great interfaces for doing that.
Speaker 7:And,
Speaker 1:I I actually, are you a GPT maximalist? I mean, it sounds like the the you're this is I I mean, I love it. It's great.
Speaker 7:Love the outcome. Positive, but that's also because maybe it's because I studied, like, a ton of neuroscience. Like, my first job, I actually accidentally worked under a DARPA grant writing, like, encoding neural networks in Java under the auspices of this, like, really ridiculous, like, teach kids math using their own perception thing. Anyways, there's weird weird labs that have been doing stuff like this for a while, like, in corners of universities, and I stumbled into one, when I was, in college. But, I don't know.
Speaker 7:I think that they are very, very interesting tools, but they're not the if we're worried about stuff like religions and stuff, it's not the LLMs that we have to worry about. It's everybody's reaction to them. Very similar to the banking crisis thing. I think the thing to worry about is everybody's reaction. And so I also might be, like, pulling a little manifestation perhaps in that, like, I think if we are excited and optimistic and see the benefits here, we have the opportunity to build a future where we live harmoniously alongside it.
Speaker 7:And then there's other options too, but I like those less. So I wanna build the, like, really positive situation. I
Speaker 4:need I think
Speaker 1:that's cool.
Speaker 7:So Oh
Speaker 1:my god. I love that. No. I love that. I really do because I I feel that I mean, I do feel like optimism is something that's been, it's been sprained.
Speaker 7:As a human in the world. So anybody who actually knows who I am is probably surprised to hear this. But
Speaker 1:But I but I think, like, we can be both. Right? And so, like, we can be realistic and also and and certainly be cognizant of the perils of a technology. But I think that that fundamental optimism is really, really important because it it just I I think that it does it there's so much that is going to be made possible by this. That would be just to to Keith's point a little while ago about that kind of a 100 x multiplier.
Speaker 1:And, you know, I I think of, like, the number of people that have that have been in search of, like, a boy, I need a technical cofounder. Like, I've got an idea, but I don't know how to, like, manifest that at all. And I've always been, like, you should actually literally learn the code. Like, you don't actually need a technical cofounder. You need, like, a book on Python and some patience and some real resolve on this idea.
Speaker 1:But, boy, this is gonna make that a lot easier when people can actually get jump started. And I I it just feels like it if we think of it optimistically, I think we're much more likely to think of it as a tool. Is that a is that a fair summary of what you just said, Ashley?
Speaker 7:I mean, I definitely think so. I think it's also worth noting, like, people often respond really defensively to change. I don't like talk about human nature a lot, but it's certainly in my experience that people really respond defensively to it. But if I were to, like, evaluate how I feel about software right now, like, in general, I would say that the vast majority of people are suffering just, like, absolutely terrible software that is not customized for what they need to do. And it it's, like, a terrible experience for them.
Speaker 7:And I think that there's an opportunity for us to, like like, think of it like plug in ecosystems, but, like, much better. Totally. And, like, I I think that that could potentially be, like, really fantastic. And again, yes, it's, like, 100% seeing Chad GPT as a tool. Now, I mean, there's a lot of interesting, like, market economics around this, like, with, like, the rise of these massive centralized SaaS's, like, came because of the giant mismatch between supply and demand of software developers.
Speaker 7:And, like, there's been people who have been afraid of expanding the pool of software developers for a really long time. Somebody in the chat, like, asked after I made a comment, like, oh, there are people who are, like, afraid of boot camps. Yes. They're absolutely thought that literally, like, boot camps were coming for their jobs. Like, software developers are extreme I mean, I don't think it's unique in professions, but I happen to be very close to it.
Speaker 7:But they're really defensive of, like, this, like, guarded knowledge that they have. And so I do think that we're seeing some of this here too is like, oh. And this is what I see when people are like, oh, look at that answer. Like, that's so technically wrong. Like, I'm gonna code review the code review.
Speaker 7:Oh, that code is bad. I'm just like, most people can't write any code. This is an opportunity to get so many more people writing code, and I've always thought that the more people we have writing code from the most diverse backgrounds possible, it means that we're gonna end up with, like, the coolest, most interesting software. And we have, like, boring software right now because there's only this tiny margin of people writing it. So, yeah, I'm excited.
Speaker 1:Yeah. That's interesting. Well and then maybe in in in kinda that worldview, it's actually a little bit better that this thing is not writing perfect software because it writes you something that, like, kinda works but kinda doesn't. And then as a result, like, you've gotta actually understand that the thing that it wrote, because you're gonna need to figure out why it's wrong, or help work with it to find out why it's wrong, and as a result, get some understanding. So it's not I think that the the fear that people have is we don't want the software to become this kind of opaque box where people don't understand at all how it works.
Speaker 1:But, of course, like, we already don't I mean, I
Speaker 7:I was gonna say that's most software. This is, like, the housekeeping source. Like, people are like, oh, it's different source. I'm like, oh, yeah, you went and read that repo and you know how that code works. No.
Speaker 7:There's already so much code out there. It's all a black box. Like, it doesn't matter if it's available. That's the joke. Like, everything like, the level of complexity of the information held in our software now is so many more times greater than it was at the, like, beginning of open source that, like, literally people stopped caring about the software being available because it was simply too much to understand.
Speaker 7:So, like, there's already all these black boxes all over the place. I think you have to just decide which layer you wanna pay attention to.
Speaker 1:Totally. Yeah. Because I I mean, as I even as I was saying that, I'm like, wait a minute. We consume these APIs without understanding what's on what's on the other end of that API. People don't understand they don't and they expect to understand these lower layers of abstraction, and and how, like, how this is actually turning into instructions in the machine, and how those instructions are turning into operations on the bus, and how how the operations of the bus are turning into fields.
Speaker 1:I mean, it's, like, we actually don't need to understand all that. We actually can have layers of abstraction that allow us to build stuff. That's actually it's actually okay. It's actually good. It's how we actually are allowed to do more with less.
Speaker 7:Yeah. I mean, it's really funny that programmers are just, like, literal abstraction machines and they are, like, super threatened by any layers of abstraction on top of what they happen to be doing. And, like, I see chatgpt and, like, it writing code as, like, a layer above, like, I don't know. It's kind of like a it's like Webflow, and I didn't see a lot of people get freaked out about that. So
Speaker 1:What is Webflow?
Speaker 7:Webflow is if you wanna make, like, one of those fancy JavaScript pages that's, like, really slow because it loads too much JavaScript, but you don't wanna write any of that JavaScript.
Speaker 1:Right.
Speaker 3:Does that help you, Brian?
Speaker 1:Yeah. That helps. That helps.
Speaker 7:Honestly, Webflow isn't really hard.
Speaker 1:Briefly, what if it what if
Speaker 2:it wasn't shoddy, though? Like, what if the output was actually fantastic?
Speaker 1:That'd be great.
Speaker 7:No. I mean, I I really shouldn't have ragged on Webflow. I'm just laughing because I can tell what sites are made with Webflow because, like, everything's animated. I'm just like, okay. This is made in Webflow.
Speaker 7:Like, the output's not bad. I think we have lots of tools today that, like, make great output. I mean, also, like, I don't know. Just look like look at WordPress.
Speaker 4:I don't
Speaker 7:know. Developers also love to hate WordPress.
Speaker 1:So But I think in terms of the of of the the degree to which it it lowers the barrier. I mean, just like I mean, blogging did lower the barrier for people to write online. I mean, the and we have over and over again seen that that has created that just creates more. We're able to do more with human potential when we when we lower the barriers. And so, actually, one of the questions I was asking the chat is, like, do you think software's boring because there aren't enough people who can write it, or because of the economic incentives, and and the needs of the people who can?
Speaker 1:I think it's
Speaker 7:Why? Why not both?
Speaker 1:I'll go so why not both? Yeah. Exactly.
Speaker 7:Do I have to pick? Those are both reasons why, like, those are 2, like, affordances that I think do cause some issues. I don't know. I think in some ways, the financial incentives, like, the economic incentives. I don't know.
Speaker 7:Design loves constraints. Right? So I think I think constraints can be very interesting in, like, the types of things that we end up building. But, yeah, I don't know. The the Internet is, like, a massive, like, SAT solver for, like, all of human everything, and I definitely think it would we'd see cooler, more interesting software if we had more people writing it.
Speaker 7:At the very least, we could have people who have some domain knowledge writing the, like, rails generate for, like, their domain, and it won't make really horrible, just, like, embarrassing mistakes. I think that'd be cool. Like, if doctors wrote doctor software, we'd probably have better doctor software.
Speaker 1:Well, I think also that we we've seen if, I mean, if history is any guide where these also technologies that that tend to have a a a broader base that appeal to many more people tend to bring just more people in. They tend to get more people excited about it. And we will I mean, there's an argument to be made that that, GPT based software will ultimately result in more people who are writing assembly because there'll be people who are introduced to software that way and begin to, I mean, just by lowering that barrier and getting more people excited about actually creating things. Yeah. The the actually your optimism is infectious.
Speaker 1:I'm I'm I'm waiting.
Speaker 7:So glad. I felt like we were getting to a little dark place in New Zealand.
Speaker 1:I think we are. I think we're getting a little dark.
Speaker 7:And I was like a little nervous about that because I do genuinely think there's some positive things about it and, I think one of the things that we haven't chatted about that just lets me talk about something that I, like, really am excited about is, like, a lot of people like to say, like, also my dog is having, like, a tantrum. I apologize.
Speaker 1:But the dog's trying to get into the dog is actually, like, I know I can go write software now. I know exactly this software. I can try
Speaker 7:to write some apps for sure.
Speaker 1:That's right.
Speaker 7:But, like, a lot of people are like, oh, well, the software that it writes isn't accurate. Right? Like, it's not perfect. It has bugs and stuff. And I think one of, like, if we had to talk about, like, the original sin of computing and, like, Richard Feynman actually just writes this in general, but, like, when we think about trying to make computers act like humans, it was that we focused on building machines that were, like, a 100% precise, but it's not really what most people need.
Speaker 7:Like, if you look at everything in the world, like, analog style, like, nothing is a 100% correct, even, like, super smart computer geniuses like me. Just kidding. But, like, we we don't need that perfection, and so I think what's going to be interesting, and I'm curious to see how this plays out on, like, a systems level, is, like, what does it look like when we see the proliferation of, like, not quite, like, great software? Or is this gonna mean, like, does it not matter that it's not quite great because we're not building these massive centralized general machines and people are making all these little tiny specific machines for themselves?
Speaker 1:So I think that the the well, so first of all, I I do think that there are, the reason that we are able to build the stuff we have built is because the foundations actually do work reliably. So I think there is actually software. Actually, I would say that where actually it correctness is the constraint. And I I mean, I don't know how wild we are about, for example, GPT based a a GPT committer to to, LLVM might make me a little bit nervous. Might make a lot of people a little bit nervous, because we actually do really require, correctness there.
Speaker 1:We we require correctness. Correctness is a constraint. I do think that you and I think that part of what I love about Rust is that it actually, broadens that the amount of software where correctness really is a constraint. And I think that that is actually great. At the same time, I yeah.
Speaker 1:Right. And I okay.
Speaker 7:You're getting roasted in the comments, Brian.
Speaker 1:No. I know. I know. Well, I I I was I but, I mean, there is, like, you you actually you actually do rely on your operating. I mean, like, there was a time when your operating system would actually just die, and it actually made the the technology when it's not a reliable substrate, it actually is a problem.
Speaker 1:And we you do need that to be reliable, and we actually do need to understand when it fails. But the but the that software being ultra reliable allows you to build a bunch of the stuff up stack that where you you don't need I I I think Yeah. More being more approximate is fine.
Speaker 7:But so I think this is why the threat from chat gpt is, like, something that folks are more worried about than, say, like, the example I used before, which is Webflow, is because chat g chat gpt at least appears to be able to do this on all layers of abstraction. Whereas, like, previously, a lot of no code and low code tools tend to be, closer to, like, the human than the metal, I guess. And so it's like, what would Webflow look like for an operating system?
Speaker 2:And also just very rigid in their construction, generally, I feel like.
Speaker 7:Oh, you mean, like, some I don't know.
Speaker 2:A lot of no code tools end up, like like, you can tell that it came from a particular no code tool, I think, because the output is very similar and and like the swim lane that you're in when you're using those tools is often because someone had to write all of that stuff,
Speaker 7:you know, all all
Speaker 2:of the configurable bits and all of the flexible bits. Like, and there's just only so much complexity you can manage in a software product.
Speaker 1:I feel like
Speaker 2:it had that it often has that kind of limitation.
Speaker 7:I do sort of wonder, like, do we think that chat gpt will have a tell? And if so, like, what will it be? Like, will we be able to be like, oh, I can tell that this was from chat gpt. I wonder if that's possible. Kind of like, I don't know, like It's the
Speaker 1:the fake street thing. Right?
Speaker 7:You see. What was that?
Speaker 2:The the fake street thing. When people were producing street directories in atlases and so on, they would put fake streets into the maps so that Oh, to
Speaker 1:figure out who who who
Speaker 2:point out in court. Well, obviously, you know, Boysenberry Street is not there. That's that's you stole that from our map. Like
Speaker 1:So, Josh, are you gonna start naming? Are your variables gonna get very idiosyncratic names? So you already
Speaker 2:have idiosyncratic names. Clippy is always complaining.
Speaker 1:That's because Clippy wants to actually throw you into the, the chat gpt wood chipper. It actually needs it's gonna make for, for terrible fuel for our GPT overlords if it has this idiosyncratic variable naming.
Speaker 2:I don't I don't have time to type more than one character or variable. I'm competing with AI. Come on.
Speaker 1:Well, so and I'm curious and then, Jacob, during the chat, I would love to get your perspective and maybe, on just terms of students and GPT, because it, it sounds like you are currently a student, and what how is GPT kinda changing your relationship with software or the relationship of of your classmates?
Speaker 5:Yeah. Awesome. I'd love to chip in. It's both, I think, awe inspiring and frightening. I definitely have had some conversations both with people, job recruiters and stuff like that where I feel like it's almost a warning that I have to give along with my classmates where it's like, I don't really know whether it's going to be a huge positive or a huge negative.
Speaker 5:I have some friends who rely on ChattGPT to do almost everything in their schoolwork just because they can and there's no cost to it.
Speaker 1:And so so you are an undergraduate or graduate? What do where are you kind of in education?
Speaker 5:Undergraduate, I'm in computer engineering, of British Columbia. Okay. And, yeah, I'm in, like, 2nd, 3rd ish year. And so, yeah, seeing people learning like, you know, learning how to write assembly per se and relying on to like debug like arm 64 output or to, you know, solve stack overflow problems or, you know, for writing Malek and stuff like that, are things that I've like seen people try and like wield this really not spec for tool to try and solve. Sometimes the results are good, but sometimes I think, like, I had some conversations with people in the chat here, like, you've people spend a bunch more time just trying to get chat gpt to solve it than figuring it out themselves.
Speaker 1:Right. It would actually would
Speaker 6:have been you would have
Speaker 1:been actually faster to just write this yourself. Interesting. And what, what was how, do your professors react to, GPT assisted work? What what is there a policy, or how does that work?
Speaker 5:I I think right now, the best is that, like, most of them, probably from a coding side, can mostly rely on the fact that most of the time it's just wrong. But like there definitely is, I think like a lack of wanting to really search for like ways to weed this stuff out. Like, because the prompt generation is so varied that, like, you know, you can query these things multiple times and get it to give different answers. Even if like you're on, I think, chat GPT's website and you can like move the slider, not for chat GPT per se, but some of the other models, for like how random it wants to give the output. There isn't like a widely deployed tool for them to use to, like, pick up chat gbt.
Speaker 5:Like, how would you? Sometimes these questions are super narrowly scoped. And so, like, it's really hard to argue what is what would or wouldn't really be chat gbt output. So that is kind of scary in a sense. But, yeah, I mean, I view it a lot of the times mostly as, like, a worrying amount of, like, how much are we understanding these things If you're kind of go to is something that's now going to, like, be responsible for understanding, you know, your code or what you want you to do.
Speaker 5:If you're not, like, evaluating and understanding the semantics, I think that that's that's the real downside that comes from, like, chat gbt. If it's especially when we're being asked to do a lot of things that are very, like, web searchable, at this stage.
Speaker 1:Right. I mean, it does kind of it it requires, assignments to be, I mean, you can't just use the same assignment or uses an assignment with another. But it does this isn't hasn't that been true for a while that stuff has been Googleable? I mean, clearly, this is, it's kind of a next level Internet searching at some level. Do you are people actually trying to when they're trying to debug their code, are they sending that at the GPT and having GPT find it for them?
Speaker 5:Definitely. Like, I see like, I've seen people
Speaker 4:put,
Speaker 5:like, error messages in into chat gpt. Like, I mean, it comes to all forms of just not understanding, which I think is, again, one of the things where it can have its upsides and its downsides. Like, I've seen people in lectures take, like, notes that they've gotten from the professor, like, when a professor doesn't say something, if they don't understand it, they hand it to chat GPT and it, like, they'll try and see if they can understand it better from that standpoint.
Speaker 1:That's I mean, it's kinda like how that's that's seems, like, useful to have a a TA that's basically with you at any given moment.
Speaker 5:Yeah. And, like, on the upside, it is really good, at doing those things in in a broad sense. And wielded wisely, it can be one of these really useful things. And I think, like, for me in my own sense, it's really hard to, you know, draw the line where, you know, when you know that there's better tools out there, how do you do and walk, you know, a path such that you learn the things properly rather than just relying on chat gpt to, you know, do your whole degree at, like, the worst, you know, nth degree. Because like a lot of things, you can skip out on a lot of learning if you know that there's fancier tools out there, but you will be like left hung out to dry if you don't have a greater actual understanding of the system.
Speaker 5:You know you know, to, you know, play up our tee up you guys and like, you know, in the field of debugging, you know, having proper understanding of the system and being able to like, you know, query these things. Sure. There could be, you know, a great GPT interface to, you know, something like DTrace where it helps you build the scripts. But at some level, you need to understand the application of what you're doing. And I my only fear as a student is that, like, I'll have classmates who aren't learning anymore in that sense, who can get so far with chat gpt that, like, if for some of these, you know, for some people, it's like, if you're getting a 60, like, what's, you know, what's the problem?
Speaker 5:Yeah. If it's a very 60, like, what's, you know, what's
Speaker 1:the problem? Yeah. If it's of any consolation, undergraduates sliding by with the least possible work is not a new phenomenon. And, so that that is something that just degrees. That's right.
Speaker 1:That, that is something that is as old as time. But the, so so we said that is your only fear. I mean, is it, for you and kind of for your generationally, does it feel net optimistic? Or, I mean, do you kinda share Ashley's optimism that, like, hey, this is just gonna allow me to do more. This is basically a good thing.
Speaker 5:That's a good question. I haven't pulled enough of my classmates on that sense. I think the most people scope initially is, like, this is so handy for insert homework assignment. I don't need people like broad scope. There definitely isn't like, you know, I've seen some, you know, doomism over, like, you know, will is is your job pointless or whatever?
Speaker 5:And like, I think obviously right now that that's mostly a joke. And I think that like at current stance, even with the understanding of like what large language models are, like, we're not yet building systems that are actually intent to replace us. It's really incredible that we've built these things that, like, the only scope and the only design criteria was to make something that's, like, good at talking to you. And Yeah. I mere coincidence, we also accidentally made a robot programmer and, you know, a robot lawyer.
Speaker 5:And not, it doesn't do those things well, but it's just a weird coincidence, you know, not really, you know, but that it, that it does these things. And so maybe if the scope was like seeing these things, seeing people really try to bring to market things that, you know, aimed to be the AI programmer and like full scope, that's what it was trying to do. Okay. Maybe there would be fear, but I think mostly it is, you know, a optimism that these things will be used, you know, in a way that makes writing code easier, writing code more helpful. You know, things like GitHub Copilot have been like I find
Speaker 1:And then what about the desire to actually understand how these things work from in terms of, like, the actual underlying technology? Does it I does it kinda wet the appetite in that regard? I I imagine the the GPT the the the the the large language model class or equivalent must be one that people are very interested in.
Speaker 5:Oh, yeah. Definitely. I mean, there's there's definitely a large opening demand in terms of, yeah, the AI course offerings, that have been given even, yeah, in, like, the tangential, like, engineering branch. There's been a push for these, these courses and let people really wanna dig in to like to using these things. But I don't know how well people have, like, a good depth of understanding of, like, you know, what these things are.
Speaker 5:And I think, again, that's kind of one of those things that, like, I've had lots of conversations with friends with is, like, you know, you you see people dismiss GPT when they go like, oh, well, yeah, but it gave me buggy code. It's like the interesting thing isn't that, you know, it was, it failed at its task of making code generation. It's this incredible thing that wasn't made to do this and look what it can do. I still think that we're like struggling for like general comprehension of what these things are. There definitely is a push to like get more into this space.
Speaker 5:But I mean, from what I see, like in specifically in my degree of like computer engineering, It's definitely a lot of, like, web fear of hardware, you know, wanting to go to cloud stuff right now. There hasn't been a huge enough directed uptick, I think, at at AI or, like, machine learning.
Speaker 1:Interesting. Well, that is that's great to hear from in terms of, from the from the trenches. I'm glad that that there's a sense of optimism. Honestly, it makes the domain feel exciting. I and I think that it feels like there's a lot of potential out there.
Speaker 2:What do you think about so my my my biggest fear is not that this is going to become something that is indispensable or very good, because I think that it will become both of those things given enough time and investment as most most labor saving devices do over time, right? But like, what about the fact that we're all sharecropping on one specific company's implementation? Like, is is there gonna be
Speaker 1:I think that that I am a I'm a little bit less concerned about just because I think that it what they have done, others can are gonna be able to do. This is not gonna be the only information I don't think. I I I I've got I've got confidence on that. I think that that if there there again, if history is any guide, I just don't think this is gonna be the only limitation.
Speaker 7:Yeah. 1st mover advantage is rarely an advantage. I think there's already, like, a whole bunch of folks building things and building things more specifically, I think, that'll end up being more more helpful.
Speaker 1:Yeah. I think so too. When and I I mean, I hope they are, because I think that that's what's actually gonna make it, really I I think that will allow for a lot more utility, frankly. And I I mean There
Speaker 7:is one way this could go sideways, though, which is, like, I think really possible. And this is why I feel like paying attention to, like, the legal side of these things is really interesting because there's a whole bunch of people getting really spicy about copyright right now. And it's possible that training chat gpt the way it was trained may not be accessible, allowed, legal, etcetera, like, in the future. And I know that there's f like, lobbying efforts from OpenAI, to kind of do that. So that's scary.
Speaker 7:I don't love that.
Speaker 1:I don't I don't love that. I I
Speaker 7:also Data providence.
Speaker 1:Yeah. I don't love that. I also I mean, again, it kinda depends on on the specifics of it, but I also feel that that by getting what I would not love I don't I don't love the idea of 1 company kind of having access to that. But certainly speaking from my own, the code that I've open sourced, I would love for as many things to train on themselves on that as possible because I I that's the point of open sourcing it. It is and I and I'm not hugely worried about that being a copyright violation.
Speaker 1:Other copyright holders, I'm sure, will this
Speaker 7:work. See, I wanna take it a step further. Like, I don't know. Maybe I'm starting an AI company now. I have no idea.
Speaker 7:But, like, instead of you saying that you hope a lot of people train on your code, I really would love to see a universe where, like, you train the model for, like, your particular thing. Like, I don't know. Maybe I'm just, like, inventing dynamic linking all over again, but I really like the idea of, like, a lot of specific models that can, like, talk to each other as opposed to all these big massive ones.
Speaker 1:Yes. I wanna drop these models in on, like I mean, speaking, like, just very directly, I wanted to be I wanted it to train on our RFPs, the request for discussion we have inside of oxide. And I would love it to find things that are inconsistencies in the things that we have done. Find the consist find the kind of the the consistent narratives. There's so much so many questions I would wanna ask someone who had been able to read and retain all of our design documentation.
Speaker 1:Oh my gosh.
Speaker 7:Well, I started doing this on issue trackers and GitHub discussions. Like, that was part of the my like, the end part of my experiment over the weekend. Yeah. I think that there's some, like, really cool stuff there. Like, I think yeah.
Speaker 7:I make I'm genuinely excited about that. Like, that's the thing I wanna see people build. I think once we all, like, can become publishers, then the fear about there being, like, one giant publisher also goes away.
Speaker 1:I that's exactly it. That's exactly it. And I think that is actually that's a great note to end on. This has been a really this has been a great conversation. I actually this is I feel like Adam, did you feel like this is another one of our stage dives?
Speaker 3:Totally. I had no idea which way it was gonna go, but this has been terrific. You know, thanks to Ashley and to Keith and to others for joining. It's been great.
Speaker 1:Yeah. Josh and Jacob and and, Yohan made a bunch of folks that were on the stage. It was great. Really interesting discussion, And I and I mean, I I obviously, something that a lot of people are thinking about. And I have to say, I love the fact that that the the disposition, I think, is very net positive, which is great.
Speaker 1:But it's it's a very positive conversation, and I gave I I feel like there's a lot to look forward to, and a little bit to fear, but mainly a lot to look forward to. Alright. On that note, thanks everyone. See you next time.