Subscribe
Share
Share
Embed
Chaos Lever examines emerging trends and new technology for the enterprise and beyond. Hosts Ned Bellavance and Chris Hayner examine the tech landscape through a skeptical lens based on over 40 combined years in the industry. Are we all doomed? Yes. Will the apocalypse be streamed on TikTok? Probably. Does Joni still love Chachi? Decidedly not.
Ned: Other than your mic stand, everything else okay [laugh]?
Chris: Yeah. As usual, with the things that I observe, this is something that I figured out, like, years ago but never put a name on.
Ned: Mmm, okay.
Chris: So, let’s start with the obvious. Coke is clearly better than Pepsi. It’s not a conversation. Put your finger back down.
Ned: [laugh]. Okay? Wha—
Chris: No, no, no. Don’t you okay me.
Ned: [sputters in the background].
Chris: Hey.
Ned: Diet Pepsi is better than Diet Coke.
Chris: First of all, they’re both bad.
Ned: Fair.
Chris: The only decent diet soda on Earth is Diet Dr. Pepper. There I said it.
Ned: Ohh, I agree that it’s good. Moving on, what was your actual point?
Chris: I don’t remember.
Ned: [laugh].
Chris: The other thing about Diet Dr. Pepper is it’s got two-and-a-half times the caffeine, so this episode is going to go fast.
Ned: [laugh]. Hello, alleged human, and welcome to the Chaos Lever podcast. My name is Ned and I’m definitely not a robot. I am a real human person who has feelings, dreams, and an ordinate desire to consume caffeine and sugar. Like, all the sugar. Give me all the sugars that you have. I will suck them out of your blood, thank you very much. With me is Chris, who is also full of delicious sugar. Hi, Chris.
Chris: One has to assume that vampires have an opinion.
Ned: An opinion on blood types? Blood sugar levels?
Chris: Yeah. So, think about beef, right? We know there’s a difference between corn-fed and grain-fed, right?
Ned: [laugh]. I believe you mean grass-fed, but moving on.
Chris: Is grass not a grain?
Ned: Oh, I thought you said brain-fed, and I was like, not aware of any cows eating brains. Isn’t that how we got the mad cow disease?
Chris: Yes.
Ned: It is. Okay, so yes, corn-fed versus grain-fed versus grass-fed, there is a definitive difference.
Chris: Right. So, if you were eating either human flesh or drinking human blood—or both I’m not here to shame anyone’s kink—you’re going to be interested in that person’s, let’s call them the donor’s diet.
Ned: Absolutely.
Chris: That’s why—I’m almost positive this is correct—on the website for the Red Cross, they say you should do five or six shots of whiskey before you donate, so that, you know, when somebody gets your blood, it’s a little pick me up.
Ned: [laugh]. Settle down. I was wondering where you’re going with that. And it turned out it wasn’t anywhere good. But hey, speaking of consuming entirely too much stuff, guess what I did this week?
Chris: Went somewhere that wasn’t good?
Ned: Oh, I went somewhere that was fine, but I consumed entirely too much of a thing, and that thing was AI. Wooo.
Chris: Yay.
Ned: Ugh. So, this past week, I attended the AWS Summit in our nation’s capitol, and home to a really good taco place. Seriously, it’s called Rebel Taco. It’s on the corner of 6th and K. It’s a less than five-minute walk from the Walter E. Washington Convention Center.
So, if you happen to be at that convention center: Rebel Taco. Walk the, like, four blocks, it’s worth it. I got three different tacos. They were all delicious, especially the lamb birria. That was really good. And the margarita I got to go with my tacos was absolutely not fucking around. Ohh, somebody had a very heavy hand at Rebel Taco, and I didn’t mind one bit.
Chris: Unrelated, I saw something interesting slash terrifying.
Ned: Okay.
Chris: The new summer drink is a pickled juice-based margarita.
Ned: Thanks, I hate it.
Chris: Anyway, enjoy your weekend.
Ned: So, when I was 15—of course I have a story for this, right?—when I was 15, I worked at a pizzeria. And I also had acne because I was 15.
Chris: Right. It’s like the rule.
Ned: And working in a pizzeria was definitely not going to this. The amount of grease in the air—Anyway. So, the owner and proprietor of said pizzeria told me that a cure for acne—or at least a treatment was drinking pickle juice—and being as we were a pizzeria that made other things, we had giant pickle jars, so she just poured me some pickle juice, and she was like down it. This will help with your acne. It did not—
Chris: No, no it wouldn’t.
Ned: —but she got to watch me drink pickle juice, which I think is, like, 99% of it.
Chris: Mm-hm.
Ned: Yeah.
Chris: That was a game of watch the 15-year-old do something dumb.
Ned: Mm-hm. That seemed to happen a lot.
Chris: [laugh].
Ned: Sometimes unprompted. Often unprompted. Almost always unprompted.
Chris: One day, you’ll stop playing that game.
Ned: [laugh]. When I’m dead, Chris [laugh]. Anyway, so I will not be drinking this cocktail, but I will be talking about AI for a while. We’re going to be drinking from the AI firehose, which ironically, is an AWS service that didn’t come up in all of the presentations that I attended. For those not familiar with tacos, that’s when you take a corn tortilla, and you place some toppings inside the tortilla, and then you fold it up—
Chris: [coughs significantly]
Ned: And then—wait, okay, no, wait, we were talking about the AWS Summit, and not delicious food. Sorry, I got sidetracked. It is lunchtime, to be fair. So, let me talk about the amazing French toast that I had at the Delegate. It was a Challah-crusted French toast.
Chris: [sigh].
Ned: The crusting was cornflakes and—stop it, Chris. All right, fine. I’m not procrastinating. You are. For those not familiar with the AWS Summits, these are many events put on by AWS all around the globe. They’re typically one or two days, and it’s kind of—you can think of it as a speed-run of re:Invent. And you don’t have to go to Las Vegas. So, that’s a gigantic win.
I’ve attended the NYC Summit three or four times, and really enjoyed it. In fact, it’s happening again in a few weeks, and I plan to go. But I also wanted to try out the DC Summit since I figured it would have a different focus and attract a slightly different crowd, by which I mean public sector, government, and military. It was interesting to see the different types of vendors this attracted at the Expo, and how the sessions focused on public sector, and intelligence applications instead of building the next Uber for Tacos, a company that I would absolutely invest in.
Chris: Yeah. Let’s talk about that instead of AI.
Ned: [laugh]. We probably could. But no. While I was browsing through the sessions, trying to plan out my schedule for the two days, I couldn’t help but notice how many of the sessions had AI somewhere in the title. I actually talked to an AWS person the following day, and they said that 55% of the titles of the sessions had AI in them. Even more of them were about AI, but that was just the ones that had it directly in the title.
Chris: Considering there are 380-something services, there’s no possible way that 55% of the services need AI.
Ned: [laugh]. No.
Chris: Explain.
Ned: [sigh]. Well, at first, I was just trying to avoid those sessions. So, I was trying to pick stuff like running EKS clusters, or AWS networking at the edge, or scaling storage best practices. You know, things that I think of as, quote, “Real topics.”
Chris: Which, ironically, would include the responsible and practical use of Firehose.
Ned: Indeed.
Chris: See, I know things.
Ned: I know you do. Once I finished trying to assemble this schedule, I looked at it and I realized it was kind of sad and boring. Like, say, your dad’s old record collection.
Chris: There’s a lot of Boz Scaggs.
Ned: It’s all stuff that’s been around for a decade or two, feels comfortable and safe, and it’s really just coasting along, playing the greatest hits. It’s the Eagles of sessions. So, I decided that I needed to shake things up a bit. Kids are talking about this new AI stuff. I don’t want to be the cranky old bastard in the back ranting about how no one makes good technology anymore. I need to keep my mind open and attend these weird AI sessions. Chris, are you familiar with Cognitive Behavioral Therapy: CBT?
Chris: Yeah, that’s what CBT Nuggets is about, right?
Ned: Is it? No. Is that really what CBT stands for?
Chris: No.
Ned: I don’t know, man. I never watched any of that crap.
Chris: [laugh].
Ned: [laugh]. It wasn’t crap, but I also didn’t watch it.
Chris: I’m sending them an email. You’re in trouble.
Ned: I didn’t know they were still going concern, to be honest. A core premise of CBT is to face your fears instead of avoiding them, so that’s what I did. Every single session I attended was AI-related. I think I went to about ten sessions. Every single one, AI-related. No running to the sweet and simple embrace of EC2 instances, or VPC design. It was going to be mainlining AI for two days.
I have to say, I am glad I did it. And that’s what I want to share today: what I learned over the two days that I was there, how I’m feeling about AI now, and what the future looks like. So, let’s get into it.
Chris: Okay.
Ned: We’ll start with the keynote, which makes sense, right? Supposed to be the beginning or, you know, the main thrust and theme of a general conference. You can watch the keynote if you’d like. It was presented by Federal CTO at AWS, Dave Levy. No relation to Eugene Levy, unfortunately.
Two quick things about the keynote. One, the keynote actually happened on day one at 11 a.m. after sessions had been going on for two hours. So, I had already attended a couple of sessions, but I’m going to start with the keynote anyway. And number two, there was blessedly only one keynote, as God and country intended.
Chris: Wow.
Ned: Yeah, thank you, sweet baby Jesus, and your tiny cherub hands, one keynote. From the very outset, Dave made the point that GenAI is changing the world. He made a reference to the tale of Rip Van Winkle, which is a story I barely remember, but if I’m recalling correctly, and he explained it properly, the core conceit is that Rip falls asleep for 20 years before the American Revolutionary War starts, and when he wakes up, the revolution is over and the world has changed drastically. I think that’s it. Don’t look it up.
The obvious parallel here is that our world is going through a similar sea change, and the public sector can’t afford to sleep on it like Rip Van Winkle did. In fact, he kept using the word revolution several times to describe what AI is doing. And the word revolution does feel, like, a little bit loaded, especially given the political climate, and the seven or eight protesters who stood up in the middle of his talk, and yelled stuff about Gaza, Israel, and how evil Amazon is, and questioned how he could sleep at night. So, I don’t know, maybe find a word that’s a little less evocative. Maybe? I don’t know. Anyhow, it has nothing to do with AI. Just word choices.
His main point is that GenAI is going to change the world, that GenAI requires the cloud to work. And he quoted a study commissioned by AWS that claims that cloud is 4.1 times more efficient than on-prem AI. And that AWS has the tools you need to take your AI experiments to production. Not sure how much weight I put behind a study commissioned by AWS that finds out, shockingly, cloud is better than on-prem. But, I mean, what else do you expect Dave to say?
Chris: I have thoughts, but we can—
Ned: Feel free to share.
Chris: Anytime you see a study like this, what do they mean by on-prem?
Ned: Yeah.
Chris: Who set it up? How much money and time and expertise did they invest in it? What models are they using? Et cetera… et cetera… et cetera.
Ned: What does it mean to be more efficient? How do you measure that efficiency? Like, so many questions. I didn’t want to get bogged down in it. I will try to find the study if I can. We will get into AWS’s AI stack in a little bit, but I do want to mention a couple other things. First, Dave said something about how AWS isn’t here to build the best AI applications. That’s what they expect you to do. They’re just providing the raw materials, which inadvertently is a dig at their own Amazon Q product, which ostensibly is a really good AI application. Whoops.
Chris: Which also should be what AWS does.
Ned: You would think.
Chris: Just provide the raw materials.
Ned: And that’s good. I actually liked that part of it. Like, yeah, you give me the raw materials. That’s what AWS has specialized in for years is giving developers, here’s a big old Lego set or an erector set, or whatever you want to call it; build your thing. You also made an offhand comment about data sovereignty issues, which I think was a subtle dig at Microsoft’s latest stumble with handling GDPR and the Scotland police force, I think?
Dave had two guests with him on stage, one at a time. The first one was the director of AI for the CIA, and the second was the head of AI for the army. And both chats turned into weird recruitment pitches for the CIA and the army, along with a plea from the army guy for input on their AI layered defense framework. What was strangely absent from both conversations was any mention of how the CIA or the army are using AWS for anything.
Chris: That’s classified.
Ned: I guess? But also, like, you would think if you have guests up on stage at an AWS event, they’re going to at least mention how they’re using AWS in some way.
Chris: I mean, counterpoint. This was hosted in DC, so it could very well have been, somebody called AWS and explained to them that these are going to be the guests.
Ned: Mmm. Could be. I mean, the CIA probably has the leverage to do that. So, since they didn’t say what AWS is doing with AI, let’s talk about AWS’s AI portfolio. It’s kind of an open secret—not even a secret—it’s just known that Microsoft has a first-mover advantage when it comes to AI products.
They partnered closely with OpenAI. They rolled out Copilot long before AWS had CodeWhisperer and GitLab had their product called Duo. Amazon and Google both looked to be caught flat-footed when it came to AI, and they are both furiously trying to catch up. That being said, a first-mover can have a big advantage, but it also means that you’re likely to fuck up a lot. You’re the one blazing the trail, you’re not following some well-trodden path, so the chances of making a logistical or tactical error are high, even if your strategy is sound.
Sometimes being a fast-follower can actually work in your favor because you can see what the first-mover stumbled over and adjust accordingly. And I kind of get the feeling when watching AWS talk about their various AI offerings, that’s kind of where they’re at. They’re keeping an eye on Microsoft, and going well, as long as we can stay relatively close behind, we’ll be fine. So, they don’t have a monster model like OpenAI’s GPT4o. I guess it’s called four-oh—why did they abandon the semantic versioning? I don’t understand. Anyway—they don’t have that running in their data centers. Instead, they have their own foundational model which is called Titan, a model that, by most accounts, is pretty mid. Meh.
Chris: More like a Teen Titan, am I right?
Ned: Ahh, I like it. All right, which Team Titan would it be?
Chris: The boring one.
Ned: Robin. It’d be Robin.
Chris: [laugh].
Ned: [laugh]. But we repeat ourselves. What is a poor cloud hyperscaler to do? Build Amazon Bedrock?
Chris: The Flintstones thing?
Ned: Yeah, yeah, they invited Fred over and just, like, hash it out over some giant ribs.
Chris: And strangely, he did a cigarette ad. Everyone was deeply confused.
Ned: [laugh]. Oh, hell of a callback. Well, done. So, when Bedrock was announced, most people were like, “Huh?” But essentially what it is—and actually find it pretty interesting now that I understand it—it’s essentially an abstraction layer on top of other existing foundational models.
So, it is not an LLM. It’s not a model in and of itself. It doesn’t tie you down to a single model. Instead, you can choose to use one or more of their various supported models, or bring your own model. Bedrock currently supports Titan, Anthropic, Meta, Mistral, and a bunch more. You can look up the current list; I’m sure it’s constantly changing.
And the first session that I attended was actually a builder session where the presenters walked through using Bedrock in a Jupyter Notebook to interact with four different models, basically, simultaneously. I very much jumped into the deep end of things, inadvertently. I [laugh] walked in, they had, like, code up on the screen, and I was like, what did I just get myself into? All right. The idea was to test and evaluate the responses from each model, and figure out which one would be best suited for a set of tasks.
And you can go back and check that on a regular basis because the models are constantly changing. Inside of the Bedrock SDK for Python, there’s a method called Converse, and this method and the service behind it abstracts the required inputs and parameters for each model into a consistent set of inputs. So, every model has its own specific parameters, and the way that it expects you to structure requests, and the way that it structures its responses, especially when you’re interacting programmatically. The key idea here was you can use this Converse method in a method call to loop through as many models as you’d like at once, and it has a consistent interface in front of those models, so you don’t have to adjust your parameters and your inputs, and you know what form the response is going to take because it normalizes it for you.
Chris: Hmm.
Ned: That seems kind of useful. If you need to, you can put in model specific-parameters in a separate field, so in your configuration of the method, you can say, “For this model, add these two or three customized parameters.” So, you can still add that level of customization in there, but if you don’t want to, you don’t have to. And since the technology and the models are evolving so rapidly, having a low switching cost between models is going to be a huge benefit, versus just hitching your wagon to OpenAI and hoping for the best from Sam Altman. I don’t feel super positive about that [laugh].
Bedrock also has a couple other neat things. One is something they’re calling Knowledge Base, which is essentially RAG, aka, Retrieval Augmented Generation. Like the name implies, you can load up additional documents in S3 and feed them to a generic model to help it better understand your unique business and requirements. Each model has a different way of doing RAG, so it’s nice to have Bedrock just handle that abstraction. Hey, I want to use these six different models. Send it in the document way they prefer from this S3 bucket that I’ve loaded things up with.
The other interesting concept—and I’m still trying to wrap my brain exactly around how it works—but it was this concept of agents and tools. And I’m not talking about agents in the sense of chatbots, or digital assistants. Instead, it’s a way that Bedrock lets you define tools that models can pass work off to in their application flow. So, the tools are really just Lambda functions that interact with some other API or data source, and the agent is the part of Bedrock that does the tool invocation and passes information back to the model. So, you can ask your application to do something like parse a bunch of emails for sales orders and then add them to your ordering system, and the model can use natural language parsing to understand the mail content, and then invoke the tool to actually interact with the ordering system. This is the sort of thing that’s going to put a lot of interns out of work.
Chris: Yeah, it’s basically fancified RPA.
Ned: Yeah, that’s a great way to put it. It is very much like—yeah, whatever RPA stands for, Robotic Process Automation?
Chris: There you go.
Ned: That’s the one. I got there. But yeah, if you have existing APIs and other systems that you want your language models or your foundational models to interact with, you don’t have to write that interaction into your application. You just define this tool spec—I think they called it a tool spec; it something along those lines—and then implement that tool spec in Lambda, and now your foundational model can interact with whatever API you want to hand it off to, if it’s internal or external. I was like, “Oh, that sounds pretty awesome, but also possibly incredibly dangerous because now we’ve allowed AI to just do whatever the hell it wants.” [laugh]. I’m sure that won’t come back to bite us.
So, that’s Bedrock in a nutshell. There’s more to it, I’m sure, but those are the pieces that jumped out to me. AWS also went to great pains to say they do not store your requests, the model responses, or the augmented data anywhere inside of Bedrock. All of it lives inside your account, and the models that you bring with you. That’s nice. I think they went to special pains to explain that because of the audience.
Chris: Right.
Ned: So, from there, I want to jump ahead to a presentation that was about using GenAI in DevOps, which was a topic that I found very interesting. The two presenters, Tyler Lynch and Pranusha Manchala—I’m hope I’m getting that vaguely right; apologies to Pranusha—they lead the chalk talk and polled the audience about issues and challenges they encounter at all stages of the software design and deployment pipeline. If you’ve never attended a chalk talk before, they basically have a whiteboard, and, like, a loose presentation, but it’s very audience-driven. So, they’re asking for input: what is your experience? What are you seeing out there?
As we rolled through the issues, they tried to give examples of how AWS’s various developer services could help alleviate some of the challenges people were feeling. So, I mean, you’re at an AWS conference; the solution is going to be AWS, right?
Chris: One would think.
Ned: Yeah. They highlighted three products. One was Amazon Q Developer. Q Developer used to be called CodeWhisperer. I don’t know if that was better [laugh] or worse.
Chris: I think you do know.
Ned: I do know. They’re both terrible. It’s very similar to what GitHub Copilot does. It plugs into your IDE, and it provides suggestions and a chat interface. Do you want to know what a chunk of code does when you’re trying to get to know a codebase? Highlight it, ask Q to explain it to you, and it will look at it and say, “This is what this function does,” in plain English.
Do you need to refactor some code to be more efficient or scanned for vulnerabilities? You can do both of those things, making suggestions for how you could rewrite the code. Q can also write unit and integration tests, something that basically no one ever wants to do. As someone who’s working on developing a Go tool right now, I can tell you that I spend more time writing unit tests than I do writing actual code. It sucks.
Chris: Not great. It’s not great.
Ned: Yeah. Tyler also mentioned that Q can help with code migrations to newer language versions, and this was really interesting to me. Amazon used this functionality to migrate from Java 8 to Java 17. Now, as someone who is not a Java developer, I wasn’t really sure, okay, but I guess that is a significant jump, and those are two largely implemented versions. I guess there’s not a ton in between. I don’t really follow the numbering.
But basically, they said they had a thousand applications that use Java 8 internally at Amazon, and so they took Q and turned it loose on those thousand applications to upgrade them all to Java 17, and they were able to do it in two days. Eyebrow-raising indeed.
Chris: [laugh].
Ned: That’s bananas. If you think about the amount of work that would be for developers to update the code and run the tests for it, they saved a tremendous amount of time, and also washed away a tremendous amount of tech debt they had being on this older version of Java. They also have a similar path to get from Java 4 to 8, and then 17 for much older applications. I feel like Java 4 was current when I was developing Java back in, like, 1998. They also are working on similar upgrade functionality for .NET to .NET Core, and Python 2 to 3. This is the kind of thing that no one ever wants to do. No one ever sits down, they’re, like, “Today’s the day when we’re going to pay off our tech debt and upgrade from Python 2 to 3.”
Chris: Yeah. For most companies, that day is never.
Ned: Right because you look at it, you’re like, there is no immediate business value in me doing this. The application still works, I am not doing anything until I absolutely have to. But doing that upgrade actually does come with some benefits.
Another session I attended was put on by GitLab, and they were talking about their code assistant, which is called Duo, and Duo can do refactoring from C to Rust. And they talked to a public sector organization whose name they couldn’t say, but they talked to one, and they had this C-based application that was a bit of a bottleneck for what they were doing. They had Duo port it over to Rust. And this was just an experiment. It was supposed to be like a three-month-long migration process. They had Duo try to do the migration of C to Rust, and move it to running on Lambda instead of EC2. Duo and the senior developer managed to do it in two days.
Chris: It’s this the rule? Everything happens in two days?
Ned: Apparently [laugh]. The resulting application was 90% smaller, ran a hundred times faster, and could scale by 50x. In part because of Lambda.
Chris: Well, and just because of Rust. And something interesting about this is it’s not just simple refactoring, but you’re looking at optimizing how the code does what it does. You know, like, the difference between a junior programmer writing this program for the very first time and somebody that’s got 35 years of experience picking up that code and saying, “This is what it should have looked like.”
Ned: Exactly. And I think that was the point that the presenter made was, senior developers will actually get some of the most benefit out of this because they can look at the code that is produced, and they understand the context of the original code, so they have a better chance of making sure the code actually works after that conversion, and works properly. It’s just that this GitLab Duo is going to save them a ton of time in doing the actual code writing. So, that was pretty impressive to me.
He also talked about how so many companies will do, like, a lift-and-shift, and then never actually refactor the code, and so you have all these inefficient applications eating up cycles on EC2 in AWS or wherever you decide to host your code, if you can go back and use one of these code assistants to refactor things pretty quickly—you know, instead of a six-month lead time, you’re now talking about a two-week lead time, maybe—that’s actually worth doing, and could be game-changing for some organizations.
Chris: Yeah. I mean, a lot of times, people simply hide bad code behind larger instances.
Ned: [laugh]. Sometimes? Sometimes it’s doing a lot of lifting there, buddy [laugh].
Chris: [laugh].
Ned: Yeah, the fact that we can just throw more hardware at it, and have done that for years, this is actually an opportunity to slim down your code, maybe make it safer—because if you’re moving from C to Rust, it’s now memory safe at least—and maybe squash some security bugs in the process. Like, that’s a win for a lot of teams.
Moving on from the code assistant, the other tools that were mentioned are DevOps Guru and Code Guru. Terrible names. The former does scanning of CloudTrail logs looking for anomalous events, and the latter does code scanning, testing, and assessment. From what I understand, Code Guru charges by the line scanned and is prohibitively expensive.
Chris: Okay.
Ned: DevOps guru, similarly expensive for looking at CloudTrail logs. But they mentioned them, so I guess I should, too. The last part of the AWS GenAI sandwich is hardware. AWS has some hardware to throw at the problem, including their own in-house developed chips called Trainium, and Inferentia.
Chris: Trainium?
Ned: Trainium, which are for model training, and inference, respectively. But I think you already guessed that. Are these chips any good? No idea. Are the names any good? No. They also have Nvidia A100s and soon will have H100s, so maybe that’s a better bet if you’re doing this model training thing.
Chris: Just a thought.
Ned: Yeah. Another session I attended was all about the training process for a foundational model called Falcon, and to train the model, they use SageMaker, and ECR, and S3, the biggest version of the model that they train, they did three versions of it. The biggest one was called 180B, which I guess is for billion parameters. This thing used 3.5 trillion training tokens, 46,000 petaflop days—I’m not even sure what that means, but that’s what they quoted—and 4096 GPUs. The training process took six months, and used P4d EC2 instances that each have eight A100s in them.
This was not cheap… by any stretch of the imagination. They actually, through this process, hit the performance limit of S3, and had to write new custom libraries that go beyond the 3500 PUTs and 5500 GETs per second that S3 normally supports.
Chris: I was going to say, S3 is not supposed to have limits.
Ned: It does [laugh]—
Chris: [laugh].
Ned: But you have to work real hard to get there. They did it. Congratulations. It turns out, if you use—there’s a version of S3 that is a single zone—not single region, but single zone—S3—I forget exactly what it’s called—if you pick that version of S3, the limits go up because it no longer has to do any replication or spreading across availability zones.
Chris: Right. I think that version, when you look at it in the drop-down, is called don’t use this for DR. Don’t—for God’s sake—use this for DR.
Ned: Yeah, it’s intended for exactly what they were doing, which is these large data models that they’re loading.
Chris: People are going to use it for DR, aren’t they?
Ned: They absolutely are. So, that session was super cool because they had AWS and folks from Falcon’s company—it’s called TII—talking about the model design, the training methodology, the data sources and efficiency. I feel like that could be its own episode, so I’m just going to stop there.
Chris: Cool.
Ned: Geez. I will brief—
Chris: You said you were going to stop.
Ned: I was going to stop talking about that. I’m briefly going to touch on AI security, just to mention that I attended a session that talked about how to implement—or security things to worry about when it comes to interacting with LLMs, and they mentioned that OWASP has a top ten for LLMs, which I had no idea. That report was brought together by 500 experts and 125 contributors, and came up with a list of possible vulnerabilities for LLMs. I read through part of the report. It is fascinating. Once again, this paper could be its own show, and I would like to have someone from OWASP or one of the contributors come on and explain it to me like the third-grader that I am.
Chris: I support this plan. It means I have to do less work.
Ned: Well, there you go. Me too [laugh]. So, just to bring it all down to brass tacks, am I an AI convert now? Was drinking from the firehose enough to bring me over to the AI side? No. My goal at the conference was to enter each session with an open mind. I did my best not to scoff at ridiculous claims, I tried to take people and the services at face value, to listen, to not judge.
And for the most part, I was able to turn off the supercritical and cynical parts of my brain, and just kind of absorb it as it was presented. Now that I’ve had a little time to digest, here’s what I think: for starters, GenAI is still very much in its early days. There were several comparisons to the dotcom bubble, and I think that’s fair. There’s an embarrassing amount of money being poured into anything AI right now in the same way that any company that had a vague association with the World Wide Web was showered with funding in the late-’90s. That bubble must and will pop.
There will be some genuine use cases that are worth pursuing, some that are a little ahead of their time, but they’ll come back around, and some that are just absolute crocks or complete scams. A lot of them are that. The most compelling use case I saw—and you may have already guessed this—was around developer and ops productivity. There’s a metric ton of tech debt at every large company, little desire to actually fix it—well, I won’t say ‘a little desire.’ There’s desire to fix it, but not enough time or money to fix it—this may actually be the best use case for GenAI AI as it exists right now.
GenAI is good at understanding code. It’s good at migrating it, it’s good at testing it, and it’s good at ingesting a lot of data and making it available. So, it’s going to help Dev and Ops tremendously. Although I have a little concern around junior talent, and them being developed if we’re going to end up replacing them with AI. So, that is a potential problem.
Everything else was AI whitewashing. Walking the expo floor, basically, every single booth had AI plastered on it somewhere. I am not exaggerating. Every company, from storage to monitoring to professional services, had AI something, and 90% of them were talking about some kind of shitty chatbot or some basic-ass machine learning. In a few years, it will be something else in the same way that cloud was at one point. But most of it is just bullshit.
Am I glad I took the plunge? Yeah. It was informative to see what people are really doing versus the marketing and the general messaging. I can see there’s a lot of potential for disruption coming down the pike, but it’s not nearly as dire or revolutionary, as some suggest. Or I don’t know, Chris, maybe I’m wrong and AGI will be born next week and put me to the sword. Whatever. I’m going to be on vacation. My digital assistant can handle it.
That’s it. Hey, thanks for listening or something. I guess you found it worthwhile enough if you made it all the way to the end, so congratulations to you, friend. You accomplished something today. Now, you can sit on the couch, query Developer Q—Q Developer?—for the best way to rewrite your code, and just relax. You’ve earned it.
You can find more about this show by visiting our LinkedIn page, just search ‘Chaos Lever,’ or go to the website, chaoslever.com where you’ll find show notes, blog posts, and general tomfoolery. We’ll be off next week because I actually am genuinely going on vacation. So, you’ll have to wait two weeks to see what fresh hell is upon us. Ta-ta for now.
Chris: When are you not on vacation?
Ned: I’m sorry, I was on vacation. What? [laugh].