Episode 17: Do We Really Need All These Data Centers?

Data centers have dominated the AI news cycle lately, from local concerns about noise and heat to questions about who pays for the added strain on the electric grid. In this solo episode, I take stock of the build-out and why I think it’s happening with very little mindfulness. A conversation last week likened it to the railroad boom of the Industrial Revolution, when tracks went down everywhere, towns boomed or withered depending on where the line ran, and nobody planned for the environmental damage.

Two questions rarely come up in the coverage. First, how much of this capacity is actually needed? Every frontier lab is building its own, with no real effort to treat compute as a shared resource that could lower the environmental impact. Second, why are we running every query through massive general-purpose language models? I compare them to department stores, where everything is on the shelf and none of it is as good as it could be.

Running makes a good example. The AI coaching features in Strava, Runna, and Garmin lean on frontier models, and as a coach I think they suffer for it. A narrow model trained only on running knowledge would likely coach better and draw far less from the grid.

So where’s the hope? I’d like to see companies build data centers alongside environmental groups and communities, pay for their own grid impact instead of passing it to taxpayers, and invest in narrower, more efficient models. For the rest of us, it comes down to mindfulness. Before reaching for a chatbot, ask whether the use case is warranted or just a dopamine hit. Is the scratch really worth it?

0:58 Environmental and grid concerns
2:22 How much capacity do we really need?
4:11 AI as a department store
6:12 Where the hope lies

What is AI Hope?

What's your AI success story? What made you realize you had to change? And what does a hopeful future with AI look like to you? AI Hope with Geoff Livingston asks three simple questions of people doing the hard work of adapting and thriving in the AI era.

Data Centers
===

[00:00:00] Hey everybody. My name is Jeff Livingston and this is AI Hope. My book, Now Is Gone, is about never-ending change. This podcast is about what comes after, the stories, the people, and the reason for hope.

Happy Monday. Sorry this is a little bit late. I ran the Marine Corps Marathon yesterday. ~Uh, ~I paced a friend. It went very well, but as a result, I was mush and just could not put together two sentences very well yesterday. So you get the podcast a little bit late. ~Um, ~next week we do have a great guest, ~uh,~ but this week I did wanna chat with you a little bit about, ~um,~ data centers only because they've been in the news a ton recently, and, ~uh,~ generally speaking, it's, ~uh,~ becoming more and more of an issue with, ~uh,~ the political side of it, but also with ~the, uh,~ general hype in how every organization is promising to build data centers or data centers, and, ~uh, you know, ~what d- what does this mean?

And [00:01:00] is there any hope with this stuff? So, ~I mean, ~I think the environmental concerns are very well documented now, or they're becoming well documented with the noise, the heat. ~Uh, ~I've seen conjecture about cancer, ~uh,~ for local folks. And then there are other issues, ~uh,~ as relating to just general commercial viability, ~uh,~ the association with the e-electric grid and the tax on that and people paying.

So generally, this arms race to create, ~uh,~ data centers across the country and globally is creating a lot of pain, is the way I would put it. And I don't think that these, ~uh,~ data centers are being built necessarily mindfully. So ~I, ~I did wanna just take a second on it and, ~uh, I, I,~ I did have a conversation last week without, ~uh,~ getting into full disclosure of it.

The person did liken, ~uh,~ data center build-outs to kind [00:02:00] of like the Industrial Revolution and railroad build-outs and how that just happened all over the place without thought and how it created, ~uh,~ all sorts of, ~uh,~ impacts, ~uh,~ creating economic booms for some folks, ~uh,~ depending on whether or not the railroad went through their town, but also the environmental damage and all the things and how that was done without any real, ~uh,~ mindfulness.

And ~I, ~I think one of the... There are a couple things that I feel as somebody that's been involved in AI for a while now, ~uh,~ that aren't very discussed well with the, ~um,~ data center boom. One is, I don't know how much of it's actually needed. ~Uh, we, ~we hear that the compute power is needed. We hear this from Anthropic, we hear it from OpenAI, et cetera, et cetera.

But then we also look at, ~uh, you know, ~every company is basically in an arms race saying that they're gonna build capacity, and there's no real shared responsibility or shared attempts to build this capacity [00:03:00] so that all AI companies could use it. And not every company needs its own dedicated data centers, ~um,~ and that these could be more of a community resource and, ~uh,~ lower the environmental impacts.

And that conversation's just not happening, right? Like ~I, I,~ I just don't see that happening at all. ~Um, ~there's just general joint initiatives maybe with ~the,~ the media and the press. ~You know, ~we're gonna contribute this amount of funding and all that kind of stuff, which is all just more vaporware as far as I'm concerned until you see like hard contracts built.

But no real, ~uh,~ accountability for how much of this is actually needed and how much of it can be shared resources. And then the other things that are, ~uh, uh,~ particularly disturbing about it, just from an industry perspective, is that a lot of this is because we're using these big, inefficient large language models for AI.

~Uh, ~which, ~you know, ~again, every time you're making a call, you have this massive database of training knowledge that's being used. Not only can it answer your [00:04:00] question about like I'm wearing a New York City Marathon shirt, ~uh,~ questions about the New York City Marathon, it can wash your dishes, help you code, ~uh,~ work on your professional services plan, et cetera, et cetera, et cetera.

And so do we really need an, a- an AI that's that wide and deep in knowledge? 'Cause the truth of the matter ~is, ~is ~it's, ~it's wide and deep, but not that deep. It lacks domain specificity. So what you're really likening it to, or what I like to liken it to, is a department store, right? It's got everything, ~uh,~ and none of it's as good as it could be.

And so at some point we have to get narrow with AI. We have to become more efficient, ~uh,~ maybe more specific in the algorithms that are trained. Maybe it becomes more, ~uh,~ beneficial to do that with LLMs to create those new AIs that are, or those new, ~uh,~ models that are specifically created for industries or use cases like running or [00:05:00] whatever.

And I look at Strava in particular. I look at Runna. ~Um, ~I look at a, ~uh,~ Garmin, and these AIs are not very good, right? And some of them use You got it. Frontier models, large language models that do their summarization, and ~I, ~I think it really suffers quite a bit, ~uh,~ as a result of that, as somebody that's experienced and a coach.

So that's a great example of, ~like,~ at some point, does it become efficient to have running AI, ~you know, ~specific AI built only for the purpose of coaching and helping people with their runs using only running knowledge and not having access to all that stuff? 'Cause ~I'll~ tell you one thing, it'll be a lot less of a draw on the power grid, and it'll be a lot less of a draw on the compute power.

And so, ~you know, that's, ~that's one real serious issue that we're gonna have to get over. And the other thing is that the massive inefficiencies that are coming with AI are also creating a lot of other problems like [00:06:00] cybersecurity, accuracy, trust, on and on and on and on. ~Uh, ~again, you use these large language models that are basically department stores for getting the results, and ~it, ~it's just wildly inefficient.

~Um, ~okay, so there's that. And then, ~uh,~ from a larger hope standpoint, 'cause ~I,~ I think we're supposed to talk about hope on this podcast. It's AI Hope, right? ~Uh, I, ~I would really like to see some more thoughtful, mindful development, some work with environmental groups to make these data centers better for communities, ~uh,~ to use them in a manner that have less tax on the grid, that are paid for by the companies building them, not the taxpayer.

~Uh, and, ~and ~you know, I mean, ~these are all the things that people are saying. ~Uh, ~I don't know why I have to say them other than to say that ~I, ~I believe in them too, and ~I, ~I'm not really a big fan. ~It, ~it's getting to the point now we have to think about when we're using AI, whether or not this use case is warranted or [00:07:00] if we're just, ~uh,~ getting a dopamine hit of talking to, ~uh,~ ChatGPT or Claude or whatever other AI you're using and, ~uh,~ Copilot, Gemini I won't say the other G one.

I- ~if, ~if you're using these AI just to, ~you know, ~bounce things off of when you could probably just as easily research it somewhere else, then yeah, ~maybe,~ maybe it would be better not to use it, ~you know, if we're, ~if we're looking at it. I know that the general cost of a call is a lot less than the cost of a mile from an environmental standpoint, but that doesn't make it better.

~Um, ~and I think that as a result of that, ~uh,~ again, it's all about mindfulness. It gets to the same thing as bottled water, all the different issues from an environmental conservation standpoint. Do I really need to do this? Do I really need to use this, ~uh,~ tool? ~Uh, ~when is this something that's gonna benefit me and allow me to do things, ~uh,~ that I could not do before and it's gonna be helpful to the larger mission versus just convenience?

And, ~um, you know, ~I don't know what the right answer is [00:08:00] for people on that. I know that we're all getting pushed to it, and that all of our software now is starting to include, ~uh,~ these chatbots and the AI, and it's, ~uh,~ it's gonna be, it's gonna be ~a, ~a ride. And, ~uh, I, ~I just encourage us to think with that lens like, is this the right kind of impact?

Is this worth... Is the scratch really worth it here? ~Um, ~and so I'd be interested to hear your ideas. I don't have a lock on this, but, ~uh,~ I am increasingly, ~uh,~ mindful and skeptical of the need to have this mass proliferation of data centers when I think really we should just be more mindful and, ~uh,~ more intentional about the AI that we're building and how we use it.

Just a thought. Looking forward to yours. Cheers.