The hidden environmental costs of a single AI prompt, with Dr. Sasha Luccioni

DR. SASHA LUCCIONI: What I’ve noticed is that people have this like disconnect. They’ll be like, oh, I care about climate change. I try not to take the plane, I try to take the train, whatever. And then when you talk about machine learning, people don’t necessarily have this reflection about their own work.

RANA EL KALIOUBY: That’s Dr. Sasha Luccioni. She’s making the point that even the most eco-conscious people may not be aware of the environmental footprint of AI. So, in this episode, we’re turning our lens to the climate impact of AI, with Sasha’s help.

LUCCIONI: I think people in general can be like super dedicated to fighting climate change, but then in their professional job it’s like, well, that’s my job. And kind of like, I don’t really think about it from that perspective. And so I do think that awareness is a huge part of it.

EL KALIOUBY: Sasha is the Climate Lead at Hugging Face – a hub for all things machine learning. She’s made it her mission to grow this awareness. Maybe you’re like me … I’m generally an environmentally conscious consumer, and I use AI a lot. And I know my use of AI has an impact … but so far that hasn’t affected how I use it.

Then there are people so concerned about AI that they won’t use it at all. Like these folks posting on TikTok:

TIKTOK1: “I don’t mess with generative AI as a scientist, an activist, or as a creative and it’s because I understand the environmental impacts of it.

TIKTOK2: “Stop using AI it’s a climate disaster”.

TIKTOK3: “So stop using it, which is easier than you think”.

EL KALIOUBY: Sasha is here to help with some nuance.

We asked her on to do something pretty cool – and super useful. She’s helping us unpack the entire energy footprint of a prompt, step by step. Starting with all the infrastructure and compute power that’s needed to build these AI models – to the moment when you enter a prompt and get an AI generated answer.

I’m Rana el Kaliouby and this is Pioneers of AI, a podcast taking you behind-the-scenes of the AI revolution.

[THEME MUSIC]

Welcome to Pioneers of AI, Sasha, it’s so great that you’re joining us today.

LUCCIONI: Very thrilled to be here.

Copy LinkWhy AI needs an energy score not just smarter benchmarks

EL KALIOUBY: All right, so we are going to get into the nitty gritty of the energy consumption behind generative AI. But before we get into all of that, I would love to hear more about your work. So you’re the Climate Lead at Hugging Face, which just got listed on the Forbes top AI 50 companies. So congratulations on that. Yeah, so tell us more about what Hugging Face does and what your role there entails.

LUCCIONI: Yeah, so Hugging Face is essentially the biggest platform for sharing AI models and data sets. And what’s really exciting about it is that it’s used as much by big tech companies as by academics and nonprofits to share models and to build upon each other’s innovation.

So, for example, you take like a big model trained by Meta, and then you tweak it to your context, your use case, and then you re-share it with the community. And my job in all that is to evaluate the environmental impact. So for example, a project that I’ve been leading recently is creating AI energy score ratings to help people pick, like if they wanna do a given task, whether it’s image generation or object detection, to pick a model that’s the most efficient for that task based on essentially how much energy they use.

They don’t necessarily have to act upon it. I’m not forcing anyone to pick models based on only energy, but I still think that it’s just like a reflection we should be having in general.

So essentially to guide people in making sustainable decisions.

EL KALIOUBY: So, I’ve been thinking a lot about AI benchmarks. And you know, today a lot of the benchmarks are focused on how smart, how accurate these models are. So for example, all the various GPT models have already passed the bar exam, which is of course the exam for lawyers. It’s also passed the medical exams. And I think it’s really interesting because nobody’s paying attention to other aspects of AI, but of course you are. And you kind of referenced the AI energy score. Tell us more about what that is, how you built it, and maybe some of the results so far.

LUCCIONI: Sure. Like people, essentially, when they’re picking AI models or even training AI models, they tend to focus on these technical benchmarks. But there’s also so many other aspects to that because, so for example, if you have a model that’s, you know, yeah, sure, like 99% accurate, but it uses 10 times more energy than this other model that’s maybe like 98 or 97% accurate, but it’s a lot more efficient. And we did give like absolute measurements, like kilowatt hours, but people don’t really know what that means when you give them the actual measurements. So that’s why we started doing relative comparisons. And so essentially we tested hundreds of models and we’re ranking them instead of giving the absolute value per query.

Copy LinkHow smaller models can outperform on efficiency

EL KALIOUBY: That’s very cool. What are some of the models that are most energy efficient out there? If you can share some.

LUCCIONI: What’s interesting is that there’s really a trade off. Like for example, recently Hugging Face trained a couple of SmolLM models. Yeah, exactly. And they’re really energy efficient.

EL KALIOUBY: SmolLM as in S-m-o-l-L-M, are a collection of small language models that Hugging Face developed.

LUCCIONI: But like training took a little bit longer, it actually enabled converging upon an end model that was super efficient. And like for a smaller model you can get a lot more performance, which is really cool.

EL KALIOUBY: Compared to large language models, these models can be run locally and reduce inference cost. They also outperform other models in their size category.

LUCCIONI: And conversely, everyone was like, oh, DeepSeek is like, it’s actually a win for the environment, whatever. And sure, like the training of the model might have supposedly taken less time, but the ensuing model, like the result of that training, is massive. Like it takes like four GPUs just to load it into memory. And so, you know, there’s a trade off here because like sometimes you need to kind of train a little bit longer, use more resources upfront, but then you have models that can be used by the community. Or vice versa, you have these huge reasoning models that are essentially unattainable for the average person without access to a super computer.

EL KALIOUBY: Basically, there are lots of tradeoffs to the AI models out there. Some take so much power to train, but are lower on inference cost. And then other models could never run locally because they’re just too big.

It’s not always clear cut what is the most energy efficient model – because it all depends what you measure – the training? The deployment once it’s in use? But we definitely do know that some consume a lot more energy than others. For example Hugging Face’s LLM Bloom is trained on less data than say GPT-3 and in this case, it needs less compute and consumes less energy.

But why exactly does AI and specifically generative AI take so much energy? After a short break, we’re digging into the energy lifecycle of an AI prompt, from before you even write your prompt to when you get your AI generated answer. Stay with us.

[AD BREAK]

Copy LinkWhy a single prompt has a hidden physical footprint

Let me set the scene for you. You open an AI chatbot of your choice and enter a prompt — something like “Write a thank you note for my dogsitter.” You hit ENTER. That starts a process that uses a LOT of energy, we’re talking 20 to 30 times more energy than a regular internet search. And we’re going to get into why.

I think when the average user prompts an LLM, it is like a black box, right? It’s like disconnected from what actually happens when it spits out an answer. And I think a lot of people don’t realize that AI exists in the material world. It’s not just ethereal. So I wanna spend the bulk of our conversation looking at this lifecycle from start to finish. And I wanna start with AI infrastructure and particularly step one, which is we need AI chips to both train and deploy these models. What are some of the energy and environmental considerations when we are manufacturing chips?

LUCCIONI: So chip manufacturing is actually also a black box. We have very, very little information. So like Nvidia, the main maker of GPUs, hasn’t yet shared any specific numbers, but we do know that there’s a lot of rare metals that get used in that process. Like gallium and germanium and cobalt, and just mining those metals, extracting them from the earth, emits a lot of pollution.

And it’s just like resource intensive. So there’s that. There’s also the energy it takes for the actual connecting of all the different components and it’s very, very precise work. And you need like really high powered tools. And all of the fabs, as they call them, are actually based in Taiwan.

EL KALIOUBY: Fabs – as in the specialized factories where chips are produced. Not all of them are based in Taiwan – but for such a small country, it definitely houses a significant portion of these facilities. Most of Taiwan’s energy comes from fossil fuels.

LUCCIONI: So you know that the energy being used is also pretty carbon intensive. And then the water — there’s a lot of water that has to be used for purifying every little layer of silicon that gets connected into the chip. And so essentially it’s like millions of liters of water that get used by these fabs as well. And like a couple of years ago, there was a drought in Taiwan, and then the government actually had to make a choice between farmers planting crops and chips being produced, and then they chose the fabs and the farmers couldn’t plant their crops.

Copy LinkThe overlooked waste problem behind the AI hardware race

EL KALIOUBY: Yeah. What about e-waste? All these chips that are becoming obsolete or outdated, where are they going and how do you think about the consequences of that?

LUCCIONI: Yeah. There was a recent study that came out that said that in the next couple of years we’re gonna see like tons and tons of e-waste specifically from AI. Part of that is because people want the latest and greatest. So essentially it’s like, whereas before you could use a computer for 10 years, now it’s like you always want the newest GPU, the most powerful.

And so you’ll switch it out every three years or two years. And so that really adds up. And also, we don’t have very good ways of recycling all of those electronic components like cell phones, computers, servers, what have you. It just takes a lot of effort and human time in order to kind of get all the different components. It’s definitely an imperfect process and a lot of that just goes to waste.

EL KALIOUBY: Goes to waste … because old chips are being replaced by new, more powerful chips.

So, assume your prompt asking for a thank you note to your dogsitter is processed by new, shiny chips that took a lot of power to make, from mining to manufacturing. Now, we’re ready for our next steps in the AI lifecycle: training and deployment of these AI models, which happens in data centers. These data centers aren’t only energy hungry. Sasha says they also need ..

LUCCIONI: Millions of liters of water. We don’t have any specific numbers per data centers, but for example, Microsoft did provide some numbers in general for Azure, and we’re really talking about the equivalent of like a small town. And data centers are actually huge. It’s like a football field situation filled with computers. It’s so loud. You can’t talk, you have to wear earplugs. It’s hot. Like if you put your hand near the actual servers, they’re like super hot.

They’re like, you can burn yourself hot. And then they have these pipes running through between the servers. And then cold water gets brought in and then it goes through these circuits. It’s actually like, it’s overwhelming. Like visiting a data center is a very massively overwhelming experience.

Copy LinkWhy cooling data centers is becoming a water challenge

EL KALIOUBY: Yeah. That’s crazy. Are there like creative innovations happening around alternative cooling mechanisms?

LUCCIONI: So the problem is that water is a really good coolant. That’s why it’s being used. But of course it’s scarce and there’s all sorts of issues, especially if you’re building data centers in Arizona or Texas. Like, there’s not a lot of water to begin with. But yeah, people are doing all sorts of work like improving the efficiency, like sometimes they use other liquids other than water.

Sometimes it’s air and essentially yeah, there’s a lot of innovation happening. But data centers are such huge investments and they take a lot of time to build. It’s not like a pair of shoes — you have to change the whole circuitry. And like for example, you can’t use sea water because the salt erodes the pipes or whatever. And so they are working on things, but it’s definitely gonna take a while.

EL KALIOUBY: So in the meantime, for the most part, it’s water cooling these data centers.

Before you can even enter your prompt asking for a thank you note to your dogsitter, the AI model needs to be trained on lots and lots of data. For example, Bloom – one of Hugging Face’s models – was trained on 46 natural languages and 13 programming languages – in total, 1.6TB of data!

Copy LinkWhat training emissions miss when we ignore the full lifecycle

Okay, so now let’s move on to the model training step. So we’ve got the GPUs, we’ve got the AI chips, and we’re gonna put them to work. What’s the data on emissions at this stage?

LUCCIONI: So essentially that’s kind of the bulk of where we have the information. We have numbers about training AI models. So we know that it goes anywhere from a couple of tons of CO2 for kind of smaller large language models to, you know, 25 tons of CO2 for a model like, for example, Bloom, which I worked on, and up to like 500 tons of CO2 for models that are bigger, trained for longer with like non-renewable energy essentially.

EL KALIOUBY: So I read in one of your reports the training an LLM with 213 million parameters is responsible for CO2 emissions roughly equivalent to the lifetime emissions of five cars. And then just for reference, GPT-3 has 175 billion parameters. That’s a lot of cars.

LUCCIONI: Yeah. So that was the initial number that was provided in a study by Emma Strubell and her colleagues. And since then we’ve been kind of getting more information about what are the factors that influence that number. For example, a couple of years ago we looked at what part of the overall footprint is like the actual GPUs doing the training and what part of it is like the overhead, like the heating and the cooling and the data transfer.

Because all of that actually also plays a role. Like we don’t think about it ’cause we seem to focus on like the actual active training part, but there’s like these huge data centers that have all this heating and then you have all the storage and the internet and all of that also adds up. And so we kind of did a lifecycle assessment, kind of like what we’re doing now. And we found that if you try to count all of this, then the resulting numbers will double from what you thought initially.

EL KALIOUBY: Okay. Let’s unpack that a bit. So yeah, it’s one thing to measure the CO2 emissions of the model as it’s being trained. But you’re saying there’s also additional considerations where if you’re drawing on data that’s stored in a different server, yeah. Like what are the other things that are happening while you’re training a model?

LUCCIONI: There is the storage, the input output. Now people will try to train across different cloud compute servers and so there’s a lot of data transfer and logistics going on. If you’re looking at one model trained on a specific cluster for a specific time, you can get an idea of these things, but nowadays so much of it is distributed, so much of it is—

EL KALIOUBY: Dynamic too, right?

LUCCIONI: Yeah, exactly. And so it’s so hard to get exact numbers for most of this.

Copy LinkWhere the biggest efficiency gains in AI may come from next

EL KALIOUBY: Yeah. Where do you think the innovation is headed? Is it in building these like smaller language models that require less compute and less data? I don’t really know if there’s a lot of innovation happening in these like ancillary parts of the problem.

LUCCIONI: I think the innovations are coming from different parts. Like people are working on smaller models, like the SmolLMs I mentioned. People are working on distillation, on different techniques that kind of make the model smaller for when you’re deploying them. People are working on specific hardware because the thing is GPUs, which are the ones that we use for most of AI training, weren’t actually made for AI.

They were made for graphics and video games. And so there are all sorts of people who are working on hardware more customized for AI, for both training and deployment, which I think is interesting as well. There’s all sorts of really interesting innovations going on actually.

EL KALIOUBY: Alright. I think also a lot of people think about training as a one and done thing. Like you train the model and then you’re done. But people don’t realize that once you ship a model, you’re onto the next, you’re continuously training the next version of this foundation model. Do you have any additional data on the emissions of this continuous training?

LUCCIONI: It is really hard to get any numbers. The thing is, for training, you can kind of, it’s relatively tractable because it’s like there’s only so many labs. There’s only so many people training models from scratch, right? But when it goes to like fine tuning or adapting, there’s so much going on.

And some people will make a major overhaul. Some people will just change the model slightly. And so there’s just a lot happening there. And for example, Allen AI recently released Olmo, their model, and I think they talked about like the different steps. So I think that it’s starting to be a thing where you don’t only talk about training, you also talk about fine tuning and you try to have more granular numbers. But overall, what we’ve seen so far is really like a huge focus on training only.

EL KALIOUBY: Yeah. Do you get a lot of pushback from companies sharing their data? Because I imagine with your AI energy scorecard, you can do that for inference, but not for training, right? Because training is kind of a, unless the company decides to share data with you, you can’t get access to that.

LUCCIONI: Yeah. And you can’t run trainings from scratch ’cause it takes too many resources. Yeah.

EL KALIOUBY: Now, we’ve added in the energy use for how an AI model is trained and finetuned. And it’s ready to power up and give you its best effort at writing that thank you note. This is the final stage of the lifecycle: deployment or inference. So training is one thing, but now like we’ve deployed these models, so every time you prompt a model, right? The question actually, for most models, a lot of the models are not local models. A lot of the models run in the cloud. So the question travels all the way to some data center somewhere and it gets cranking at the answer. And I guess you’re saying the energy cost there is not always obvious.

LUCCIONI: No, not at all. We don’t have any information about that. AI models, especially large language models, are seen as like products or commodities. And so people have given less and less information and nowadays they don’t even wanna say how many GPUs they used or how long the training time was. It is crazy because for me it’s not a secret, like why would it be so secretive? But yeah, companies are really cracking down sadly. And that creates a lot of like urban legends and back of the napkin calculations that then start having a mind of their own. Yeah, life of their own. Exactly. Like I try to avoid that. ‘Cause like the question that haunts my nightmares is like, how much energy does a ChatGPT query use? And the thing is, we don’t know. They’ve never given us a number and I don’t think they will. And most people use like a proxy number that somebody estimated. And now people are like, oh, that’s the actual number. And then it becomes like this whole snowball effect of like, oh, let’s compare ChatGPT to Google. It’s 10 times more. But both of those numbers, people just invented them. So like, what are we even talking about?

Copy LinkWhy inference costs are hard to measure but impossible to ignore

EL KALIOUBY: Different prompts and different AI tasks have different energy costs. So for example, if you are prompting ChatGPT for a dinner suggestion, that’s gonna be very different from an energy perspective than if you’re generating an image or a video. Or even like, my son’s been using all these AI tools to generate like research, right? So he uses OpenAI’s deep research agent. What kinds of deployment tasks cause the most emissions? And again, like do we even know?

LUCCIONI: Well, so in the work that I’ve done, we found that image generation tasks, like generation tasks in general, like generating new content is more energy intensive, uses more compute. Image generation is more energy intensive than text generation, which makes sense, because for an image you’ve got like pixels, you’ve got essentially a lot more data, whereas text is kind of more 2D. Of course the length definitely plays a role as much for the length of the input as the output.

But it’s interesting sometimes, like for a reasoning model, right? You’ll ask a question and then the input is short, but the output is super long. And then in the case of deep research, the input could be really long and then the output is like a sentence. But we don’t have enough numbers. I’m actually working on a follow up to Power Hungry Processing that’s gonna look specifically on input and output lengths. And I wanna figure it out because what we did in the first study was kind of to establish ranges for different tasks, but now we can go deeper and figure out like what the variance is.

EL KALIOUBY: That’s so fascinating. So, let’s say the prompt is complete, and so is the energy cycle that goes with it. You have your thank you note ready to send to your dogsitter. Mission accomplished.

We wish we could give you some neat and tidy answer – like that the thank you note equaled one light bulb running for 15 minutes and three bottles of water. But the reality is, we just don’t have the data we need to make those calculations. If this is important to you, let the companies behind your favorite AI tools know, and ask for answers!

After a short break, Sasha gives practical advice on how we can be more savvy users and looks towards a more hopeful future. Stay with us.

[AD BREAK]

Copy LinkHow to use generative AI more selectively in everyday life

Some of our listeners don’t wanna necessarily use ChatGPT because they don’t wanna be part of that environmental impact. What, how do we kind of reconcile the environmental impact? Like, I use ChatGPT for some like, really petty questions, right?

So Sasha, how do you tweak your use of AI given all these environmental considerations?

LUCCIONI: So essentially how I see it is I think about the tasks that I wanna do and think about alternatives that exist. Sometimes generative AI is really the only thing, like if you wanna generate a cute kitty picture with a specific context or a specific look or whatnot, like yeah, sure, like generative AI could be the way to go. But in other cases, the information’s already out there, so if you’re looking for a recipe, you can use a website or a book.

If you wanna answer a question that’s fairly straightforward, you can use Wikipedia, et cetera. But I totally understand people who use generative AI for more in depth stuff, like as you said, the deep research, it really makes sense. It’s like the reasoning aspect and it could actually really help you learn or understand a topic when you have this long chain of thought process. Personally, the one thing that I found really useful in terms of using, for example, ChatGPT, is like once I’ve written a paper, taking the abstract and brainstorming titles for the paper. And I’m so bad at coming up with titles, I just can’t be funny on demand essentially.

And then it could just be like, oh, add a pun, add a metaphor, add a cultural reference, and I find that really, really fun. But that’s pretty much my only usage. And for tasks like, I guess when you know AI or when you’ve thought about AI a lot, you start seeing types of tasks.

So there are types of tasks that are generative, like creating a new image or synthesizing research papers. That’s an inherently generative task. But finding information on the internet, like if I wanna know what species of bird or whatever, that’s an extractive task.

It doesn’t really need generative AI for that. And that’s when I started turning to, for example, Ecosia. It’s a search engine that only does kind of good old fashioned extractive search and they use renewable energy and whatnot. And I find that it’s kind of like liberating. It’s like, well, I will turn to generative AI when I actually need it, but I’m not gonna be using it as a go-to tool for everything I do.

EL KALIOUBY: You are in the AI space. You have this distinction between what tasks are optimal for generative AI and when is it really kind of an overkill to use AI to solve a specific task or ask a specific question. But for most people, they don’t know that distinction. And so they’re just using AI potentially for everything, not really recognizing the environmental impacts of doing so. Is there a way to like, visualize this for people who are just going to AI to solve every question they have?

LUCCIONI: Well in the Power Hungry Processing study, we found that using like a big generative AI language model for an extractive task, like answering a question like literally what’s the capital of France, can use like 20 to 30 times more energy than using just an extractive model, like the good old fashioned models that we used to use.

And so it’s kind of like if you wanna go from point A to point B, if you only wanna go to your local pharmacy, you can bike there, right? Or even an electric bike or whatever. Or walk. And then if you’re going further, sure, use a car. But also like what kind of car — and like making all these choices.

And I think it’s kind of like that with AI — thinking about the right tool for the right task is really important. And I know that a lot of people in their daily lives do take the environment into account. But I think we haven’t developed these habits for AI yet, and that’s something that we should start working on.

EL KALIOUBY: Yeah. You’ve also been doing some really interesting work on how AI can potentially and is potentially changing our behaviors, leading to more consumption, and that’s often referred to as Jevons’ paradox. Tell us more.

LUCCIONI: Yeah, it started out almost like a year and a half or two years ago. So Kate Crawford, a friend, and Emma Strubell, who wrote the original paper about AI’s climate impacts, also a friend. We were talking about this whole rebound effect and how Jevons’ paradox was observed in the 19th century by an economist who saw that as the use of coal was getting more efficient.

Like essentially you could get more energy from the same amount of coal. People were actually still using more coal because they were doing more things. Like before that they would be more frugal, I guess. They were limiting their use and now they’re just using it for everything. And we were like, well actually for AI we seem to see something similar.

People keep talking about how GPUs are getting more and more powerful, that you can train bigger and bigger models, that you can do more, but yet we’re still using more. And so we started talking about that and we started working on this paper. And the goal was to look at rebound effects and to look at, sure, there’s the direct emissions from AI, there’s the direct resource use, but if we open it up a little bit and we start thinking about how behaviors are changed, like do we travel more because we can get cheaper tickets with a recommendation engine? Or do we use ChatGPT more as opposed to like opening a book or looking on Wikipedia just because it’s accessible and free so we don’t see the cost of that? And so I think that a lot of our behaviors are being changed. Like for example, targeted advertising has gotten so good that we probably do buy more stuff just ’cause we’re like, oh, that Instagram ad, that’s exactly what I wanted.

But you wouldn’t have bought it like five years ago ’cause you didn’t get the ad. And so AI is actually changing our behaviors, changing our structures, and that all comes with environmental impacts that are hard to quantify and hard to really pinpoint. ‘Cause it’s like, for example, if I buy more stuff now because of targeted advertising on Instagram, would that be Instagram’s emissions? Would it be my emissions? So it’s really, really hard to get any numbers. But I think it’s really worth thinking about how AI’s influencing our behaviors.

EL KALIOUBY: The Stanford AI survey just got released and I remember reading a section where, like last year a lot of the AI usage was very kind of business applications. Like a lot of people were using it even as individuals, using it to solve work problems. And there’s an increased trend where people are using it for a lot more personal related stuff, right?

Like personal advice, coaching and whatnot. So I think that kind of ties into what you’re saying too, right? As it becomes more accessible, it’s actually increasing demand for the technology.

LUCCIONI: And there’s also really interesting other impacts that we mentioned, for example, like dematerialization. I was looking at, for example, e-readers versus books, right?

Like, it depends on how long you read the book for. Like e-readers can be more environmentally friendly, but on the other hand, you do have to produce the device. It’s such a complex issue. But it is really interesting, honestly, to think about it. And even if you don’t have any answers, at least you have some questions that can guide you in your everyday life.

Copy LinkWhen AI helps the climate and when it adds to the problem

EL KALIOUBY: There are a lot of folks out there that really believe that AI can be the silver bullet to mitigating climate change.

So, for example, AI is helping create better climate prediction models. Of course, it can help us solve some of the electric grid inefficiencies, and it’s also compelling some of the big tech giants to invest in more sustainable energy sources. So, is AI a net positive or negative?

LUCCIONI: This is a really hard question because the models that are the most harmful in terms of energy use or resource use are also the models that are least useful. So there’s really this weird trade off because most of the models you mentioned that do climate modeling, that do methane leak detection and biodiversity monitoring and all this cool stuff, are actually super efficient.

They’ll run on your laptop, they’ll run on a phone, on a Raspberry Pi. And yet they’re so powerful. And so for those algorithms, for those kinds of models, it’s not even a question — of course net net positive. Then you look at LLMs and chatbots and foundation models, or however people call them nowadays, they haven’t really been that useful in terms of like fighting climate change.

Like sometimes people will create chatbot assistants that can answer questions based on IPCC reports. Or nowadays people are trying to do some multimodal climate prediction, but intrinsically they have yet to prove their worth and yet they use so many natural resources.

So it’s really interesting because AI is such a broad term, it’s really not the same kind of tools on either side.

EL KALIOUBY: So everybody’s scrambling to power this explosion of generative AI. Is anybody doing it right?

LUCCIONI: I think that we should stop thinking about these like huge monolithic data centers that are in the middle of nowhere, essentially. We never see data centers ’cause they’re never close to cities, they’re hidden in the countryside. And like for example, I visited a data center here in Montreal and it’s actually a lot smaller.

It’s like maybe one one-hundredth of the size of like these big football field sized ones. But it’s underground in a university campus, the heat actually gets recuperated for heating. Here we have hydroelectric energy. So it’s green. And I was like, well, you know, maybe instead of having one massive data center that takes so much time and energy, et cetera, like why don’t we have a hundred smaller ones that we can integrate. And of course from like a logistical perspective, it’s easier just to have a massive data center. It’s just the way we do things. But if we make a little bit more effort, then we can really do things a lot more sustainably.

EL KALIOUBY: Yeah. Do you think we’ll get to a world where, because consumers can really drive behavior, right? Where they kind of prioritize green AI.

LUCCIONI: Yeah, definitely. I think that we underestimate our power as consumers, as users. We’re all like, oh, the cat’s out of the bag. But no, if we all collectively stopped using whatever, Google, ChatGPT, whatever, if we collectively woke up and were like, actually, no, that would make a big difference.

And I think that this thing of like it’s already done is of course something that benefits the companies themselves, ’cause you don’t really think about is it really done? Like do I actually have a choice? But once you start thinking about alternatives, you see that there are alternatives and you don’t need to take it as a given.

EL KALIOUBY: Well, thank you Sasha for joining us on the show. This was great.

LUCCIONI: Thank you for all those questions.

I learned a lot in my conversation with Sasha. Even though I’ve been in the AI space for over two decades, unpacking the energy lifecycle of AI in this way was so helpful to me.

Today most benchmarks for AI models are focused on accuracy – like if the model can pass the Medical Licensing Examination. But Sasha is calling for a different kind of benchmark. She’s created an energy scorecard for AI – one that quantifies the energy implications for models during training AND deployment.

As AI users, we have a choice – the same way we have choices about what car we drive or what food we eat. And when it comes to AI, we can make decisions on both the accuracy of the models, but also their energy consumption.