The important part remains internalizing emission costs into the price of electricity. Fussing over individual users seems like a distraction to me. Rapid decarbonization of electricity is necessary regardless of who uses it. Demand will soar anyway as we electrify transportation, heating, and industry.
If you are old enough you remember posting to Usenet and the warning that would accompany each new submission:<p><i>This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny]</i><p>Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.
> Tech companies like Meta, Amazon, and Google have responded to this fossil fuel issue by announcing goals to use more nuclear power. Those three have joined a pledge to triple the world’s nuclear capacity by 2025.<p>Erm ... that's a weird date considering this article came out yesterday. They actually pledge to triple the world's nuclear capacity by 2050[1]<p>There are a couple of weird things like that in this article, including the classic reference to "experts" for some of its data points. Still ... at least somebody's trying to quantify this.<p>[1] <a href="https://www.world-nuclear-news.org/articles/amazon-google-meta-and-dow-back-tripling-nuclear-goal" rel="nofollow">https://www.world-nuclear-news.org/articles/amazon-google-me...</a>
> The largest model we tested has 405 billion parameters, but others, such as DeepSeek, have gone much further, with over 600 billion parameters.<p>Very quickly skimming, I have some trouble taking this post seriously when it omits that the larger DeepSeek one is a mixture-of-experts that will only use 12.5% (iirc) of its components for each token.<p>The best summary of text energy use I've seen is this (seemingly more rigorous, although its estimates are consistent with the final numbers made by the present post): epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use<p>Estimates for a given response widely for a "typical" query (0.3 Wh; 1080 joules) and a maximal-context query (40 Wh; 144k joules). Assuming most uses don't come close to maximizing the context, the energy use of text seems very small compared to the benefits. That being said, the energy use for video generation seems substantial<p>I would be interested in seeing the numbers corresponding to how LLMs are typically used for code generation
This series of articles is driving me insane. The authors or editors are using inappropriate units to shock readers: billions of gallons, millions of square feet. But they are not putting the figures into context that the reader can directly comprehend. Because if they said the Nevada data centers would use 2% as much water as the hay/alfalfa industry in Nevada then the entire article loses its shock value.
The number of people in this comment thread defending this gargantuan energy footprint for a technology that currently in large measure is being used for a tremendous amount of dogshit things (and oceans of visual/text spam) is amusing considering the hysterics that this same energy use problem caused when it came to crypto.<p>I guess it becomes okay when the companies guzzling the energy are some of the biggest tech employers in the world, buttering your bread in some way.
There's a certain irony here in the fact that this page is maxing out my CPU on idle, doing some unclear work in javascript, while I'm just reading text.
Best article I've ever read about the energy needs of AI.<p>Impressive how Big Tech refuses to share data with society for collective decisions.<p>I'd also recommend the Data Vampires podcast series:<p><a href="https://techwontsave.us/episode/241_data_vampires_going_hyperscale_episode_1" rel="nofollow">https://techwontsave.us/episode/241_data_vampires_going_hype...</a><p><a href="https://techwontsave.us/episode/243_data_vampires_opposing_data_centers_episode_2" rel="nofollow">https://techwontsave.us/episode/243_data_vampires_opposing_d...</a><p><a href="https://techwontsave.us/episode/245_data_vampires_sacrificing_for_ai_episode_3" rel="nofollow">https://techwontsave.us/episode/245_data_vampires_sacrificin...</a><p><a href="https://techwontsave.us/episode/247_data_vampires_fighting_for_control_episode_4" rel="nofollow">https://techwontsave.us/episode/247_data_vampires_fighting_f...</a>
I believe we're currently seeing AI in the "mainframe" era, much like the early days of computing, where a single machine occupied an entire room and consumed massive amounts of power, yet offered less compute than what now fits in a smartphone.<p>I expect rapid progress in both model efficiency and hardware specialization. Local inference on edge devices, using chips designed specifically for AI workloads, will drastically reduce energy consumption for the majority of tasks. This shift will free up large-scale compute resources to focus on truly complex scientific problems, which seems like a worthwhile goal to me.
I found this bit interesting:<p>> In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023.<p>As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly <i>not</i> generative AI.
When companies make ESG claims, sensible measurement and open traceability should always be the first proof they must provide. Without these, and validation from a credible independent entity such as a non-profit or government agency, all ESG claims from companies are merely PR puff pieces to keep the public at bay (especially in "AI").
What’s the net energy footprint of an employee working in an office whose job was made redundant by AI? Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. In other words, let’s calculate the “economic output per energy unit expended.”<p>On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?
Solving climate change will take a lot of energy.<p>I found this article to be a little too one sided. For instance, it didn’t talk about the 10x reductions in power achieved this past year — essentially how gpt4 can now run on a laptop.<p>Viz, via sama “The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use. You can see this in the token cost from GPT-4 in early 2023 to GPT-4o in mid-2024, where the price per token dropped about 150x in that time period. Moore’s law changed the world at 2x every 18 months; this is unbelievably stronger.”
<a href="https://blog.samaltman.com/three-observations" rel="nofollow">https://blog.samaltman.com/three-observations</a>
The brain uses 20% of the human body's energy.<p>I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment.<p>But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially.<p>At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.
- "4.4% of all the energy in the US now goes toward data centers"<p>- "by 2028 [...] AI alone could consume as much electricity annually as 22% of all US households."<p>What would the 22% be if compared against all US energy instead of just all US household?
Energy use is going to continue to climb.<p>I'm worried about the environmental impacts of this, but from everything I've seen society values model output more. Curious to watch this over the rest of the decade.
What's the energy imprint of the human doing the same work that AI is now going to do? If AI's imprint is less, what is the right thing for us to do?
We really need to improve the power grid. I don't think about "A. I." very much, but I am glad that <i>something</i> is making us upgrade the grid.
> When you ask an AI model to write you a joke or generate a video of a puppy, that query comes with a small but measurable energy toll and an associated amount of emissions spewed into the atmosphere. Given that each individual request often uses less energy than running a kitchen appliance for a few moments, it may seem insignificant.<p>> But as more of us turn to AI tools, these impacts start to add up. And increasingly, you don’t need to go looking to use AI: It’s being integrated into every corner of our digital lives.<p>Forward looking, I imagine this will be the biggest factor in increasing energy demands for AI: companies shoving it into products that nobody wants or needs.
40% of electricity consumption in Virgina will be data centers in 2030?<p>Table A1 , PDF page 29:<p><a href="https://www.epri.com/research/products/000000003002028905" rel="nofollow">https://www.epri.com/research/products/000000003002028905</a>
>you might think it’s like measuring a car’s fuel economy or a dishwasher’s energy rating: a knowable value with a shared methodology for calculating it. You’d be wrong.<p>But everyone knows fuel economy is everything but a knowable value. Everything from if it has rained in the past four hours to temperature to loading of the vehicle to the chemical composition of the fuel (HVO vs traditional), how worn are your tires? Are they installed the right way? Are your brakes lagging? The possibilities are endless. You could end up with twice the consumption.<p>By the way, copy-pasting from the website is terrible on desktop firefox, the site just lags every second, for a second.
Today Google launched a model, Gemma 3n, that performs about as good as SOTA models from 1-2 years ago that runs locally on a cell phone.<p>Training SOTA models will, like steel mills or other large industrial projects, require a lot of environmental footprint to produce. But my prediction is that over time the vast majority of use cases in the hands of users will be essentially run on device and be basically zero impact, both in monetary cost and environment.
I find it weird that on the whole there wasn't as huge a pushback against AWS and Google and Facebook (by far the biggest fleets of data centers) for power and carbon emissions over the last couple decades. Maybe they managed propaganda better about paying for future renewable resources or how low their PUEs are.<p>Turning 4% of the U.S.'s electricity into cat videos and online shopping and advertisements and heat already sounds like a lot of use. Maybe the rapid rise of AI use is what's alarming people?<p>Contrast the 2016 study[0] of data center energy use where use was recently flat because of efficiency improvements in 2010-2020 but historically there was a ton of growth in energy consumption since ~1990; basically we have always been on a locally exponential growth curve in data center energy use but our constant factors were being optimized by the hyperscalers in that 2010-2020 period.<p>We also need to compare the efficiency of AI with other modes of computation/work. The article goes into detail on the supposed actual energy use but there's a simple metric; All the large companies provide costs per unit of inference which can put a hard ceiling on actual energy cost. Something like $20/1M tokens for the best models. METR used a 2M token budget. So you can currently price out N hours of work at $40 from whichever latest METR benchmarks come out and have a worst case cost for efficiency comparison.<p>Lastly, if we're <i>not</i> on a trend toward having Dyson swarms of compute in the long run then what are we even doing as a species? Of course energy spent on compute is going to grow quadratically into the distant future as we expand. People are complaining about compute for AI but compute is how we figure things out and get things done. AI is the latest tool.<p>[0] <a href="https://eta.lbl.gov/publications/united-states-data-center-energy" rel="nofollow">https://eta.lbl.gov/publications/united-states-data-center-e...</a>
> The new model uses [energy] equivalent to riding 38 miles on an e-bike... AI companies have defended these numbers saying that generative video has a smaller footprint than the film shoots and travel that go into typical video production. That claim is hard to test and doesn’t account for the surge in video generation that might follow if AI videos become cheap to produce.<p>"Hard to test", but very obviously true if you make any attempt at guessing based on making a few assumptions... like they seem comfortable doing for all the closed source models they don't have access to being run in conditions they're not testing for. Especially considering they're presenting their numbers as definitive, and then just a couple paragraphs down admit that, yeah, they're just guessing.<p>Regardless, I know for a fact that a typical commercial shoot uses way more energy than driving across the TMZ in <i>an e-bike</i> (considering they're definitely using cars to transport gear, which gives you less than 4 miles for the same energy).
> unprecedented and comprehensive look at how much energy the AI industry uses<p>Not sure about comprehensive claim here if end-to-end query chains were not considered.<p>For example the mobile wireless node (that're being used by the majority of the users) contribution to the energy consumption are totally ignored. The wireless power amplifier or PA for both sides of users and base-stations are notorious for their inefficiency being only less than than 50% in practice although in theory can be around 80%. Almost all of the current AI applications are cloud based not local-first thus the end users energy consumption and contribution are necessary.
Interesting, thanks for sharing! I share some concerns others have about this piece, but I’m most shocked about their finding that image generation is cheaper than text. As someone who’s gone down this rabbit hole multiple times, this runs against every single paper I’ve ever cited on the topic. Anyone know why? Maybe this is a recent change? It also doesn’t help that multimodal transformers are now blurring the lines between image and text, of course… this article doesn’t even handle that though, treating all image models as diffusion models.
I ponder this a lot, but the interface of "MIT technology Review" is unbearably overdesigned, its got that annoying narrow smartphone format where you can't zoom out, and then all these fancy graphics. Can't we have crisp, easy-to-read HTML? The format annoyed me so much I didn't read the article because this kind of design makes me doubt the source. Alas
With all the issues and inefficiencies listed, there is a lot of room for improvement. I'm hopeful that just as the stat they give for data center energy not rising from 2005-2017, so to will the AI energy needs flatten in a few years. GPUs are not very efficient. Switching to more task specific hardware will provide more efficiency eventually. This is already happening a little with stuff like TPUs.
Well, this was disappointing:<p>> There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o.<p>Otherwise this is an excellent article critiquing the very real problem that is opacity of these companies regarding model sizes and deployments. Not having an honest accounting of computing deployed worldwide <i>is</i> a problem, and while it's true that we didn't really do this in the past (early versions of Google searches were undoubtedly inefficient!), it's not an excuse today.<p>I also wish this article talked about the compute trends. That is, compute per token is going significantly down, but that also means use of that compute can spread more. Where does that lead us?
I would like to see more data centers make use of large-scale Oil Immersion-Cooling. I feel like the fresh water use for cooling is a huge issue.<p><a href="https://par.nsf.gov/servlets/purl/10101126" rel="nofollow">https://par.nsf.gov/servlets/purl/10101126</a>
I wonder how the energy requirements are distributed between training and inference. Training should be extremely flexible, so one can only train when the sun shines and nobody uses the huge amount of solar power, or only when the wind turbines turn.
the numbers in the article are all over the place. I mean the article seems to try and some of the more general calculations on paper should work out but especially the image gen ones I can sorta disprove with my own experiences in local gen.<p>Even were it matches sorta (the 400 feet e-bike thing) that only works out for me because I use an AMD card. An NVIDIA card can have several times the generation speed at the same power draw so it all falls down again.<p>And the parameters they tried to standardize their figures with (the 1024x1024 thing) is also a bit meh because the SAME amount of pixels in a different aspect ratio can have huge variations in gen speed and thus power usage. for instance for most illustrious type checkpoints the speed is about 60% higher at aspect ratios other than 1024x1024. Its all a bit of a mess.
Seems like a solved problem. We have an unlimited sources of clean power from nuclear and solar. China alone now has almost 1TW of solar capacity.<p>Build more nuclear, build more solar. Tax carbon.
Weird I was assured that Bitcoin would be using all of the worlds electricity by now.<p>Which I already thought was odd, because London would need all that electricity to see through the giant mountain of poop piled up by all the horses the british use for transportation.
As more and more people use brute force loops to make their AI agents more reliable, this hidden inference giant will only continue to grow.
This is why I put my framework together, using just 2 passes as opposed to n+ can increase accuracy and reliability by a far greater amount than brute force loops while using significantly less resources.
Supportive evidence and data can be found in my repo: <a href="https://github.com/AutomationOptimization/tsce_demo">https://github.com/AutomationOptimization/tsce_demo</a>
I work in DCO, thats Data Center Operations if you’re not aware. I’ve tried explaining the amount of power used to my elderly mom; it isn’t easy! But here’s my best take:<p>The racks I am personally responsible for consume 17.2kW. That’s consistent across the year; sure things dip a bit when applications are shut down, but in general 17.2kW is the number. Presuming a national average of 1.2kW per home, each rack of equipment I oversee could potentially power 14 houses. I am responsible for hundreds of these racks, while my larger organization has many thousands of these racks in many locations worldwide.<p>I’ve found no other way to let the scale of this sink in. When put this way she is very clear: <i>the price isn’t worth it to humanity</i>. Being able to get, say, Door Dash, is pretty neat! But not at the cost of all our hoarded treasure and certainly not at the cost of <i>the environment on the only planet we have access to</i>.<p>The work done by AI will only ever benefit the people at the top. Because to be frank: they won’t share. Because the very wealthy have <i>hoarding disorder</i>.
This gives me the silly idea to go try to measure the power consumption of the local data center by measuring the magnetic field coming off the utility lines.
Shameless plug . . . I run a startup who is working to help this <a href="https://neuralwatt.com" rel="nofollow">https://neuralwatt.com</a> We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.
From "The Limits to Growth" to peak oil and beyond: As a rule, scarcity doom and neo-malthusianism hasn't played out the way proponents claim. I've been very critical of the AI hype cycle here and elsewhere, but this isn't it. In the long run technological advancements increase our productivity and quality of life.<p>Yes, much of what is being promoted is slop. Yes, this bubble is driven by an overly financialized economy. That doesn't preclude the possibility of AI models precipitating meaningful advancements in the human condition.<p>From refrigeration to transportation, cheap and abundant energy has been one of the major driving forces in human advancement. Paradoxically, consuming cheap energy doesn't reduce the amount of energy available on the market. Instead it increases the size of the market.
We shouldn't be hand wringing on energy usage, we should be generating energy too cheap to bother metering and too clean to bother thinking about. Nuclear baseline, renewables everywhere they make sense, we know how to do this we just need to do it.
> This leaves even those whose job it is to predict energy demands forced to assemble a puzzle with countless missing pieces, making it nearly impossible to plan for AI’s future impact on energy grids and emissions. Worse, the deals that utility companies make with the data centers will likely transfer the costs of the AI revolution to the rest of us, in the form of higher electricity bills.<p>... So don't? Explicitly shift the cost to the customer.<p>If I want to hook up to the energy grid with 3-phase power, I pay the utility to do it.<p>If a business wants more power and it isn't available, then the business can pay for it.<p>Then only businesses that really need it will be willing to step up to the plate.<p>No amount of "accounting" or "energy needs prediction" will guard against regulatory capture.
Might have missed it but was disappointed to see no mention of externalized costs like the scraping burden imposed on every IP-connected server. From discussions on HN this sounds quite substantial. And again, why exactly should the few AI companies reap all the value when other companies and individuals are incurring costs for it?
Jesus, who writes this stuff?<p>> AI is unavoidable<p>> We will speak to models in voice mode, chat with companions for 2 hours a day, and point our phone cameras at our surroundings in video mode<p>This is surely meant to be an objective assessment, not a fluff piece.
The point that stood out to me as concerning was<p><i>"The carbon intensity of electricity used by data centers was 48% higher than the US average."</i><p>I'd be fine with as many data centers as they want if they stimulated production of clean energy to run them.<p>But that quote links to another article by the same author. Which says<p><i>"Notably, the sources for all this power are particularly “dirty.” Since so many data centers are located in coal-producing regions, like Virginia, the “carbon intensity” of the energy they use is 48% higher than the national average. The paper, which was published on arXiv and has not yet been peer-reviewed, found that 95% of data centers in the US are built in places with sources of electricity that are dirtier than the national average. </i>"<p>Which in turn links to <a href="https://arxiv.org/abs/2411.09786" rel="nofollow">https://arxiv.org/abs/2411.09786</a><p>Which puts the bulk of that 48% higher claim on<p><i>"The average carbon intensity of the US data centers in our study (weighted by the energy they consumed) was 548 grams of CO2e per kilowatt hour (kWh), approximately 48% higher than the US national average of 369 gCO2e / kWh (26)."</i><p>Which points to
<a href="https://ourworldindata.org/grapher/carbon-intensity-electricity?tab=chart&time=latest..2023&country=EU-27~KHM~OWID_UMC~USA" rel="nofollow">https://ourworldindata.org/grapher/carbon-intensity-electric...</a><p>For the average of 369g/KWh. That's close enough to the figure in the table at <a href="https://www.epa.gov/system/files/documents/2024-01/egrid2022_summary_tables.pdf" rel="nofollow">https://www.epa.gov/system/files/documents/2024-01/egrid2022...</a><p>which shows 375g/KWh (after converting from lb/MWh)<p>But the table they compare against shows.<p><pre><code> VA 576g/KWh
TX 509g/KWh
CA 374g/KWh
</code></pre>
and the EPA table shows<p><pre><code> VA 268g/KWh
TX 372g/KWh
CA 207g/KWh
</code></pre>
Which seem more likely to be true. The paper has California at only marginally better than the national average for renewables (Which I guess they needed to support their argument given the number of data centers there)<p>I like arxiv, It's a great place to see new ideas, the fields I look at have things that I can test myself to see if the idea actually works. I would not recommend it as a source of truth. Peer review still has a place.<p>If they were gathering emissions data from states themselves, they should have caclulated the average from that data, not pulled the average from another potentially completely different measure. Then their conclusions would have been valid regardless what weird scaling factor they bought in to their state calculations. The numbers might have been wrong but the proportion would have been accurate, and it is the proportion that is being highlighted.
It's hard to believe that as a society we are prioritizing AI queries (which are fine, nothing wrong with them in principle, and they do increase productivity in some cases) over the future wellbeing of our species (and planet). Winter is Coming, and these big companies and their investors couldn't care less.<p>I can't decide whether they think it won't be that bad, or the scientific forecasts are wrong, or that they just don't care because whatever turmoil results from serious climate change, they'll be able to rise above it, and well, fuck the rest of humanity.<p>I'm disgusted by our sense of priorities. (Though maybe I shouldn't be since we live in a country that values subsidizing the industrial-military complex over the health and education of its citizens.)
> ...and many buildings use millions of gallons of water (often fresh, potable water) per day in their cooling operations.<p>This is outrageous. People still struggle to access fresh water (and power), but hey "sustainability is all to our company" is always promoted as if something nice is being done on from the behemoth's sides. BS. What a waste of resources.<p>I truly condemn all this. To this day I do still refuse to use any of this technology and hope that all this ends in the near future. It's madness. I see this as nothing more than next-gen restrictive lousy search engines, and as many have pointed out ads are going to roll soon. The more people adopt it the worse will be for everyone.<p>I always emphasize this: 10-15 years ago I could find everything through simple web searches. Everything. In many cases even landing on niche and unexpected but useful and interesting websites. Today that is a difficult/impossible task.<p>Perhaps there is still room for a well-done traditional search engine (haven't tried Kagi but people in general do say nice things about it) to surface and take the lead but I doubt it, when hype arrives especially in the tech industry people follow blindly. There are still flourishing "ai" startups and from night to day everyone has become a voice or expert on the subject. Again: BS.<p>Traditional web engines and searches were absolutely just fine and quite impressed with their outputs. I remember it. What the heck has happened?
The fact that none of these companies want to tell you how much power they're using should be enough to reason that it's utterly horrible. If it were anywhere reasonable, they'd be boasting about how low the number is.