New
articleMigrating a 20-Year WordPress Blog to Be Agent-Readytechnology bytesAGI Has Arrived. Now Give It a Simpler Problem.podcastEpisode 100: The Next Era of AI — From Agents to Enterprise IntelligencearticleMigrating a 20-Year WordPress Blog to Be Agent-Readytechnology bytesAGI Has Arrived. Now Give It a Simpler Problem.podcastEpisode 100: The Next Era of AI — From Agents to Enterprise Intelligence
Navveen Balani Making systems intelligent, efficient & responsible

Day 2 · The Invisible Bill

View as markdown

Every AI response has a physical cost — energy, water, carbon, hardware, money. Once you can see that bill, every other decision in the stack gets sharper.

A single conversation with a large language model — roughly 20 to 50 prompts — uses about half a liter of fresh water in data center cooling. A 500 ml bottle, drunk by the servers, every time you finish a working session with a model.

A typical image generation uses about as much energy as charging your phone halfway.

The training run for one frontier model carries the carbon weight of hundreds of round-trip transcontinental flights.

And here is the number that changes the shape of the opportunity: inference now uses more energy, water, and carbon over a model’s lifetime than training ever did. Training is the single, dramatic moment. Inference is the daily one — present in every query, every token, every retry, every agent loop. It is already the larger number, and the curve has barely started.

The picture grows richer still. Humans send one query at a time. Agents send ten — to plan, retrieve, reason, verify, and try again. As agentic systems move into production, every human request expands into a cascade of machine requests. The invisible bill compounds with users, and now compounds with the agents working alongside them.

Underneath all of this is the hardware itself. Every GPU your model runs on carries a manufacturing footprint paid before the first inference — often 20-40% of its lifetime emissions, embedded in mining, fabrication, and shipping. A GPU running at 15% utilization uses nearly the same power as one running at 85%, with much of its capacity still available. A GPU retired at three years rather than seven leaves embodied carbon on the table that could have been amortized further. A data center built for peak load operates at peak-load cooling even when capacity is open.

A meaningful share of the full bill — across software and hardware — is waste, which means it is also the biggest opportunity. Retried calls. Oversized models on small tasks. Regenerated answers a cache could have served. Agent loops that ran twelve times when one was enough. Idle GPUs. Hardware retired with life still in it. Waste is the largest line on the bill, and the most rewarding to recover, once it becomes visible.

A note on the numbers above: every figure is an approximation, and the published estimates vary widely. The point of this newsletter is not to defend any single number — it is to show that every one of these costs is real, growing, and currently outside the view of the people best placed to act on them.

These numbers do not yet appear on the model card. They are not yet in the API response. They have not yet made it onto the cloud bill, the sustainability dashboard, or the engineering ticket.

They are real. They are growing. And they are, today, waiting to be made visible.

That is the bill this newsletter is about — and today is the day we begin making it visible.

Every AI response carries five costs at once: energy, water, carbon, hardware, and money. Most of them travel quietly today — and most of them are met not at training time, but every single time a human, or an agent, asks the system to think. Efficient AI begins the moment all five become visible at the point of decision.

The reason most AI systems can run lighter than they do today is not that anyone is being careless. It is that only one of the five costs — money — shows up on a dashboard the team already has open. The other four travel quietly with every call, every token, every image, every embedding, every GPU hour.

When you pick a model, you see latency and accuracy. The fifth axis — energy per token — is the one waiting to be added. When you ship a feature, you see request volume. The water it draws is the next layer of the same picture. When you scale to production, you see infrastructure spend. The carbon curve runs alongside it, ready to be plotted. When you provision hardware, you see capex and capacity. The embodied emissions, utilization headroom, and lifecycle value are the next levers in the same toolbox. When you sign the cloud bill, you see total cost. The four costs inside it are visible the moment you choose to look.

What you can see, you can weigh. What you can weigh, you can improve. The discipline of efficient AI starts with one move: bringing the four quiet costs into the same field of view as the visible one.

  • Engineering. Every line of code that calls, consumes, tests, debounces, retries, caches, or renders the output of an AI system is a line on the invisible bill. Whether you pick the model, build the interface around it, run it through CI, or ship it to a mobile device — you have a lever. Every one of those levers is knowable, and once measured, becomes a third axis to optimize on alongside latency and accuracy.
  • Platform / Infrastructure. You run the hardware AI lives on. Your region, your instance type, your utilization, your PUE, your refresh cycle, your decommissioning path — those are levers nobody else can pull. The invisible bill is largely written here, in both energy and embodied carbon, which makes it largely rewritable here.
  • Architecture / CTO. Your AI spend curve, your AI carbon curve, and your AI hardware curve are the same line drawn three times. Make one visible and you have made all three visible — to finance, to sustainability, to the board.
  • Sustainability / ESG. AI is about to be your largest and fastest-growing line item — across Scope 2 (operational energy) and Scope 3 (embodied hardware emissions). It is also the line item most ready to be measured well from day one. The teams shipping it want to help you measure it; they just need to know the unit.
  • Business. The unit cost of intelligence is falling. The volume is rising ten times faster. Knowing the full bill per feature — software and the hardware it runs on — is what tells you which AI investments scale beautifully today, and which scale even more beautifully once their full bill is understood.

Five seats, one bill, one conversation.

One thing to do this week: Pick one AI feature or agent workflow you have shipped or are about to ship. Write down its five costs — energy, water, carbon, hardware footprint, money — even as rough estimates. Vendor figures, public model cards, utilization dashboards, and back-of-envelope math all count. Precision matters less than presence.

Then share the five numbers with the five other seats. The conversation that follows is the one this newsletter exists to start.

Every AI response carries five costs — software and hardware. Four of them are still quiet today. Efficient AI begins the day all five sit on the same page.

Coming up next. The five costs do not behave the same way. One scales with users, one scales with geography, one scales with time of day, one scales with the model you chose, and one was met before the system ever ran. Knowing which lever moves which cost is the next principle.

See you in the next issue.