Green AI Bytes

Day 25 showed that the interface between a user and a model decides how much inference gets called. Today closes the arc at the newest interface in the stack — the one between agents — where fan-out, escalation, accumulated state, and loose contracts compound into a bill no single team owns end to end.

Days 21 to 24 closed the embodied arc. Every AI system now has a widened ledger: operational carbon on one side, hardware, model, and dataset receipts on the other. Today's issue turns to what creates demand on that ledger. A user creates intent. The…

Day 23 opened the model's environmental receipt: every model in production carries the footprint of the training run that produced it, spread across every inference it serves. Today's issue turns to what came before the training run. Every AI model remembers…

Day 22 opened the first embodied category by putting the silicon in the rack on the ledger, and named the two levers behind it: how long the hardware runs, and the grid it runs on. Today's issue turns to the second embodied category. The model itself. Every…

Day 21 opened the embodied carbon arc by drawing the boundary between operational emissions (what the system draws while running) and embodied emissions (what the system cost to exist). That framing named three embodied categories: hardware, training, and…

Day 20 closed the arc on the system around the model with a single frame: once capacity is standing, the bill runs on wall-clock, and utilization becomes the number that matters. That frame covered operational emissions — the electricity, cooling, and compute…

Day 19 covered the compute the team triggers itself — evals on merge, LLM-as-judge sweeps, notebook experiments — and walked the Meter, Gate, Sample, Substitute ladder that moves it from an unbudgeted reflex into a governed line on the bill. That closes the…

Day 18 covered the data that AI systems leave behind — vector stores, indexes, embeddings that outlive the features that created them. That was the persistence side of the invisible bill. There is a compute side to it as well, and it fires before a feature…

Day 17 examined what every call carries on the way in — the system prompt, the tool catalog, the examples, the payload that pays a bill on every request whether it needs to or not. Today's issue steps back from the call itself. Every AI feature leaves…

Day 16 traced the compute that safety adds on the outside of every call. Day 17 opens the same call up and looks at what is actually inside it — the request the model reads, not the message the user sent. The gap between the two is usually enormous. A user…

Day 15 covered the routing discipline — the choice of which model answers a request, and the four-rung ladder of Filter, Classify, Route, Call that decides whether a call should ever reach a heavyweight model at all. That settles one piece of the system…

The model is usually the most expensive component in an AI system. The compute it consumes, the memory it occupies, and the energy it draws all converge on a single moment: inference. Yet in many production systems, every request still reaches the model. That…

Most production AI systems run in a region somebody picked years ago, on hardware procured for a different workload, and on schedules chosen for operational convenience rather than carbon performance. None of those choices were carbon-aware. Most of them are…

Most inference traffic is not waiting on a human. It is waiting on a loop. Embeddings for an index that refreshes hourly. Classifications for a moderation queue. Summaries for a digest that ships at seven in the morning. Evaluations that run overnight. None…

Days 8–11 worked the loop itself — its turn count, its route, its tool shape, its context weight. That closes the loop chapter. This issue opens a new one. The next three days work on the platform underneath the loop, so the same work costs less every time it…

Day 8 made the loop the unit of work. Day 9 cut turns by planning the route. Day 10 cut turns by shaping the tool. Days 8–10 all worked on the same lever — fewer turns per task. This issue works on a different one. When the turns you do run carry more weight…

Day 8 made the loop the unit of work and showed why each extra turn costs more than the one before it. Day 9 cut turns at the front by planning the route. This issue cuts turns at every step by changing what the agent calls when it gets there. Here is the…

Day 8 made the loop the unit of work and showed why it costs more than the calls inside it: every turn re-reads everything before it, so cost climbs faster than the number of turns the loop takes — closer to the square of the turn count than to a straight…

Most teams optimise what goes into an AI call. Few optimise what comes back — even though output tokens cost roughly three to five times what input tokens cost, across every major provider. The return trip of a call is where the meter runs fastest, and it is…

Day 3 was about right-sizing the model — picking the smallest technology that does the job. This is the same discipline, one level up, applied to an agent loop: a model that plans, calls a tool, reads the result, decides again, and keeps going — sometimes…

Most production prompts are not written. They are assembled. A user asks a question. Before the model sees anything, a retrieval layer runs — pulling rows from a database, passages from a document store, results from a search index, nodes from a knowledge…

Published on --- A surprising amount of AI waste hides inside calls you already decided to make. Most production prompts are two to ten times longer than necessary — packed with instructions the model already knows, examples that no longer matter, and context…

The cheapest, fastest, lowest-carbon inference is the one the system never made. Providers already reduce some waste below the API line through prefix caching. But the biggest opportunity lives above it — in calls the application never needed to make in the…

The smallest technology that solves the task is the right one — sometimes a frontier LLM, often a small one, sometimes a classical model, and sometimes no AI at all. Once you choose by task instead of by default, every cost on the bill moves with you. A…

Every AI feature ships an invisible bill. Cost. Energy. Water. Carbon. Hardware. Most teams see one. Almost none see the full picture. Welcome to Green, Efficient AI — a newsletter about building AI that is intelligent in output and efficient by design. ---…

Every AI response has a physical cost — energy, water, carbon, hardware, money. Once you can see that bill, every other decision in the stack gets sharper. A single conversation with a large language model — roughly 20 to 50 prompts — uses about half a liter…