---
title: Day 7 · Shape the Answer Before You Ask
type: newsletter
date: 2026-05-31
source: linkedin
summary: Most teams optimise what goes into an AI call. Few optimise what comes back — even though output tokens cost roughly three to five times what input tokens cost, across every major provider. The return trip of a call is where the meter runs fastest, and it is…
newsletter: Green AI Bytes
draft: false
---

Most teams optimise what goes into an AI call. Few optimise what comes back — even though output tokens cost roughly three to five times what input tokens cost, across every major provider. The return trip of a call is where the meter runs fastest, and it is the layer where the least design effort has been spent.

Days 3 through 6 worked on the trip in. Choose the right model. Make fewer calls. Trim what goes inside the call. Make the retrieval layer earn its tokens. All of that decides what reaches the model. None of it decides what the model sends back.

The shape of an answer is set in one of two places. Either the team decides it before the call and tells the model what to return — schema, structured-output mode, function signature, a single line of instruction. Or the team does not decide, the model returns prose, and someone downstream pays to wrestle that prose into a shape the system can use. The choice between those two places is the entire principle.

A team builds an invoice-processing pipeline. The model reads a document and returns three fields: vendor, date, total. In the demo, every field arrives clean. The pipeline passes review.

In production, the answer comes back wrapped in a sentence — *"I found the following details on this invoice…"* — and the total arrives sometimes as digits, sometimes as words, sometimes with a currency symbol, sometimes without, occasionally with a footnote about VAT. The downstream parser cracks on the third edge case. The fix that ships, under deadline pressure, is a second model call whose only job is to turn the first call's answer into clean JSON.

The pipeline now makes two model calls for every invoice. The second one produces no new information.

> It exists only to repair the shape of the first — a shape the first call would have returned for free, had anyone asked. At three-to-five times the input rate, the output side of that second call is the most expensive line on the bill, and it is pure cleanup.

The team eventually moved the formatting contract into the first call: a schema specifying vendor as a string, date as ISO 8601, total as a number, currency as a three-letter code. The second call was deleted. The invoice volume did not change. The model did not change. Half the calls on that endpoint went away.

Three patterns produce this kind of waste across nearly every team that has shipped an AI feature, and all three feel like reasonable defaults:

**The free-text default.** The first thing a model returns is prose, because that is what the most accessible API call gives you. Structured-output modes exist on every major provider, but they have to be asked for explicitly. Teams that ship fast inherit prose and pay for it on every call afterwards.

**The "we'll parse it later" reflex.** The system handles whatever comes back by writing a regex, then a more careful regex, then a parser, then a fallback model call when the parser fails. Each layer of cleanup feels cheaper than rewriting the prompt. Together they cost more than the rewrite would have.

**The discarded reasoning.** The prompt asks for a label and an explanation. The system uses only the label. Every call generates several sentences of reasoning that nobody reads — billed at the premium output rate, on every request, forever.

The model gives you exactly the shape you ask for, and exactly the shape you forget to ask for. The bill reflects both.

### The structured-output question

A fair objection: most major providers now offer JSON mode, structured outputs, or function calling as a built-in feature. If the platform handles the shape, is the discipline still the team's problem?

The feature solves the syntax, not the contract. JSON mode guarantees that what comes back is parseable. It does not guarantee that the fields are the ones the system needs, in the units the system expects, at the level of detail the system can actually use. A perfectly valid JSON response can still have a as a string, a date in the wrong format, or a field with a courtesy salutation attached. The parser succeeds. The downstream logic still fails.

Schema-bound structured outputs go further — they constrain the model to the field types declared, and where the feature is available, the team should be using it. But a schema that asks for a three-sentence on every call, when no downstream system reads the explanation, is a perfectly valid schema and an expensive one. The platform feature gets the team to a parseable shape. The principle gets the team to the shape the system actually needed.

The operating principle on the return trip of the call is a four-step hierarchy:

**Shape → Cap → Validate → Repair.**

*Shape* the output before the call. State the format the downstream system needs — a schema, a structured-output mode, a function signature, a single line of instruction (*"return only the three-letter currency code"*). The contract is set on the way in, so there is nothing to negotiate on the way out. *Cap* the length. Set max output tokens. Add a stop sequence. Ask for *"one sentence"* where one sentence is what the downstream system will consume. At three-to-five times the input rate, every uncapped output is a meter running without a needle. *Validate* what came back against the contract — types, ranges, required fields, allowed values — before any downstream system acts on it. Catch a malformed answer where it is cheap to catch. *Repair* locally when validation fails. Fix the formatting in code, or retry the smallest possible slice of the call. Never re-run the whole call to clean up the shape of its own output.

The first rung is the most often skipped. Teams reach for output cleanup before they have asked whether a schema, a function signature, or a one-line format instruction would have made the cleanup unnecessary. Shaping the answer in the prompt is the cheapest output optimisation available, and it is the one nobody talks about because it does not feel like an optimisation. It feels like prompt writing.

The second rung is the most often mis-set. Default max-token settings on most platforms are generous enough that the model almost never reaches them in practice, which means they are not constraining anything. A cap that never binds is decoration. Set it to the longest valid answer the downstream system will actually consume.

**Engineering.** Measure two numbers per call: total output tokens generated, and total output tokens actually consumed by the downstream system. The gap is your output waste. Move the formatting contract into the prompt or into a schema. Cap max tokens to the longest valid answer, not to the platform default. Add a validation layer between the model and anything that depends on its answer.

**Platform / Infrastructure.** Output tokens are the most expensive tokens on the bill and the most invisible — they are generated, not retrieved, so they do not appear on any storage or index dashboard. Track output tokens per request as a first-class metric, alongside prompt tokens and retrieved tokens. The three numbers together tell the full story of a call. If you only see two of them, the most expensive one is the one hiding.

**Architecture / CTO.** Output contracts are part of the system's interface, not a property of a prompt. They deserve the same discipline as any other API contract — owned, versioned, tested, and evolved with the consumers downstream. A change to the output shape is a change to the system, not a tweak to a string.

**Sustainability / ESG.** Every unrequested token in an output — a preamble, a restated question, a reasoning chain nobody reads — consumes compute and the energy, water, and embodied emissions that compute carries. Output discipline is one of the most overlooked carbon levers in a working AI system, because it acts on the most expensive tokens at the highest billing rate.

**Business / Product.** Reliability of an AI feature is downstream of the shape of its outputs. A feature that returns the right answer in the wrong shape fails in production exactly as a feature that returns the wrong answer does — the user sees a broken experience either way. Output shape is a product surface.

Five seats, one ladder, one return trip that earns its tokens.

Pick one feature this week whose output is being parsed, transformed, or cleaned up by code downstream of the model call. The one with the largest post-processing block, or the one that occasionally makes a second model call to repair the first.

Pull twenty real production traces of that feature. For each one, capture two numbers: how many tokens the model generated, and how many of those tokens the downstream system actually used.

The ratio of used tokens to generated tokens is your output signal-to-noise. In most untuned systems, it sits well under half. Every output token that did not feed a downstream system was billed at three-to-five times the input rate, generated under full compute, and carried the full carbon cost of inference.

Walk the ladder. *Shape* the output in the prompt or in a schema, so the downstream parser has nothing to wrestle. *Cap* the length to the longest valid answer the system can actually consume. *Validate* the response against the contract before any code acts on it. *Repair* locally — never run a second model call to fix the shape of the first.

One feature, one ladder, two ratios, twenty traces. That is the work.

> *The cheapest output token is the one you told the model not to write. The second cheapest is the one it wrote because the answer actually needed it.*

Shape, cap, validate, repair — that is the discipline on the return trip of a single call. And with that, the call is complete. Over five days we chose the right model, made fewer calls, trimmed what goes in, fed the retrieval layer only what the question needed, and disciplined what comes back.

But most AI systems are no longer single calls. They are loops. An agent receives a query, plans, calls a tool, reads the result, decides again, calls another tool — sometimes ten or twenty turns deep, each turn a fresh call carrying everything we have just learned to optimise. The next principle is what efficient AI looks like when the unit of work is no longer a call, but a loop. See you in the next issue.

---