New
articleMigrating a 20-Year WordPress Blog to Be Agent-Readytechnology bytesAGI Has Arrived. Now Give It a Simpler Problem.podcastEpisode 100: The Next Era of AI — From Agents to Enterprise IntelligencearticleMigrating a 20-Year WordPress Blog to Be Agent-Readytechnology bytesAGI Has Arrived. Now Give It a Simpler Problem.podcastEpisode 100: The Next Era of AI — From Agents to Enterprise Intelligence
Navveen Balani Making systems intelligent, efficient & responsible

Day 6 of Green, Efficient AI is live — The Cheapest Tokens Are the…

Read on LinkedIn View as markdown

Day 6 of Green, Efficient AI is live — The Cheapest Tokens Are the Ones You Never Retrieve.

A team was sending their model an 8,600-token prompt to answer a question
whose actual answer was one paragraph.

Twenty passages retrieved. One useful. Nineteen paid for in full.

The model was never the problem. The retrieval layer was — and most teams have never looked at it.

Most prompts in production are not written. They are assembled. And the cheapest tokens in any prompt are the ones you never retrieve.

How to find and fix the waste, in today’s issue.

Green Software Foundation #greenai #efficentai

green software