How would you reduce the cost of running an agent?

Answer

Cost is tokens times price, summed over every iteration. Attack each factor.

Reduce context size. The biggest lever, because you pay for the whole context on every call in the loop. Remove unused tools, avoid dumping large files or logs into results, and start fresh sessions rather than accumulating history.

Use prompt caching. Keep the stable prefix, system prompt and tool definitions, at the front and unchanged. In an agent loop the same prefix is resent every iteration, so caching typically produces the largest single saving available.

Route by difficulty. Not every step needs the strongest model. Classification, extraction, and routine edits often run acceptably on a cheaper model. Reserve the expensive one for hard reasoning.

Cut iterations. Each loop iteration is a full priced request. Better tools reduce iterations: one search_codebase call beats six exploratory greps. Clearer instructions reduce wrong turns.

Cap output tokens where the response should be short. Output tokens usually cost several times more than input tokens.

Batch where latency permits. Some providers discount asynchronous batch processing substantially for work that is not interactive.

Measure before optimising. Log tokens per task, split by input, output, and cached, and find where the money goes. It is frequently one verbose tool dumping large results into every iteration.

Common wrong answer

Jumping straight to the cheapest model. If it fails more often, you pay for retries and for the human time to fix wrong output, which usually costs more than the model difference. Optimise context and iterations first, then consider routing.