You’re Giving Your Agent Too Much To Do

Most agentic systems I’ve come across ask the model to take on a remarkable amount of responsibility. The agent is often expected to interpret the goal, plan the work, choose the tools, determine what context it needs, do the work, evaluate the result, and decide what to do next.

That’s a lot to ask of a single probabilistic component — and “probabilistic” matters, here.

We need to start thinking of models as smaller parts of larger, more deterministic systems.

It seems like forever ago now that ChatGPT was released. I fell into the hype — much of it warranted, mind you, but still hype. I dreamed about the possibilities. I dug in. When the API arrived, I built product after product around the models.

They were all garbage.

They were garbage because I made the model the centerpiece. I gave it too much to do.

There’s a principle in software engineering that is near and dear to every experienced programmer’s heart: the Single Responsibility Principle.

The idea is simple. Give each part of a system one clearly defined job.

An LLM should be treated the same way.

For architectural purposes, an LLM is a function. Give it one thing to do.

Start breaking the problems that feel nondeterministic into smaller and smaller pieces. You’ll begin to find parts that can be handled deterministically.

Keep going, and I think you’ll find that most of the system can be.

The hype has trained us to begin with the model. We should begin with the system.

Consider a system built for a portfolio advisor that produces a spreadsheet every morning.

The system gathers each client’s positions along with current stock and market data. It compares those positions against the firm’s market outlook and derives the relevant values. It does all of this deterministically.

The completed analysis is then given to the LLM. The LLM reviews the results, judges how the advisor’s clients are doing, and composes a summary — the story of the advisor’s clients that morning.

That’s all it does.

Determinism takes back over. The system takes that summary, produces an email or text message, attaches the spreadsheet, and sends it.

The LLM is doing exactly what it was designed to do: reason over the context it is given and provide its judgment.

I know what you’re thinking — especially if you’re a developer. But that doesn’t mean building agentic systems has to be boring. The excitement just shifts upstream.

Instead of asking an agent to run the system, let the agent build the system. Let it build the workflows, rules, calculations, and handoffs that keep its own role small once the system is running.

And let it do that inside a deterministic framework.

That’s a different kind of agentic architecture, and I’ll show you how to build it in the next article.