A workflow can be sequenced by code or by a model. Write it both ways and the two versions match step for step — yet they are not the same kind of machine.
An LLM can show up in a workflow in two ways. It can perform a task inside it — summarize this, judge whether that result is relevant, draft the reply. Or it can orchestrate it — decide which step runs next. The two are independent — you can use a model for either, both, or neither. We have written about the distinction already, in The Two Roles of LLMs: Workers and Planners.
Each role carries risk, and the two risks are nothing alike. Put a model to work on a step and you inherit the familiar set — it invents a citation, it satisfices, it produces something plausible and wrong. That is the subject of Principled Orchestration and we will not re-run it here. Put a model in charge of sequencing and you inherit something else entirely, which is less discussed and easier to miss.
This article is about the second one: the LLM as orchestrator. What handing over the sequencing costs, why the cost is invisible until it is expensive, and the technique that answers it — encoding workflows in tools.
In the example below I hold the first role constant: an LLM performs the steps in both versions. That keeps the comparison clean — the only thing varying is who sequences the steps.
Take deep research. Here is one reasonable way to carry it out:
The example is a linear sequence with one loop, because that is the easiest shape to follow. Nothing in this piece depends on that. The same argument applies to any control-flow structure a workflow can have — branches, parallel steps, and most commonly task graphs, where steps run when the steps they depend on have finished.
You can encode it: write a program that holds the sequence, runs the loop, and calls the model at each step. Or you can prompt it: write the five steps into a file, hand it to an agent, and let the agent decide when it is on which one.
Here is each one written out. The prompted approach is a skill the agent reads,
research.md. The encoded approach is a program the agent runs,
research.py. The same five steps both times — the only difference is
which kind of file they live in.
# Deep research
Start by clarifying the question. Ask
what the user actually wants compared,
and on which dimensions, before doing
anything else.
Then write down what a complete answer
requires — an explicit checklist.
Do not skip this.
Then loop. Find the biggest gap
between what you have and what the
checklist asks for. Write a query for
it. Run the search. Judge each result
against the gap that motivated it,
and keep only what holds up.
Check the checklist. If anything is
still missing, loop again. Stop after
six rounds.
Finally, synthesize the answer from
what you gathered — cite sources,
and say plainly where gaps remain.
def research(question): # clarify the question q = ask(CLARIFY, question) # write down what a complete # answer requires checklist = ask(REQUIREMENTS, q) kb = KnowledgeBase() for i in range(6): # find the biggest gap gap = biggest_gap(checklist, kb) if gap is None: break # write a query for it query = ask(WRITE_QUERY, gap) hits = search(query) # judge each result against # the gap that motivated it kept = ask(JUDGE, gap, hits) kb.add(kept) # synthesize from what # was gathered return ask(SYNTHESIZE, q, checklist, kb)
ask() is a
model call. · It also sets the key for every diagram that
follows: stochastic on the left, teal, the
model decides the next step;
deterministic on the right, blue, the code
decides.
The two files match step for step. The one thing that differs is who decides which step comes next — and that single question is the whole distinction.
Stochastic — the prompted version. The sequence lives in the context. The model reads the five steps along with everything else in its window and decides, each turn, which step it is on and whether the loop continues.
Deterministic — the encoded version. The sequence lives in the code.
The loop exits when biggest_gap returns nothing or the counter hits six.
The model is called into each step and has no vote on what comes next.
That one difference runs deeper than it looks. Same decomposition, same sequence, same six-round cap, same insistence that the checklist precede the searching — whatever you know about how research works is fully present in both.
But they are not the same machine.
In research.py, the steps are the mechanism. The loop is a loop.
The exit test is a test. Step 3 follows step 2 because of how the program is
built — it is a fact about the machine, true before the machine ever runs.
In research.md, the steps are text in a context window. They are not
inert: the agent reads them and behaves differently for having read them. But the
way they take effect is statistical. The words shift the distribution over what the
model generates next. They make the intended path more likely. Nothing in the system
makes it necessary.
Statistical rather than structural, because a language model has no grip on state or truth — only on plausibility. It emits the continuation that fits. A sentence reporting that it has finished step 2 is a continuation that fits, and there is nothing underneath it keeping count. We have written this up at length as the absence of ontological grounding, in Principled Orchestration; the consequence to carry into the rest of this piece is narrow. Instructions land as pressure on a distribution, never as structure in a machine.
That likelihood can be pushed high. A better model, a clearer file, a worked example, a firmer tone — all of it helps. None of it changes what kind of thing the sequence is.
Stochastic orchestration can be made more or less deterministic, in a relative sense. You can hand the model a bare goal — research this question and report back — and let it invent the procedure. You can hand it the five steps in order. You can hand it the five steps plus worked examples, the edge cases, and an instruction in bold not to skip step 2. Each addition makes the intended path more likely and the behavior more predictable.
But none of these differ in kind. Each is text in a context window. Each raises the probability of the path you wanted; none of them makes it necessary.
The mistake this invites is to picture encoding as the endpoint of that progression — as though a sufficiently detailed prompt eventually hardens into a program. It does not. Deterministic orchestration is not more instruction. It is a different mechanism, and no amount of added instruction turns one into the other.
When a prompted agent writes "I've covered pricing and availability — moving on to synthesis," that sentence was generated the same way every other sentence was. It is a claim about control flow, not control flow. Part I says why: there is no mechanism with steps in it for the claim to be true of.
Principled Orchestration named three structural shortcomings of language models and described all three as threats to the content of a step. They threaten the sequence just as readily:
biggest_gap returns nothing.Delete the checklist line from the encoded program and biggest_gap raises
on the next call. You find out on the first test run, and the stack trace points at
the line.
Skip the same step in the prompted version and you get a fluent, cited, confident report. Nothing throws. The step it skipped might have been the one that clarifies the question — but the run continues without it, and the answer reads exactly like one that did everything right.
The failure is silent because nothing steps back to catch it. Anything that could — a review pass, a verification step — would be another judgment by the model, as stochastic as the run it checks. There is no deterministic check to fall back on; that is precisely what a prompted workflow does not have.
This is why prompted workflows demo well and fail silently in production. The failure produces no signal. You do not get worse-looking answers; you get answers that look identical and are quietly less complete, and you hear about it from a customer.
Which is also why you cannot test your way to confidence here. Run a prompted workflow twenty times, watch it do the right thing every time, and you have learned that the likely path is likely. You have not learned that the other paths are closed, because they are not. Repetition measures the weighting; it cannot tell you about the tail, and the tail is where the run that skipped step 2 lives.
A language model is a probabilistic system. Not as a defect, and not as a phase it is working through — that is what the thing is. So a prompted workflow is probable rather than certain. A well-written one is very probable. Nothing converts a probability into a certainty.
Which makes the design question a short one. Does this step have to happen? If it merely ought to, and usually happening is good enough, a prompt is a perfectly good way to get it — and most steps are like this. But if it must happen, because something downstream assumes it did or because you have told a customer it does, then a probabilistic system cannot supply what you need, and no improvement in the models will change that.
That is the entire argument. The rest of this piece is what follows from it.
Part II explained what prompting cannot give you. This part compares the two approaches directly, factor by factor.
Deterministic orchestration. The sequence is a program. Code holds the loop, calls the model into each step, and decides what runs next by evaluating a condition.
Stochastic orchestration. The sequence is a prompt. The agent reads the steps and decides, turn by turn, where it is and what comes next.
They differ on a short list of factors: the sequence and its boundaries, what state is made of, the context each step sees, what a repeat run does, what failure looks like, the branch you did not foresee, and what it takes to change the workflow. The table compares them on each.
| Factor | Stochastic — prompted | Deterministic — encoded |
|---|---|---|
| The sequence | Something the prompt makes likely | A fact about the program, true before it runs |
| Boundaries | “Enough,” felt | gap is None or i == 6, evaluated |
| State | Text in the window, competing with everything else in it | A data structure the model cannot narrate past |
| Context per step | One window, growing all run, carrying every dead end forward | A purpose-built call per step, same size every run |
| The same input, twice | A different path each run | The same path every run |
| A skipped step | A fluent answer that looks like every other answer | A crash, and the stack trace points at the line |
| The branch you did not foresee | The agent can notice, stop, and say so | No line of code for it |
| Changing the workflow | A conversation, in the session where you noticed | A programmer, a test, and a release |
The deterministic column wins the first six rows, and they are all the same kind of advantage: guarantees. The stochastic column wins the last two, and they are also the same kind of advantage: flexibility. The run can leave the path when leaving is right, and the workflow can be changed without a release.
The stochastic advantages are worth spelling out. Suppose the third search returns
something that shows the question itself was wrong. An encoded loop just runs its
next iteration; there is no line of code for notice the question was wrong.
A prompted agent can stop and say so. And when the workflow itself needs to change,
research.md can be changed by talking to the agent, in the session
where you noticed the problem. research.py needs a programmer, a test,
and a release. The person who knows how the research ought to go is often not the
person who can edit Python.
But you cannot keep this flexibility and add the guarantees, because they are the same property. The capacity to leave the path is the absence of enforcement. An agent that can depart from the workflow when departing is right can depart when departing is wrong, and nothing distinguishes the two at the moment of the draw. You do not keep the upside and engineer away the downside.
There is one more difference, and it is organizational rather than technical. As described in The Agent in the Middle, encoded workflows are more durable than prompted ones, and governance follows durability. Hardening a workflow and tightening control over who can change it are the same motion.
Agents are nondeterministic by nature. That is the central problem for anyone trying to get reliable work out of them, and Part II showed that better prompting does not solve it. Encoding workflows in tools is one of your strongest weapons against it: every step you move into code is a step that is guaranteed to happen — every run, in the same order, against real state.
That does not mean encode everything. It means examining every workflow with the limitation of its stochastic elements in mind. Where you need 100% reliability, go deterministic. Where you are prototyping — when the question is still whether the workflow is right at all — prompting is usually the practical choice: faster to write, and changeable by talking.
The two answers connect, because a workflow typically starts stochastic and hardens as it proves itself: said in conversation, then written down as a skill, then encoded as a tool.
The instinct is to wait for evidence — harden the step once you have seen it misbehave. That instinct is wrong here, because the evidence does not arrive.
An LLM can perform the steps of a workflow, and it can decide which step comes next. This piece was about the second. The first is unchanged by any of it.
The workflow was never the hard part. Writing down the five steps of good research takes an afternoon, and they come out the same whichever file you put them in. The decision that matters is narrower, and it is usually made by accident: which steps are guarantees and which are suggestions. Encode a step and it will happen, in a clean context, against real state, and it will need a release to change. Prompt it and it will probably happen, in a window full of everything else, and you can fix it by talking. Probably is a fine answer for some steps and a disqualifying one for others. It is never a smaller version of will.
What you cannot do is leave it unmade. Control flow that ended up in the model because nobody chose where to put it is not a flexible system. It is one that returns plausible answers while skipping the step that would have told you.