aronchick: The right question for LLM data pipelines is where to draw the determinism boundary

On September 26, aronchick posted a thread systematically laying out where LLMs fit in data pipelines. His core argument: everyone keeps asking "should we use an LLM?"—but that's the wrong question. What every pipeline team actually has to answer is "where do we draw the deterministic boundary?"

Confirmed

Classic pipelines have a contract of "one code path per edge case," trading flexibility for reliability. Adding an LLM stage changes that contract—the model classifies unknown records by reading content and may return different results tomorrow, so retries are no longer safe.
aronchick flags a common mistake: treating "determinism vs. judgment" as a single either/or choice. The right approach is to assign per stage—joins, aggregations, dedup, and validation go down deterministic paths; messy-input work like extraction, classification, and narration is where models deliver value that deterministic rules can't.
His rule of thumb: if an edge case would need a human adjudicator, that's a good spot for an LLM as judge—but let it only opine, never make the final call.
With Expanso, he published ten real pipeline examples. Take log triage: all deterministic parts (ingestion, fingerprinting, counting, routing) run inside Expanso nodes with fixed rules, never touching the model; only judgments that can't be written as rules—"is this actionable, how severe, who owns it"—get handed to an LLM via a single HTTP call.

Why it matters

As LLMs move into production data pipelines, retry safety and output reproducibility become new engineering risk points. aronchick's framework—split deterministic boundaries by stage, let the LLM judge but not decide—gives teams actionable partitioning principles and real-world case references for designing hybrid pipelines.

2026-09-26 ~ 2026-09-26 · 7 related posts

Primary sources

The wrong question about LLMs in pipelines: which stages get determinism guarantees — aronchick · 2026-09-26 03:15 (UTC+08:00)
The right question for LLM data pipelines: which stages keep determinism guarantees — aronchick · 2026-09-26 03:15 (UTC+08:00)
Log triage pipeline: deterministic steps and LLM judgment seam via one HTTP call — aronchick · 2026-09-26 03:15 (UTC+08:00)

[source] The wrong question about LLMs in pipelines: which stages get determinism guarantees — aronchick · 2026-09-26
Assign determinism and judgment per pipeline stage, not as an either-or — aronchick · 2026-09-26
The only pipeline shape that survives production: LLM proposes, rules dispose — aronchick · 2026-09-26
The judgment layer always floats on a deterministic substrate — aronchick · 2026-09-26
Where to draw the determinism boundary in LLM pipelines — aronchick · 2026-09-26
[source] Log triage pipeline: deterministic steps and LLM judgment seam via one HTTP call — aronchick · 2026-09-26

1 near-duplicate retellings: aronchick