There are two guiding principles I try and mentally check in with when I am working with coding agents.
The first is that determinism beats inference. Determinism is free, predictable and fast. Inference is expensive, unpredictable and slow. If you can do something deterministically then do it that way. But beware — staying deterministic can create a situation of over-fitting, which coding agents frequently fall into. To avoid overfitting you need some inference. Use as little as you need, and only pull out the frontier models for the most complex tasks. I widely deployed the ladder pattern: from determinism, to some inference, to more inference.
The second principle is the feedback loop. Every failure is the opportunity to prevent that failure. Log failures, log inference calls, and look for opportunities to improve the flow through the system, always pushing back down the ladder towards determinism. Feedback creates flywheels, improving your system iteration after iteration.
Example 1: The scraping ladder
Scraping is a perfect ladder problem, because the web is unevenly hostile. Most pages will give themselves up to a plain HTTP request costing nothing and taking under a second. A minority need a JavaScript render. A small, stubborn few sit behind residential proxies, CAPTCHAs and other challenges, and getting those costs real money and real minutes.
So the fetcher climbs. Tier one is a plain request with a browser user agent — the cheap, fast path. Tier two hands the URL to a scraping API with JavaScript hydration for the React and Vue microsites that return an empty shell to tier one. Tier three is a local headless browser: slower, but free, and it waits out the single-page apps that the API’s renderer gives up on. Tier four is the unblocker — residential proxy, stealth, CAPTCHA solving — expensive and reserved for sites that actively fight back. Tier five is a full visible browser with the automation flags scrubbed, the only thing that reliably beats complex challenges.
Two details matter more than the tiers themselves. First, the gates. A tier doesn’t just have to return something — the result is checked for status, length, anti-bot keywords, cookie-banner debris and empty JS shells before it is accepted. A ladder without gates is just a sequence of ways to accept garbage. Second, the shortcuts. If tier one comes back with a anti-bot challenge, the fetcher doesn’t politely climb through tiers two to four and burn five minutes of timeouts — it jumps straight to the top rung, because it already knows how this story ends.
And this is the flywheel: the system remembers. Every domain gets a fetch strategy recorded against it — this one is static, this one needs a JS render, this one needs the unblocker, this one is dead and not worth a single request. Next time a URL from that domain arrives, the router goes straight to the right rung. No cascade, no wasted calls. Failures are logged too, and the failure log is where new strategies come from. The ladder handles the unknown; the flywheel makes sure nothing stays unknown for long.
Example 2: Matching messy names, and the loop that tightens
Consider a case where every entity extracted from a document must be resolved to a canonical entity. Company names, project names, people etc. Consider the project “Atlantic Fibre Link 2 (AFL2)”. We need to know if we already know about this project, if it is the same project as a project already in the database? This problem is harder to solve than it first appears. Fuzzy string similarity says “Project Big 1” and “Project Big 2” are nearly identical, yet a human knows they are different phases of different assets, because a human recognises that the numbers carry more identity than the words.
The resolver is a ladder. Rung one: exact match on the canonical name — free, instant, most of the traffic. Rung two: exact match against an alias index built from every name the entity has ever been known by, quality-filtered so that ambiguous, code-like fragments never get indexed. Rung three: clean the LLM’s extraction — strip the noise, pull aliases out of parentheses — and try again. Rung four: fuzzy matching, but fuzzy matching that thinks like a human — phase identifiers and numeric tokens are weighted so that “Phase 1” versus “Phase 2” is a hard separation, not a 95% match. Only when all of that fails to produce a confident answer does the top rung fire: an LLM is handed the candidates along with the disambiguating context the caller already knows — country, asset class, expected capacity, owners — and asked to reason its way to a selection. The same context a human analyst would use, encoded.
The feedback loop is what makes this more than a matcher. Every non-exact match is written to an audit collection for review. When the LLM resolves a messy name, that name becomes an alias on the entity — so the next time it appears, it resolves deterministically, for free. When two records turn out to be genuinely different, that fact is recorded so they are never confused again. And when there isn’t enough disambiguating context to decide, the resolver refuses to guess — a logged failure is a signal that a problem needs attention.
Run that loop for two hundred days and the expensive rung goes quiet. The LLM solves each hard case once; the ladder remembers the answer forever. That is the ratchet: every failure makes the system slightly harder to fail.