Putting the Horizon in the Harness

Nancy Sadkov¹ · Claude²

¹ Independent researcher (NovaClaw) · ² Anthropic

Draft — 2026-07-26

Terms

Abstract

Small local language models fail as agents in one usual way. They name the correct algorithm, and they write correct small parts of a program. Then, at the integration step, they stop. They write an apology. They give a very wrong estimate of the size of the work that remains. They end the program before it is complete.

We say that this is a horizon failure, and not a capability failure. Horizon can move. It can stay in the model, or it can move into the harness. Usual agent harnesses keep the horizon in the model. They give only two thin services: a tool loop, and a function that makes the context smaller. A weak model in such a harness does many useless actions.

This paper presents jh. jh is a deterministic controller around a model that proposes. The model gives a different answer each time. The controller always does the same thing with the same input. The work of the model becomes one small task that repeats.

For the current node of a plan tree, the model emits one tool call, or it emits a decomposition. The harness keeps the loop, the context, the check, and the recovery. The loop is: read the state, select one action, do the action, measure the result, then write the new state. The harness enforces two properties. It does not ask for them.

Every leaf does exactly one tool call. A mechanical check controls every leaf. The model never checks itself.

We report an engineering study of 22 waves, from 2026-06-28 to 2026-07-19. The study drove one small model against three levels of check. The model was qwen3.6-35b-a3b at 4.75 bits, with vLLM on a 128 GB DGX Spark. These are the results:

• Tier 1, code and build: 10 successes out of 10 runs on a test group for build environments. The harness also made repeated end-to-end successes on a hard task: 100 digits of pi from the original prompt of the user.
• Tier 2, mechanical media pipelines: 9 successes out of 9 runs, with no change to the engine. This tests the claim about other domains directly.
• Tier 3, prose with a judge: 0 successes out of 6, then 5 out of 6. The change was a new size for the atomic Step. The new size agrees with the retry granularity of the domain.

The negative results tell as much as the positive results. Ten measured waves of correct and validated changes never moved the rate of the pi task above about 1 success in 6 runs. An increase of the time limit by 37% moved this rate not at all.

The output of the study is nine empirical laws. Three of them carry the most weight:

• L1: On small models, the harness is the dominant experimental variable. This law also acts against the claims of the harness author. Two times, we read a defect of our own harness as a limit of the model.
• L2: Instruction changes engage, but only mechanism changes convert. Advice in a prompt reliably changes the behavior of the model, but it does not change the rate of success. A change that makes a bad action impossible does change the rate.
• L3: The capability floor is a property of the triple (model × domain × harness). It is not a property of the model. Thus, no reader can interpret a published capability number for a small model if the report does not give the harness.

We also report a methodological result that has independent value. An earlier check of the citations of this work made a cascade of false retractions. It marked four real systems as invented (§9.4).

We do not claim that jh is better than other harnesses for small models. We ran no direct comparison. This is the principal limitation of the study.

1. Introduction

1.1 The observation

We saw the same behavior many times, on local hardware, with models from 8B to 35B. The task was this: write 100 digits of pi in C99, compile the program, and verify the output. The model then did three things:

1. It named a correct algorithm. The algorithm was the formula of Machin with fixed-point arithmetic.
2. It wrote correct primitives, when we asked for each primitive alone.
3. At the integration step, it stopped. It wrote an apology: "I can't write 300 lines...". It gave a very wrong estimate of the size of the work. The loop that was absent had approximately 15 lines. Then it wrote return 0, ended the program.

Each sub-task was inside the capability of the model. The control of the long sequence was not. We call this a horizon failure. The model is competent at each step, but it cannot put the steps in sequence with no help.

1.2 Why the model stops

Recent work on architecture gives a mechanism. Transformers usually solve compositional tasks in an incorrect way. They match linearized subgraphs. They do not execute the steps. Thus, accuracy falls when the computation graph of a task becomes deeper. The authors show this for multiplication of many digits, for logic puzzles, and for dynamic programming [1].

The fall is abrupt, and not gradual. Reasoning is stable inside one regime. Past a critical logical depth, it collapses.

The authors compare this behavior to a phase transition in physics [2]. The faculty that fails is the dynamic track of the state. Attention is a mechanism for retrieval. It is not a mechanism for the iterative update of a latent variable, and a task that changes needs such an update.

Thus, weak models fail after a number of steps, even when they understand the task [3]. The limit comes from the architecture. Feedforward layers push the state deeper into the stack, until no more depth is available [4].

Other authors measure the same failure on agents directly. There, it appears as a phase transition into a corrupted world model. The authors do a grid search over four axes: state cardinality, dependency density, horizon, and branching.

The result has three parts: a plateau where the agent solves the task, then a transition band, then a floor where the agent collapses. In the transition band, the fidelity of the world state fails before the validity of the action. Thus, the agent does not only select a bad action. It acts from a corrupted world. Stronger models move the critical boundary, but they do not remove the transition [5].

That last result gives the design brief of this paper. We read it in three ways:

• The failure is a corrupted world model. Thus, the harness must own the world (§3.4, §3.5).
• The boundary only moves with scale. Thus, the scaffold stays useful (§8.3).
• The control axes of the transition are computable from the dataflow of a plan. Thus, the trigger for decomposition can measure them (§3.3, §4).

These works give the diagnosis. None of them tests the cure that we propose. Note also the remedy of [4]. Those authors change the architecture to recurrent models or state-space models. jh keeps the transformer and moves the state out of it.

1.3 The thesis

Horizon can move. Take a task with a horizon of 20. Decompose it into twenty tasks with a horizon of 1. Then let the harness hold the position, the dependencies, and the check.

The model does not hold them. Adaptation, error recovery, and long-horizon coherence then become properties of the controller. They are no longer properties of the model.

The first idea came from pedagogy, and it is important, because it made the design. A teacher does six things:

1. Measure the difficulty.
2. Decompose the work to the level of the learner.
3. Give one tractable sub-task at a time.
4. Test the understanding of the learner before the next sub-task.
5. Do not give too much at one time.
6. Remove the help as the learner succeeds.

In the vocabulary of architecture, this pedagogy is a control loop. The teacher is the idea. The mechanism is a deterministic controller around a model that proposes.

1.4 What is new and what is not new

We are precise here. An earlier draft of this work was not precise. Also, most of the parts are known:

• Recursive decomposition, done only when it is necessary, is not new. ADaPT [11] established it.
• One tool call for each step is not new. It is the usual behavior of the ReAct family.
• The output of a model as a program, with the harness as compiler and runtime, is not new. See PAL [12], Program-of-Thoughts [13], CodeAct [14], LLMCompiler [15], and DSPy [16].
• Typed dataflow between steps is not new. See the Signatures of DSPy [16] and the typed State channels of LangGraph [49].
• A scaffold that lifts a frozen small model above its usual level is not new. Other authors show this at the same time as our work. "Three Roles, One Model" [38] gives a frozen Qwen3-8B three roles: summarizer, agent, and an isolated corrector with no conversation history. That system approximately doubles the goal completion rate on AppWorld. It also passes a model that is 4 times larger, on one 24 GB GPU. That isolated corrector with no history is the same mechanism as law 6 in our §3.3.

These are our contributions:

1. An architecture (§3). It combines four elements. The first is a fixed schema for introspection with many axes. The second is a hard rule: one tool call for each leaf. The third is typed dataflow that the harness owns. The fourth is a mechanical check on every leaf, which the harness always applies. The check is the element that carries the most weight. The decomposition is not. We know of no earlier harness for small models that makes the check of each step obligatory. EffGen [37] is the closest framework for small models. It does decomposition, prompt compression, complexity routing, and memory. It does not check each step.
2. A long measurement program (§6). It has 22 waves and approximately 150 runs with fixed seeds. Each wave has a written prediction before its runs. We analyze each failure at transcript level. We publish the counts of every wave, and this includes the many waves that changed nothing. We know of no comparable published record of the development of a harness under continuous measurement. We think the trajectory tells more than any single number inside it.
3. A framework of check tiers, with measurements at all three tiers (§6.1–§6.3). It includes a controlled test of generality. In that test we changed only the task prompts and the check. The engine stayed identical, byte for byte.
4. Nine empirical laws (§7). Laws L1 to L3 constrain how a report must give results for agents that use small models.
5. A negative methodological result (§9.4). Our own adversarial check of the citations made a cascade of false retractions. We document it, because the failure mode is a hazard for all persons who use a language model to review literature. That failure mode is a check that says, with confidence, that real work is invented.

1.5 Non-goals

jh is not a harness for frontier models. Frontier models want less scaffold, not more. jh is not a better model. It gives orchestration, not knowledge.

A capability floor exists, and below it a scaffold cannot help. jh is also not minimal, and this is deliberate. The opposite philosophy for small models is "do not obstruct the model". That philosophy is coherent. We argue against it. We do not ignore it.

2. Related work

2.1 Decomposition, program as action, and typed pipelines

ADaPT [11] introduces recursive decomposition that happens only when it is necessary. The authors show that the pattern moves without change across ALFWorld, WebShop, and TextCraft. jh uses this discipline. jh also extends it to the plan itself (§3.3, the amendment to law 3).

Two lines of earlier work give the frame of the harness as a runtime. The first line makes the program the reasoning: PAL [12] and Program-of-Thoughts [13]. The second line makes code the action: CodeAct [14], which reports up to approximately 20% more success across 17 models. LLMCompiler [15] lets a model emit a task graph. DSPy [16] compiles declarative pipelines with typed Signatures.

jh differs from all of these systems at one point: the time when the structure becomes fixed. They plan a full program or a full graph at the start. jh expands one level at a time. jh also reduces every leaf to exactly one tool call.

2.2 Self-correction does not work at this scale, but external checks do

For our target, this is the result in the literature that controls the most design decisions. Reflexion finds that self-correction is "an emergent quality of stronger, larger models". It reports StarChat-beta on HumanEval at 0.26 before and 0.26 after.

The gain is zero [7]. Self-Refine reports that Vicuna-13B cannot reliably make feedback in the necessary format. When the authors give it correct feedback from an oracle, it repeats or invents. It does not refine [8].

The self-knowledge of a model is also weak at a small scale. Calibration and self-evaluation improve with the size of the model [9]. All tested models are too confident, and their discriminative power does not reliably increase with capability [10].

Scaffolds that use external feedback from an executable check do lift weak models. Examples are LATS [43], AgentCoder [44], MapCoder [45], Agentless [46], and the --auto-test and --lint loop of Aider. That loop sends the errors of the compiler, the lint tool, and the tests back to the model, and it iterates [42]. Note one caution about the numbers. The large differences that persons quote from this literature usually compare against a weak baseline, which is zero-shot or chain-of-thought. The direction is the part that carries weight, and the direction is clear.

jh makes this direction a hard constraint. Introspection is structured. It has fixed axes and short fields. It is never free text with self-criticism.

The check is always mechanical. A Corrector with a fresh context does the correction. It sees only three items: the goal, the artifact that failed, and the exact error. We deliberately remove the transcript of the failure, because a small model cannot use it. [38] found the same design for an isolated corrector, independently.

2.3 Verifiability is the axis that selects the domain

The guarantees of jh need an objective check that costs little. This is also the thesis of Reinforcement Learning with Verifiable Rewards (RLVR). There, a deterministic function replaces a learned reward model [32]. The gains concentrate where "a compiler can be used to generate feedback based on predefined test cases" [33].

Outside code and mathematics, the check becomes subjective. It becomes a learned judge, and a model can hack its reward [34]. Authors who try to extend RLVR to general domains report that dependence on domain-specific checks "results in prohibitive complexity and limited scalability" [35]. Authors who move into medicine, chemistry, and economics had to leave rule-based checks. Only approximately 45% of the answers in their many domains were single clean values. In mathematics the number is approximately 60% [36].

Our tiers (§6) are the reading of this literature from the side of the harness. Enter a domain when that domain gives you an objective check that costs little.

2.4 Agent runtimes with event logs, and the harness as controller

OpenHands ships a pattern that makes each step hard: Intent, Action, Observation, Verification. The model generates only the Intent and the Action. The event log is immutable, and the system only appends to it. The Observation and the error events are explicitly not the output of the model. The overhead of the event log is very small [23].

ESAA agrees at the level of architecture. Its agents emit only structured intentions as validated JSON. A deterministic orchestrator validates the events and appends them to a log. This separates the cognitive intention from the mutation of the state [24]. Below both systems is ordinary Event Sourcing [25].

The control-theoretic reading is also established. The harness is the controller of a closed-loop dynamic system, and the model is the stochastic policy that it controls [20]. Other authors do model-predictive planning, which is also called receding-horizon planning [21]. Other authors separate the planner, the executor, and the feedback in a closed loop [22].

Adaptation at runtime is also concurrent work. CODA derives difficulty from an observed signal of group success, and it uses that signal to allocate compute [26]. "Adapting the Interface, Not the Model" monitors telemetry after execution to trigger recovery. It works with 18 different backbone models [27].

The contribution of jh inside this frame is narrow and specific. It adds a constrained action space, which is the fixed schema for introspection. It also adds an obligatory deterministic check after every action. The loop itself is old.

2.5 Harnesses for small models: the contested space

Three systems occupy this space, and all three are real. §9.4 explains why that sentence is necessary.

• EffGen [37] is the one peer-reviewed agent framework for small models that we found. It does decomposition, prompt compression, complexity routing, and memory. It does not check each step. It is the nearest competitor, and this gap is exactly the difference that jh claims.
• SmallCode [40] ships decomposition into atomic steps that a TODO file controls. It validates with a lint tool and a compiler before it advances. It also has a harness for test-driven development, and it detects loops and patch spirals. Sections §3 to §3.4 of this paper overlap it to a large degree. jh is not new ground.
• little-coder [41] has the best evidence. A 9.7B model moves from 19.11% to 45.56% on Aider Polyglot, which has 225 exercises. The baseline is the same model in a standard Aider harness. The project publishes the same central claim as this paper. The architecture of the scaffold is the primary control. The scale of the model is not. Both projects reached the claim independently.
• Confucius Code Agent [39] reaches 59% Resolve@1 on SWE-Bench-Pro. It uses a scaffold SDK with a meta-agent that builds, tests, and improves. It does not target small models. It is directly relevant to the idea "change the scaffold, not the model".

Thus, the honest position of jh has three parts. Other work validates the thesis independently: [38] and [41]. Other systems occupy the market: [37], [40], and [41]. Our defensible increments are three: the obligatory mechanical check on every leaf, the tier framework with measurements across tiers, and the laws in §7. We ran no direct comparison (§9.1).

2.6 The harness is an experimental variable that reports do not give

METR gives a formal measure of the solo time horizon of an agent [47]. Success decreases as a task becomes longer, and this is why decomposition helps. Separately, [20] argues that a reader cannot compare agents correctly if the report does not give the harness. §6.7 reports our own field observation of that hidden cause. We caught it live on small models, before we found the paper. §7, law L1, reports the same problem inside our own work.

3. Design

3.1 One recursive primitive

Everything is a Step. A task is a tree of Steps.

• An atomic Step is a leaf. It does exactly one tool call.
• A compound Step is a node. It is a list of child Steps.
• The agent loop is an evaluation of this tree, depth first.

Two comparisons describe the same structure. The comparison to Lisp shows why the plan is data that a person can edit and resume. The comparison to a compiler is clearer for readers who build systems.

One cell has no equivalent, and this is deliberate. jh checks a program when it runs the program. The name "type check" claims static guarantees that jh does not have.

The wire format. The tree is an AST. The format that the model emits is a separate empirical question. Our default is JSON, because persons train small instruction-tuned models heavily on JSON formats for tool calls. Two cautions from the literature kept this decision honest:

• A restriction of the format can damage reasoning, if the format controls the reasoning itself. GSM8K fell by approximately 63 points for one model in enforced JSON-schema mode. Most of the loss came back when the authors removed the schema [17]. But light constraints, or constraints that start late, can help [18].
• For strong models, the effect of the format is contested. One study finds no causal effect in 43 of 48 scenarios [19].

Thus, we did an A/B test on our actual target model. We did not assume the answer (§6.6).

3.2 The Step schema: introspection that the harness forces

Before a Step acts, the model fills a fixed schema. The schema is the scaffold. It prevents free text, because the model fills fields and does not philosophize. Its size field is the trigger for decomposition.

Step {
  goal:             string        // one sentence: what this step achieves
  research_needed:  bool          // if true, a research child Step runs first
  consumes:         ArtifactRef[] // typed handles that this leaf READS  {id, type}
  produces:         ArtifactRef[] // typed handles that this leaf WRITES {id, type}
  tool:             ToolName|null // the ONE tool that this leaf calls (null if compound)
  size:             "atomic" | "needs_decomposition"        // <- the rule of the loop
  difficulty_prior: "trivial" | "moderate" | "hard"         // a PRIOR. The harness can replace it (§3.4)
  success:          string        // the check (does it compile? does the test pass? is the output X?)
  assumptions:      string[]      // what the step takes as true, and what can go wrong
  substeps:         Step[]        // present only if size == needs_decomposition
}

Two commitments are visible here.

(a) Typed dataflow between steps. The model declares consumes and produces. The harness resolves them and builds the context. This discipline is usual. See the typed State channels of LangGraph [49] and the Signatures of DSPy [16].

jh combines this discipline with a check on each step, which neither of those systems does. Note the scope. jh types the dataflow between steps. jh does not constrain the free text that the model generates (§3.1).

(b) The field is difficulty_prior, not difficulty. A weak model is a bad judge of its own difficulty and of its own success [9][10]. We keep the field for one reason. One token of an enum gives a retry budget before any telemetry exists. Also, a bad estimate is still better than a random one.

The field is useful as a prior. It is unsafe as an authority. The harness computes an observed difficulty from signals at runtime, and it replaces the prior (§3.4).

The same inversion applies to size. [5] shows two dimensions that control where reasoning collapses. The first is the number of objects that the agent must track at one time. The second is the number of preconditions that control an action. The harness can compute both from the dataflow that it already owns:

• State cardinality is the size of the transitive closure of consumes.
• Dependency density is the number of edges in the dataflow graph of the decomposition.

The collapse is an abrupt transition in exactly these variables [2][5]. Thus, a closure or an edge count above a calibrated threshold is a sufficient reason to force a split. The harness forces the split even when the model says "atomic".

This is a structural control before execution. It works with the observed difficulty, which is a control after execution. [5] sweeps four more axes: horizon, branching, observation mode, and mutation rate. The harness can compute several of these from a plan, so the trigger can grow.

3.3 The laws

The harness enforces seven invariants.

1. One tool call. Every leaf Step makes exactly one tool call. A leaf never chains actions.
2. Introspection before action. No Step acts before the model fills its schema.
3. Recursion, the rule of the loop. A Step that one tool call cannot do must decompose. The harness recurses until every leaf is atomic.
4. Checkpoint. Every Step has a success check. No Step advances before that check passes.
5. Minimal context. The Context Manager builds the prompt from the transitive closure of consumes for the current Step, over the artifact store. It adds the goal and the schema. It does not use a heuristic such as "the siblings that are relevant". This defeats two failure modes at one time: the loss of a necessary dependency, and the loss of that dependency among many irrelevant siblings. Models give too little attention to information in the middle of a long context. Accuracy falls by approximately 20 points when the necessary item moves to the middle of a long window [6]. Thus, the selection of the items must follow a principle. It must not be maximal.
6. Recovery. A failed success check starts an external correction. It does not start self-reflection in the model. A Corrector with a fresh context sees the goal, the artifact that failed, and the exact error. It never sees the history of the failure. It proposes the smallest fix. After that come bounded retries, then more decomposition, then a move backward. The justification is [7][8]. Useless repeated action is structurally impossible. The worst case is a clean report that says "blocked".
7. Valid dataflow, before execution. A decomposition is well-formed only when a prior sibling produces every consumes of every child, or when an ancestor gives it. The harness rejects or repairs a consumes with no source before it runs the model. This changes a lost implicit dependency from a silent stop into a caught planning error that the harness can repair.

3.3.1 Amendments from the field, which we measured

Live measurement forced the corrections below (§6). We report them as amendments. We do not fold them silently into the laws, because each one is a design lesson.

• Law 1, second reading: the leaf is a bounded loop of exploration. "Exactly one tool call" is true for each verified try. A leaf whose check fails does introspection again, and it selects the single next action. It can alternate freely between an edit of the source and a compile or a run. It continues until its check passes, or until a budget stops it. A leaf with a fixed action cannot recover from a failure whose fix is a different action. An example is a node that only compiles, while the fix is a rewrite of the source. The loop of write, compile, and fix is the center of iterative code work.
• Law 5, addition: the file system is the ground truth. For work with files, the declared closure alone stops the run. The artifact identifiers of a weak model do not reliably match the real files. Also, a compile step cannot fix a source file that it cannot see. Thus, minimal context must include two more items. The first is the actual list of the working directory, with the content of the files, at a bounded size. The second is the standard output of the last run. The workspace is the real shared board. Compare the shared-store systems for many agents [28]. The declared dataflow is a hint for routing over the workspace. It is not a replacement for it.
• Law 7, relaxed: only a consumes with no source blocks the plan. A hard rejection for a duplicate or unused produces stops weak models, which declare these fields incorrectly all the time. The store keeps the latest value, so those errors are harmless. Enforce only a consumes with no source. At the root of the task, accept even that, because the disk is the truth.
• Laws 4 and 6, hardened: the integrity of the check. A recovery can correct the command of a check, for example from pi.exe to .\pi.exe. It must never make the check weaker. We rank the checks: output_equals > run > compile > file_exists. The harness accepts a corrected check only at the same rank or a higher rank. If it does not, a weak check from a small repair lets an old binary pass incorrectly. The check must travel with the corrected action. It must not stay frozen at its first form.
• Law 3, refined: plans also decompose late. A full plan with many levels in one reply is the same anti-pattern as a full program in one write_file call. A larger reply gives more fragile JSON. Thus, the root fails hard and the run dies in minutes. The root emits only the top-level phases: 3 to 7 phases, one level. Each phase decomposes itself when the loop reaches it, and it uses everything that is already built as its context. The discipline of ADaPT [11] applies to the plan, and not only to the execution.
• Law 4, extension: a check must be durable. "Verified one time" is not "verified now". A later edit can break a primitive that the harness already committed, and nothing shows this. Also, a weak single check such as 999·999 passes while a precision defect ships. That defect appears 100 digits later as "the result diverges". Thus, the checkpoint law implies a regression discipline. The harness keeps every passed check as a persistent suite. It runs that suite again when the dependencies of a check change. A phase is complete only while its full suite stays green. Without this discipline, decomposition builds on a base that moves.

3.4 Architecture

                ┌───────────────────────────────────────────────────────┐
   task  ─────▶ │  Scheduler  (depth-first walk of the Step tree)       │
                │   ├─ Context Manager  -> builds the MINIMAL prompt     │
                │   ├─ Model (expander) -> emits an atom or a list       │
                │   ├─ Executor         -> does the ONE tool call        │
                │   ├─ Verifier         -> tests `success` (compile/test)│
                │   ├─ Corrector        -> on failure: fix, retry, split │
                │   └─ Tree/State Store -> plan, status, artifacts, log  │
                └───────────────────────────────────────────────────────┘
                         Tool Registry  (small, atomic, selected)
   read · write_file · edit_file · replace_lines · append_file · run · git_revert · note

• Scheduler. It walks the tree depth first. It enforces the laws. It controls the retry budgets. It also controls the removal of the scaffold. To do this, it computes an observed_difficulty value from telemetry at runtime. That telemetry has five signals: the retry count, the tokens, the Verifier failures, the decomposition depth, and the elapsed time. It gives more scaffold when the difficulty becomes visible, and less scaffold after a sequence of successes. Difficulty is an observed control variable. It is not a claim of the model. This is the same inversion as the difficulty_prior field. Comparable runtime systems work in this way [26][27]. Note that we say "observed", and not "learned". The harness computes the value deterministically, with no training loop.
• Context Manager. This is the central part. It replaces passive reduction of the context with active construction of the context for each step. It resolves the declared consumes of the current Step against an artifact store that names each artifact from its content. It then builds exactly four items: the goal, the declared inputs, the ground truth of the workspace, and the schema. It follows the declared edges. It does not reconstruct "what is relevant" with a heuristic. Thus, it replaces one fragile guess, from which a small model cannot recover, with a prompt that the harness can memoize from its inputs.
• Model, the expander. This is the only learned part. It receives a Step and the minimal context. It returns an atom or a list, and nothing more.
• Executor. It does the single tool call. It captures the standard output, the standard error, the exit code, and the artifact.
• Verifier. It changes the success field into an automatic check: a compile, a unit test, an assertion, or a difference. This is the check that controls the loop.
• Corrector. It is not a "reflector". It is a repair pass with a fresh context. It receives the goal, the artifact that failed, and the exact error. The Verifier stays the control. The self-assessment of the model is never the control (§2.2).
• Tree/State Store. The plan is the state. The store has two parts: an artifact store that names each artifact from its content, and an event log that only appends. Thus, the state is persistent, inspectable, resumable, and comparable. A person can pause a run, edit it by hand, and resume it.

Strict mode is the posture for weak models over this engine. It has four elements:

1. It forces the root to decompose. This removes the false "done" report from one atomic write.
2. It checks the goal after each step, and it also checks the goal of the full task.
3. It grows the tree dynamically. A leaf that cannot advance commits its best result and grows a sibling for the fix. A root that does not reach its goal grows a child for the fix. Thus, the tree never has a dead end. It blocks only at the global budget of steps.
4. The caller can supply a precise oracle. This gives success criteria above the precision of the model.

3.5 Operational semantics

eval(step, store):
  ctx    = context_manager.assemble(step.consumes, store)   # declared closure (law 5) + workspace truth
  filled = model.introspect(step, ctx)                      # Intent + (Action or decomposition)

  if filled.size == needs_decomposition:
    children = model.expand(step, ctx)
    assert dataflow_valid(children, store)                  # law 7: repair a consumes with no source FIRST
    for child in children: eval(child, store)
    return

  # atomic leaf: exactly one tool call for each verified try (law 1)
  budget = budget_for(step.difficulty_prior)                # the prior seeds it; telemetry replaces it
  loop:
    obs = executor.run(filled.action)                       # Observation:  a machine makes it
    ok  = verifier.check(step.success, obs)                 # Verification: a machine makes it
    if ok:
      store.commit(step.produces, obs); return              # write the new state (event log)
    store.observe_difficulty(step, retries, tokens, elapsed)
    if budget.exhausted:
      grow_fix_node(step) or store.mark_blocked(step); return   # never do useless actions
    fix = corrector.repair({goal, failing_artifact, exact_error})  # fresh context, NO transcript
    filled.action = fix; budget.decrement()

The mechanisms come directly from these semantics. Decomposition is the compound branch. Retry and correction are the loop of the leaf. The move backward is the growth of a fix node, or the block, when the budget ends. The model appears on exactly two lines, and it produces only the Intent and the Action. The deterministic controller does every other line.

3.6 The event log: what is free, and what you must earn

Each Step becomes hard when it has the four parts Intent, Action, Observation, and Verification [23][24][25]. This gives real properties to the system. But we must separate those properties, because some are free and some need more work. To combine them is exactly the overreach for which an earlier draft of this work was retracted.

Free, from the log:

• Resumability. The system restarts from the last committed leaf.
• Rollback and the move backward. The state of the tree is a projection over the log.
• Audit and time travel. Every action is one recorded tool call with a check.

These are properties of the log. They are not properties of the model.

Earned, and controlled by the discipline of the tools, not by atomicity:

• Deterministic replay means a replay of the log. The system returns the recorded result of each Observation. It does not execute the steps again. Model calls and most tools are not deterministic. This is exactly why durable-execution engines forbid them in deterministic code and record them instead. In a replay, a side effect does not execute again. It returns the recorded result [48].
• Memoization needs two things: a key from the content of the inputs of a leaf, and deterministic tools. This is the model of Nix and Bazel. The cache key is a hash of the inputs of the action. The system does not run the action when the output exists.
• Distributed execution needs idempotent effects. Atomic effects are not enough. A checkpoint alone prevents neither a duplicate execution nor a distribution of the work.

Thus, jh earns these properties only for leaves whose tools are pure or deterministic, or whose tools carry an idempotency key. The Tool Registry marks such tools. Our target model is weak, and it emits shell and file operations that are not idempotent. Thus, these properties are deliberate engineering decisions. They are not consequences of the law of one tool call.

A correction that we measured. The comparison to a build system was important, but in the direction that this design did not emphasize: the invalidation of a derived artifact. One failure dominated every early run that we analyzed, and it includes the first success. The check ran an old compiled binary after an edit of the source.

This made dozens of consecutive edits impossible to distinguish. Good edits and bad edits failed in the same way. It also made false loops where the run appeared stuck.

In one run, it discarded a correct fix that the model had already written. The cure is the discipline of make and Bazel, in reverse. The harness owns the records of the input hash of each product. It refuses to run an old check. It runs again the last successful command of the model that made the product. It also caches a failed check that did not change, instead of a new execution.

Memoization is future work. Invalidation is a basic necessity.

4. Calibration of the trigger for decomposition

§3.2 gives a threshold: split a leaf when its closure cardinality or its dependency density is above a critical value c*. The threshold is only as good as c*. Also, c* changes with the model and with the domain, because the boundary moves when the model or the toolset changes [5]. Thus, we must measure it.

jh can measure it unusually well, because the Verifier is an objective order parameter. [5] needs a special probe for the fidelity of the world state, and a full grid search. jh reads the collapse directly from the check that it already runs. Every committed leaf and every blocked leaf is one labelled sample of (complexity, pass or fail). The event log is the response matrix. On free local inference, these probes cost almost nothing.

The probe is adaptive. It is not a grid.

1. Order parameter. p(c) is the pass rate of the Verifier for a leaf whose declared structural complexity is c = (cardinality, density), at a fixed domain and a fixed toolset.
2. Staircase to the midpoint. A grid sweep costs O(grid). Instead, run a transformed up-down staircase. Raise c after a pass. Lower c after a blocked leaf. This converges on the crossing at p = 0.5 in O(log) probes [29]. That crossing is the estimate of c*.
3. The sharpness gives the margin. The variance, or the finite difference, of p(c) has a peak at c*. In statistical mechanics this is the susceptibility. A tall and narrow peak confirms a real transition [5]. Its width is the safety margin.
4. Set the trigger below the band. Force a split when the declared c is above c* − margin. Inside the band, and above it, split always. Ignore the claim "atomic" from the model.
5. Track it online. The observed_difficulty value estimates c* again with an EWMA over recent outcomes of the checks. The removal of the scaffold is the upward drift of c*, when the model warms or when a stronger model replaces it. Thus, the result of [5] costs jh only a new estimate. That result is: stronger models move the boundary, but they do not remove it.

One scalar for ability, if you want one. Item Response Theory uses the same pass and fail data. It puts the difficulty of a task and the ability θ of a model on one latent scale. The boundary is at θ ≈ difficulty. Adaptive testing makes this efficient.

Selection of items by Fisher information fixes θ to a given precision with tens of items, and not with thousands. One report shows up to 90% fewer items, and 41 items instead of 5,600 on HellaSwag [30]. It follows tinyBenchmarks [31]. This makes the point "measure it, do not count parameters" rigorous.

One honest caution comes directly from [5]: the collapse has many dimensions, six axes. Thus, one θ is a projection. It is a good prior for a retry budget. It is not a full description of capability.

What the probe does not give. The value c* is specific to the current Verifier, toolset, and domain. The synthetic tasks for the calibration must look like real leaves. If they do not, c* is wrong. All of this also needs an objective check that costs little. In a subjective domain, where a model judge gives the score, a model can hack the order parameter itself.

Measured. The first calibration on qwen3.6-35b-a3b gave c* ≈ 15.67, with a band from 13 to 19, on a test group of summation tasks. On the full pi task, the floor for decomposition was at first "the entire task". This is a difference above 10 times between two domains, as we predicted.

The sharper result is a third axis: c* is also specific to the harness. Thirty-two iterations of the harness moved the effective floor of the same model, on the same task. It moved from "cannot make a usable decomposition at the root" to "a verified success with 100 digits in 22 minutes". Thus, every published c*, and ours also, is a property of the triple (model × domain × harness).

5. Methodology

5.1 Setup

The think channel is off for the calls that do introspection and correction. The think channel is the private thought text of the model, inside <think> tags. With the channel on, the model sometimes uses the full token budget inside <think>. It then returns empty content with finish=length. With the channel off, the model returns the JSON directly, approximately 40 times faster, and it does not run away (§6.6).

5.2 Protocol

• Runs with fixed seeds, and a fixed time limit for each run. The limit is 15 minutes for the acceptance test groups, and 40 to 45 minutes for the hard tasks. A rig executes the runs. For each run it records the calls, the elapsed time, the parse failures, the best score, the restores, and the signatures of the stuck states. It appends all of this to a ledger.
• Sequential compared to concurrent. A concurrency of 4 matches the serving configuration. At 4 ways, the latency of each call rises by approximately 1.33 times, for approximately 3 times the throughput. Concurrency taxes the latency of each call. Thus, conditions that compare rates always run at the same concurrency. The sequential condition is the control with one variable.
• Predictions before the runs. Each wave registers its predictions before the test group runs. §6 reports each prediction as held, failed, or exceeded. It includes the failures.
• Analysis at transcript level. A count alone is not admissible. We read the failures of every wave at transcript level, and we give each failure a class from the taxonomy (Appendix A). We use the label "model-bounded" only for a failure whose anatomy shows a real defect that the model did not crack, with visible diagnostics.
• A probe before each condition. A cheap probe runs before every test group. A probe failure with no explanation blocks the test group. We adopted this rule after we dismissed one probe failure as seed variance. We then used four characterization runs to prove that the failure was real.
• Thermal safety. The rig samples the temperature of the GPU. It pauses new launches at 85 °C. The observed sustained peak was 69 °C to 78 °C. The critical trip point is 104.8 °C.
• Protocol for the judge, Tier 3 only. A frontier model, Claude, grades the artifacts of each run after the run. It grades four properties: fidelity of the voice, originality, coherence across chapters, and emotional structure. The worker never grades itself. The judge never controls a run. It grades the wave.

5.3 Rules of honesty that we adopted during the program

We adopted each rule after we broke it.

1. Never edit a rig while its own test group runs. The batch reads the script again for each child process, so two of six runs silently carried new code.
2. A test that passes must be able to fail. One integration test passed with no effect for a full wave, because a configuration patch never loaded, and the code path under test never ran.
3. Count structured events, not raw strings, when you search a transcript. Recovery prompts repeat the detail of the failure, so raw counts are approximately 2 times too high.
4. A capability probe must use a realistic prompt size. A probe with a short prompt on an idle server gave a false positive. At that length, the overhead for each token was invisible.
5. A verdict about reliability needs 6 runs or more, with fixed seeds and with an analysis of each failure. A demonstration is not enough.

6. Results

6.0 Summary

6.1 Tier 1 — code and build

The hard task, the first success. The prompt was the original prompt of the user, which started this design. The harness did not know the path to the compiler, and it did not know the environment. From that prompt, the harness drove qwen3.6-35b to a compiled pi.exe. The program printed all 100 correct digits of pi. We verified that the digits were not hard-coded. The run took approximately 22 minutes, 72 model calls, and approximately 10 structured rounds of fixes.

The check did exactly what §3.4 promises. It caught a real compile error from an #include that was absent. It caught a real defect in the output, where Pi = 0. It caught a write of a source file that was not complete. A harness that accepts a free-text report of "done" ships each of these defects. The model found the friction with the PATH of w64devkit by itself, with the help of the error feedback after each step.

The hard task, honestly. Across ten measured waves, the rate for each try never moved:

Note on wave 1: we later found that its result came from a time limit of 30 minutes, and not 45 minutes. This was a methodology incident. It produced honesty rule 5 in §5.3, and it blocked the protocol. We report the number here as we recorded it.

The successes became much faster: from 22 minutes to 15 minutes, and from 72 calls to 51 calls. The failures became much cleaner: every failure that remained carried a named class from the taxonomy, and not an opaque stop. But the rate is flat. This is the central negative result of the paper, and it drives law L2 (§7).

Individual mechanisms did work, and we verified them live. Quote drift fell from 15 to 42 events for each run, down to 1 event. Quote drift is the error old_string not found, when the text that the model quotes does not match the file. Build damage, the error REBUILD FAILED, fell from 58 to 2. Three more classes fell to 0:

• the render of a file that is not complete;
• hard blocks from cannot_split;
• hard blocks from a root that the parser cannot read.

This work bought one thing: it made the floor of the model visible. Two defects remained. The first is a bignum multiply of 64×64 to 128 bits that gcc accepts.

The model knows unsigned __int128, but it reaches for MSVC intrinsics that do not exist, and it does this inconsistently. The second is a correct Taylor recurrence for the arctan function. Its result diverges after 21 to 51 digits. A generic hint about gcc idioms lifted the first defect. Nothing in the harness lifted the second.

The new frame, and the acceptance bar. The pi task measures too much. It tests the recall of a formula, and that is not the competency that jh targets. jh targets the transformation of a static specification into a program that works. The model demonstrably reconstructs the formula of Machin at compute time.

A 35B model at 4.75 bits does not carry the formula word for word. That behavior deserves its own study, and here it is a hidden cause. Two corrections followed. First, give the model the specification as a file in the workspace, and test the transformation. Second, set the acceptance bar on tasks with a modest horizon and a high frequency. On those tasks, the reliability of the harness is the property under test.

That test group has three tasks and a time limit of 15 minutes. The tasks are the discovery of the toolchain, a build from two files, and a repair of a broken build. It measured 10 of 10 against a bar of 9 of 10. The total clock time was 9 minutes at a concurrency of 4. The runs used 4 to 26 calls, and 25 to 425 seconds. Five behaviors all pass mechanically:

• discovery of the toolchain;
• friction with the PATH;
• honest compile problems;
• a link of two source files;
• the repair of a defect that we planted.

This is the first bar that the program ever met. It is also on exactly the class of task that the new frame identified.

Note on the specification hypothesis. The specification file removed the recall problem cleanly. There were 0 degradations at the root in 7 runs, and the result was stable across seeds. But it did not lift the rate of the pi task, which stayed at 1 of 6 in sequential runs. The limit is the long work on the primitives, plus a defect in the print code. The limit is not the recall of the formula. We report this because it proved our own prediction wrong.

6.2 Tier 2 — mechanical media pipelines, the test of generality

The architecture claims that only the Tool Registry and the Verifier are specific to a domain. We tested this claim as directly as possible. We changed only the task prompts and the Verifier. The new Verifier is a graded ffprobe oracle over media artifacts. The engine stayed identical, byte for byte.

The test group has three tasks: render a title card, multiplex video and audio, and repair a broken render pipeline. The time limit was 15 minutes, at a concurrency of 4. The result was 9 full successes out of 9, in 2 minutes of total clock time.

The runs used 4 to 9 calls, and 29 to 67 seconds. There were zero parse failures in all nine runs. Thus, the repair layer on the client (§6.5) moved into a domain for which nobody tuned it. We predicted 7 of 9 or more, and the result exceeded it. We predicted no changes to the engine, and that prediction held.

Thus, Tier 2 joins Tier 1 as passed, and the tier framework becomes data instead of argument. The controller generalizes. The guarantee follows the check. Where a mechanical check exists, the same controller gives reliability at an acceptance level.

6.3 Tier 3 — prose with a judge, where the size of the atomic Step was wrong

The prose pilot asks the model to do three things. First, analyze the voice of a human author from an exemplar that we supply. Second, make a plan. Third, write an original novel with many chapters in that voice.

The task needs no knowledge of a formula. It needs control of the register, which is the style and the tone level of the writing. The check at run time stays mechanical:

• the artifacts exist;
• the word count of each chapter is inside a band;
• there are zero markers of the assistant register;
• the sentence length is inside a band, compared to the exemplar;
• no lists and no headings appear in the prose.

The external judge gives the verdict about quality (§5.2).

Four laws for the harness came from probes, before any test group ran. First, a mechanical check against copy is obligatory. The first instinct of the model was to copy the exemplar, in 3 probes of 3. Second, the primary defect must speak first, in the detail of the oracle. Third, the word bands must come from the scale of the exemplar itself. Fourth, every directive must be possible with the actual toolset.

The main result is the retrieval attractor. When a memorized exemplar is in the context, the model collapses into a copy of that exemplar. It does this even after we reject the copy explicitly.

The cure is withdrawal of the exemplar. The model analyzes the exemplar. Then the exemplar leaves the workspace. Then the model writes from its own notes. The harness enforces this mechanically. After the withdrawal, the overlap of 12-word sequences between chapters was zero.

Three waves then gave 0 of 6, 0 of 5, and 0 of 6. The reason teaches a lesson. Each wave closed one path that lets the model satisfy a check without the work. The failure mass then moved to a different path. It did not disappear.

• A check that inspects one file alone invites a copy of a sibling file, when a revision becomes expensive. One chapter, revised to satisfy a rhythm check, came back as 96% of its sibling chapter.
• We looked backward, and we found that the one "success" of an earlier wave was empty. Its chapter 2 reused 66% of chapter 1, in one literal block of 7,359 characters. The pass of the judge did not see this. Thus, the judge is fallible input. Everything that a machine can check must live in the shell.
• The pressure moves to the largest unit that nobody measures. After we checked the duplication of blocks, one run reused an identical refrain of approximately 45 words as the end of three chapters. That refrain was below the threshold for each chapter.
• A condition with a longer time limit settled the hypothesis about pace. The result was 0 of 6 with a time limit 37% longer. More time is not more success. The extra minutes go into the same long work.

The conversion came when we applied the central move of jh to prose. The size of the atomic Step was wrong. A chapter of 2,000 words as one verified tool call makes every retry catastrophic. We made the atomic Step a beat run. The harness checks the beat sheet plans mechanically.

An append_file tool lets a chapter grow in verified increments of approximately 600 words. A mechanical command oracle-check <file> goes to the runner, which intercepts it, so the oracle answers the check of a step directly. Thus, the harness is the compiler.

Measurement had already shown two facts. The necessary decode time was only 6 to 9 minutes. The deaths at 40 to 55 minutes came from the economics of the harness. Four items caused them:

• a tax of 36% of the calls for the goal check;
• a regeneration of a full chapter after each failed check;
• repeated loops at the same temperature;
• competition for the server.

Result: 5 full successes out of 6, in 48 minutes. We predicted 3 of 6 or more, and the result exceeded it. The median clock time of a completed run was 28 minutes. The share of the calls for the goal check fell from a range of 24% to 36%, down to a range of 2% to 14%. The immediately previous results were 0 of 6, 0 of 5, and 0 of 6. The test groups after this one held at 4 of 6, 4 of 6, and 5 of 6.

We found and fixed four harness defects inside that wave. Three of the four were ours:

1. The dead end at the root. The law that forbids a dead end protected every node except the root. The root has no parent. Thus, the harness cannot grow a sibling on it.
2. A tool with no entry in the tool table of the prompt. The model learned the shape of the argument only from 8 to 17 rejections for each run.
3. A defect with the PASS token. Our own oracle answered a pass with prose. Thus, the model wrote a check for an exact output against a substring of our status line. A step that genuinely passed then failed five times.
4. Prose has no lines. A block of 600 words gives an editor that works with coordinates nothing to address.

One more law came from that wave: completeness before polish. The rule "the primary defect speaks first" is correct inside one artifact. It inverts across artifacts. Our oracle checked the chapters in order, and it stopped at the first problem. Thus, a repeated phrase in chapter 2 had a higher rank than two chapters that were completely absent. The model then polished for 30 minutes.

The first change that converted the self-copy was mechanical, not informational. A write_file or append_file call fails at the tool, when its new text duplicates more than 50% of a sibling chapter. It also fails when the text carries 8 or more consecutive shared sequences of 12 words. The error names the sibling and gives the percentage. The file does not change.

The measurement is 5 of 6, against a baseline of 3 of 6 with the same seeds. Both partial runs of the baseline that had self-copy converted to success. The third run had its copy blocked, with 11 rejections and no duplicate committed. It then moved to a different failure signature.

There were zero false rejections of original prose across 79 calls. The judge confirmed that the converted chapters carry genuinely new events. They are not a paraphrase of the sibling. The worst overlap between chapters was 0.0%.

What remains on the side of the model. The verdict of the judge is consistent and precise across the waves. In 3 of 6 runs, the model makes an imitation that fully convinces, for some parts of the text. It writes real jokes of logic, asides from the narrator, and diction of the correct period. But 0 of 6 runs hold the imitation to the end.

By chapter 3 or chapter 4, every run returns to its native modern register, which is the earnestness of a growth story. In 4 of 6 runs, the model independently chose the same name for the protagonist. There was zero leakage of the assistant register. These are the measurements against the exemplar:

Two of these lines are important. The model names asides in its own notes about style, and then it writes none. Also, our own check on the mean sentence length induces the uniform polish that the table shows. Thus, a check can damage the quality that it measures.

So: surface diction and the architecture of the sentences transfer. The emotional distribution decays as the text becomes longer. The open competency is the ability to hold a register, and that competency is on the side of the model.

6.4 More tries at run time: the race method

The rate for each try stayed at approximately 1 in 6 across four consecutive conditions. Also, the Spark is limited by memory bandwidth, so concurrent sequences share the reads of the weights, and batch decode is almost free capacity. Thus, the correct next control is not more advice for each try. It is more tries.

We run N independent Strict runs on forked copies of the workspace. The first run that passes the oracle wins. The harness discards the other runs at their next step boundary, with a cooperative abort.

Measured: 4 races won out of 12, which is approximately 33%. The sequential rate is 1 in 6. The wins took 21 to 38 minutes. This is a genuine improvement of approximately 2 times for each unit of clock time.

The naive calculation 1 − (5/6)^N ≈ 63% at N = 6 does not survive the competition for the server. N = 6 gives too much load for a serving configuration of --max-num-seqs 4. The racers starve, and the latency of each call approximately doubles. Even at N = 4, a class of starved racers remained. Its cause was the cost of the JSON parse failures (§6.5), and not the depth of the queue.

The law of the race: strategy first, entropy last. The assumption of independence is false for our racers. They differ only by the seed. Thus, they write different sentences, and they make the same decisions.

In 4 of 6 prose runs, the racers chose the same name for the protagonist. In every pi run, the racers built the same bignum foundation. Racers with the same strategy fail for the same reason. A new seed buys variance in the execution. It does not buy coverage of the strategies. This is a second cause of the shortfall, beside the competition for the server, and we did not examine it.

Thus, a race has two currencies of diversity, and they have a rank. Strategy is information. It is scarce, and its yield is high.

Spend it first: give one distinct approach to each racer, while any approach remains unused. Entropy is tactics. It is cheap, and its yield is low. Spend it only for the remainder, when N is larger than the number K of real strategies. At K = 1, the method degrades to the race of today, exactly. This is the production form of the law in §4: buy diversity with information, not with entropy.

Comparison to tournaments that use a rank. A current prompt technique runs isolated sub-agents with distinct roles. A blind judge then selects a winner. The system refines the winner with the critique of the judge. The comparison lands on one axis: they rank, and we verify. Our race selects with the mechanical oracle, and it stops at the first verified winner.

We have evidence that a rank can be wrong, and the evidence is against our own judge. In one wave, the judge chose one run as the only success. Chapter 2 of that run reused 66% of chapter 1, in one literal block (§6.3).

An evaluator that ranks puts a copied chapter first. The two designs complement each other across the tiers. They are not rivals. Where the truth is executable, a check beats a rank on cost and on honesty. Where the truth is not executable, a rank is all that anybody has. Three elements of the rank design are worth adoption:

• the blind judge;
• the refinement of the winner with the critique of the judge;
• the display of the racers that lost.

On the last point, we discard N−1 complete attempts today.

6.5 The cost of parse failures, and the cheapest change that we found

At the scale of a probe, 29% to 40% of all model calls were Step fills that the parser rejected. This was the largest single consumer of the time limit.

The obvious cure was constrained decode with the vLLM option response_format json_schema. It removes the cost, with approximately 1 failure for each run. Its own probe cancelled it.

The wire format of a Step needs free-form objects. Such objects force the grammar backend off its compiled fast path, into a mask over the logits for each token. Completed calls took 8 to 154 seconds, and the p90 value hit the fetch cap of 300 seconds. Thus, the cure adds a latency cost of approximately 8 to 10 times, to remove a call cost of approximately 30%.

We removed the cost on the client instead. Of 32 real failures, 30 had three mechanical shapes. The client now repairs them. It does not reject them:

1. A reply with the shape of a tool call adopts the goal of the node, which the engine knows.
2. A size field that is absent comes from the structure.
3. Substeps that are strings become phase children.

The client never guesses outside strict boundaries. Measured: the rate fell from 29% and 40% to 0% and 6%, at normal latency. This is a fall of approximately 10 to 15 times in the largest consumer of the time limit. Every condition, every race, and every session inherits it automatically. This is the change with the highest yield for its cost in the whole program.

The model was never bad at JSON. The harness was strict about shapes that the engine can already disambiguate.

6.6 Probes of the sample settings and the format

A/B of the wire format. Free text first, then extraction, gave 7 of 10 valid against the schema. The strict json_schema constraint gave 8 of 10. The loss of reasoning that [17] documents did not appear for the fill of the Step schema.

A probable cause is that the schema is the scaffold, so the model has nothing to reason about freely. This agrees with [19], which finds no causal effect of the format in most scenarios. It is a property of this model. It is not a law.

The think channel. Disable the think channel for introspection and correction. With the channel on, the model sometimes uses the full budget inside <think>, and it returns empty content with finish=length. One prose condition ran with the think channel on and temperature 0.7. It measured 0 of 6.

Temperature. Two opposite ideas existed. The first said that the distribution of the quantized model had collapsed. The second said that a low temperature corrupts its JSON.

A probe of 2 minutes and the documentation of the vendor settled both. The file generation_config.json of the model gives a default temperature of 1.0. This is unusual, because most defaults are 0.7 to 0.8. Our server pins 0.6 on every call that we ever made.

We measured 6 runs for each temperature. Diversity rises, and the overlap of 5-word sequences between pairs falls to zero at 1.0 and above. The overlap goes from 0.021 in the range 0.3 to 0.8, down to 0.000 in the range 1.0 to 1.3. The distinct-3 metric goes from 0.92 to 0.99.

Thus, the distribution has tails, and the idea of a collapse is wrong. But the responsive region starts at 1.0, and the harness has never been there. Between 0.3 and 0.8, nothing moves, and eighteen waves ran inside that flat zone. The validity of the Step JSON was 6 of 6 at every temperature from 0.3 to 1.3.

Thus, the fear of corruption has no support in either direction. One caution: we measured this on a small step, with approximately 60 words of embedded content. We did not test the regime with long embedded prose.

This probe also caught a latent defect before it damaged a wave. A module for affective sampling had absolute ceilings for its penalties. The ceilings were not relative to the baseline of the server.

Our presence penalty was already 1.1. Thus, the clamp had the power to move the anti-repetition protection of the model downward. It had this power on exactly the defect that the protection exists to fight. The A/B test then gives the blame to the change.

6.7 The hostile harness as a hidden cause, observed directly

Before jh existed, we scored small models through a usual harness, which was opencode. We measured the intolerance of that harness. We did not measure the capability of the model.

The harness sits on a stack that throws an exception on an imperfect tool call. After the exception, the loop stops with no recovery. It killed runs at each of the four points below. None of them is a failure to write code.

• A tool call that the model emits as text, inside content. The form was <tool_call>{json}</tool_call>, or hand-made XML, which the parser of the server did not extract. The harness sees plain text with finish:stop. It treats this as "done", and it stops.
• An unknown tool name, or a name with the wrong case, for example Write instead of write. In sub-agents with no tools, this error propagates with no catch, and it crashes the turn.
• A mismatch of finish_reason, which is stop or length while a tool call is present. The loop exits silently.
• The wall at the file write. This one is decisive. We asked the model to write the C file. It emitted 18,000 tokens, and it wrote the program. The call had the whole source as its content argument. A code file inside a JSON string is the hardest thing to keep valid. The harness recorded parts: []. Thus, it dropped the entire output of the model. The model probably succeeded, and we scored it as a producer of nothing.

No configuration makes that harness tolerant. No setting, no agent option, and no plugin hook rewrites the response of the model before the validation. This is a third layer of hidden causes, after the sample settings and the truncation of the output. It is also the most dangerous layer, because a stopped turn and a dropped turn look exactly like a failure of capability.

Thus, a hidden cause damages every earlier verdict of the form "small model X failed this bar". This applies to any model that met friction with the format of a tool call. Such a score measures "emits JSON that the harness likes". It does not measure "can write code".

The distortion is also selective, because models that emitted clean calls escaped it. Thus, it silently reordered a ranking. It did not add uniform noise.

A proxy for the repair of tool calls, with the standard library only, sits between any client and the inference server. It gives this tolerance to any system today. It also proved the fact about buildability that matters: the failure modes are a short list that a person can enumerate. They are not an infinite sequence of new problems. This is the list:

• JSON inside content;
• names with the wrong case, or names with small errors;
• variants of <tool_call>, which include unclosed tags and flat arguments;
• XML of the form <tool>…<param>;
• normalization of finish_reason;
• repair of the escapes, with a request to write in chunks.

A proxy with five rules already walks a 35B model through a sequence with many tools. §6.5 confirms the same thesis inside jh.

6.8 The product

Strict mode faces the user. It has a configuration schema, a settings surface, and a toggle for each session. It also has a route that sends the turn of a session through the engine. Four legs decide if it is livable. We gated all four live against the real model, through the pipeline of the product:

1. The route for tasks and chat. Before, every message started a full engine run, and this included the message "thanks". Now one small router call decides. Ambiguity resolves to TASK.
2. The cooperative stop. The engine works on a detached fiber. The stop flips an abort latch. The harness cuts the calls in flight inside 500 ms. The engine exits through its terminal best-restore at the next step boundary. Thus, a stop never leaves a step half written.
3. Resume after a crash. A row with the status running marks a hard death. A bare command "resume" continues the saved tree. It does not do the committed steps again.
4. A real streamed final answer. The answer comes from the ground truth of the harness only. If the journal does not show that the harness verified a fact, the answer does not claim that fact.

Every engine action that changes the state materializes as a real tool part in the chat, on one shared message from the assistant. For a race, the harness replays only the actions of the winner, after the race resolves.

Two catches with empty checks are worth a record. One integration test passed with no effect, because a configuration patch never loaded against a data directory that already existed. One resume test matched old messages, because the message API returns the newest message first. Both are instances of rule 2 in §5.3.

7. Nine laws

These laws are the transferable output of the study. Laws L1 to L3 constrain how a report must give results for agents that use small models. Laws L4 to L9 are design rules that we will apply to any future harness.

L1 — On small models, the harness is the dominant experimental variable. [20] argues this independently. We observed it live (§6.7). More importantly, we observed it against ourselves, two times.

We read a defect of our own harness as a limit of the model. One time, we said "the model cannot crack this defect", while the harness hid the fixes of the model. The other time, the defect with the old binary (§3.6) discarded a correct fix, and then it sent the escalation in the wrong direction. Thus, a verdict about reliability is admissible only with an analysis at transcript level behind it. Corollary: every number for a capability floor is a property of the triple (model × domain × harness).

L2 — Instruction changes engage, but only mechanism changes convert. This is the most expensive lesson of the program. Many times, a change at the level of the prompt fired exactly as we designed it. The directive appeared. The model acknowledged it.

The telemetry showed that it engaged at the threshold. The rate of success did not move. See §6.1, waves 6 to 10, and the directive against duplication in §6.3, where the model offended again seven times. One type of change converted the behavior. That type makes the bad action impossible.

Two examples are a transactional edit check that rejects an edit that does not compile, and a refusal to write a duplicate. A third example is an editor with coordinates, which became obligatory after repeated errors in the quotes. Design implication: when a change must move an outcome, spend the engineering on the mechanism, and not on the words.

L3 — The capability floor belongs to the triple (model × domain × harness). Thirty-two iterations of the harness moved the floor of the same model on the same task. It moved from "cannot make a usable decomposition at the root" to a verified success with 100 digits in 22 minutes. The measured c* value has a spread above 10 times across two domains (§4). Thus, no reader can interpret a published capability number for a small model, when the report does not give the harness.

L4 — Push everything that a machine can check out of the head of the model, into executed mechanical checks. Treat what remains as fallible input, under machinery that suspects it. Never treat it as ground truth. The sharpest example is an oracle that was wrong from the start.

The model computes the expected values by hand. It gets them wrong. Then it fights its own correct code.

In one run, it computed the contradiction word for word, and it continued to mutate the source. Ordinary suspicion machinery cannot see this, because suspicion needs a registration, and a registration needs a pass. The mechanical closure is this.

A check can fail in a byte-identical way across different states of the source. That is sufficient reason to derive the test again. The verdict of a judge has exactly this status. It is never infallible. It is always eligible for suspicion. It is never the only check.

L5 — The size of the atomic Step must match the retry granularity of the domain. A chapter of 2,000 words as one verified tool call makes every retry catastrophic (§6.3). A new size of approximately 600 words, one beat run, moved prose from 0 of 6 to 5 of 6. The engine did not change. The selection of the atomic Step is a design decision for a domain. It is not a constant of the architecture. This is the design lesson here that transfers the most.

L6 — Pressure moves to the largest unit that nobody measures. Check a chapter alone, and the model copies its sibling. Check the duplication of blocks, and the copy returns as a shared refrain of 45 words, below the threshold (§6.3). Expect this movement. Also expect a check to sometimes damage the quality that it measures. Our check on the mean sentence length measurably induced a uniform polish. The coefficient of variation was 0.67 to 0.73, against 0.91 for the exemplar.

L7 — A check must be durable, not momentary. "Verified one time" is not "verified now". A check that passes becomes a persistent regression suite. The harness runs the suite again when the dependencies change.

If it does not, the decomposition builds on a base that moves (§3.3.1). The corollary in reverse: a harness that runs an old derived artifact makes good edits and bad edits impossible to distinguish. It also makes false loops, where a run appears stuck (§3.6). Invalidation is a basic necessity.

L8 — Buy diversity with information, not with entropy. Racers that differ only by their seed are not independent. Racers that differ by strategy are independent (§6.4). Rank the two currencies, and spend the information first.

L9 — Two regimes of execution fail in opposite ways, and their combination must have bounds. The monolith regime rewrites everything and reasons about the whole program in the context. It has no incremental safety, and it discards verified work. But it forces coherence across the whole program, so a lucky draw gives a correct whole.

The surgical regime keeps verified work and makes the cost local. But it can lock a foundation that is subtly wrong, and then it polishes the symptoms forever. Neither regime alone is enough for a weak model. This combination worked: incremental safety, plus a foundation that a regression suite protects, plus a bounded escape to the holistic regime. When one part continues to diverge, force a new derivation of that part alone, from nothing, against its stronger tests.

This reclaims the one virtue of the monolith, at the scale of a part. We validated this mechanism live. The single success of one wave reached 100 correct digits through this escape, which rescued a foundation with no progress.

8. Discussion

8.1 Where the guarantee lives

Every guarantee of jh needs an objective check that costs little. This includes the checkpoints, the external correction, and the property that useless repeated action is structurally impossible. This is why code and mathematics are the first domain: the check already exists, as a compiler, a test runner, or a difference. The RLVR literature finds exactly the same thing [32][33].

Thus, rank the domains by the cost of their check, and not by the size of their market. Data cleanup gives checks of a schema and of constraints. DevOps gives smoke tests.

Reverse engineering gives equivalence of a new execution. Subjective domains come later, in tiers that tolerate a judge. They are never the first domain. The roadmap follows the checks. It does not follow ambition.

8.2 What generalizes

The controller generalizes. The guarantee follows the check. §6.2 tests this as cleanly as possible. Only the prompts and the Verifier changed. The engine stayed identical, byte for byte. The result was 9 of 9.

Everything that is neutral to the domain moved with no change:

• the decomposition and the Step schema;
• keep-best and restore;
• the budgets and the time limits;
• the artifact store;
• the stream of milestones;
• the recovery that never has a dead end. One thing did not move automatically, and it is the thing that made every result honest: a success check that is objective and cheap. Tier 3 shows this clearly. A mechanical shell of checks exists, and jh can use it. But the core signal of quality is a judge. By our own law L4, a judge needs the same treatment as a test that never passed.

8.3 Economics

jh exchanges many cheap model calls for capability. On a paid frontier endpoint, that exchange is bad. On a local model, it is almost free.

Thus, the harness can afford much introspection, many checks, and many retries. The same economics make the self-calibration probes of §4 affordable. They also make the race method of §6.4 a rational use of memory bandwidth that the system wastes at other times. The scaffold is also independent of the model, so its value compounds as small models improve. The result of [5], that stronger models move the boundary but do not remove it, is exactly the claim that the scaffold stays useful.

8.4 A note on the prose tier

The direction with the most promise for Tier 3 is one that we did not measure. It treats the character as the specification. First, research a character and write its profile as a specification: voice, diction, register, error profile, values, and tendencies of behavior. Then model what that character says and does.

Do not "generate text". This pulls prose into the same thesis. The profile of the character is the specification. Its constraints of voice and behavior are the check.

The model generates inside stricter constraints. This also dissolves the decay of the register structurally. The model can ground itself in a persistent specification again. It does this in the way that it uses a specification of code. A single instruction "write like X" fades across chapters.

Two open questions have genuine interest. First, authentic dysfluency appears to couple to emotion. It is not random.

In raw human writing, the breakdown of the syntax and of the orthography follows arousal. If this is true, the target signal is not "N errors that we inject". The target signal is structural entropy that correlates with the emotional arc.

That signal is controllable, with modulation of the sample settings. It is also mechanically checkable: does the entropy of a passage follow the passages with high emotion, or is it uniform noise? That correlation is itself a candidate for a check. It also turns a target with low literacy into a gradable spectrum.

Second, the design of an evaluation must target a voice that discriminates. Take a canonical author whose register is already close to the default of the model. Then "the model followed the prompt" and "the model emitted its default" look the same. This is the trap of the empty test, applied to the design of an evaluation. Choose the task where the signal separates from the baseline.

9. Limitations and threats to validity

The sections below give the honest boundary of what this paper establishes.

9.1 No direct comparison, the principal limitation

We ran no controlled comparison against EffGen [37], SmallCode [40], little-coder [41], or Aider [42], on our tasks and on our hardware. A decision to build and to measure directly replaced the A/B test that we intended.

Thus, this paper does not establish that jh is better than any harness for small models that exists today. It establishes three things. The specific architecture of jh reaches the stated bars on the stated tasks.

A controller with mechanical checks generalizes across two tiers, with an engine that does not change. The laws L1 to L9 held through the whole program. Any claim of superiority is future work. The comparison that we did not run is the first point that a reviewer must press.

Our field observation in §6.7 is the closest thing to a comparison, and it is qualitative. A different harness held the same model below its capability, for structural reasons. That is evidence that the harness matters. It is not evidence of a ranking.

9.2 Statistical power and construct validity

• N is small. A test group has 5 to 10 runs for each condition. Several comparisons that sound decisive, for example 5 of 6 against 3 of 6, do not survive a significance test alone. We report them as directional evidence, with the mechanism from the transcript behind them. The run with the blocked copy convinces more than its count.
• One model, one quantization, one server, one operating system. Every number comes from qwen3.6-35b-a3b at approximately 4.75 bits, under vLLM, on Windows, with w64devkit. By our own law L3, these numbers do not transfer. The laws are the portable claim. The counts are not.
• The sample settings sat outside the responsive region of the model for the whole program (§6.6). Eighteen waves ran at temperature 0.6. The default of the model is 1.0, and diversity responds only at 1.0 and above. Read every result about diversity, and the yield of the race method above all, as a measurement inside the flat zone.
• The pi task measures too much (§6.1). It mixes the recall of a formula into a competency of transformation. We caught this in the middle of the program. Thus, the interpretations of the earlier waves are weaker.
• The task groups are small, and the author of the harness wrote them. This risks a tuning of the tasks to the harness. The Tier 2 group is the cleanest case: a new domain, an engine that nobody touched, and a check that we wrote fresh. It is the strongest evidence of generality that we have.

9.3 Conflicts of interest and the independence of the evaluator

Three items matter.

1. The author of the harness made the measurements, and wrote the task groups. No operator was blind.
2. Claude is both a co-author of this paper and the Tier 3 judge. Thus, the Tier 3 verdicts about quality are not independent of the authorship. Read them as commentary from an expert with an interest. The Tier 1 and Tier 2 results depend on no judge. They are the outcomes of a compiler, of tests, and of ffprobe. This is one more reason why we give them more weight.
3. A demonstration shows that the judge can be wrong. In §6.3, it chose as a success a run whose chapter 2 was 66% a copy. We report this against ourselves. It is the strongest single argument in the paper for the move of every checkable property into the shell.

9.4 A cascade of false retractions, a methodological result

We report this item at length, because we expect other persons to meet it.

On 2026-06-22, an earlier adversarial check ran over the related-work section of this work. It concluded that five cited systems "failed independent verification and are likely confabulated". It instructed that nobody must cite them again. The five were: SmallCode, the Aider Polyglot result of little-coder, "Three Roles, One Model", "Confucius Code Agent", and several other names. That retraction was itself the error. On 2026-07-26, we verified the systems again, directly against the primary sources:

We also verified again all 37 arXiv identifiers in the source document. All are real, and their titles match our use of them. A control query used an identifier with a valid syntax that does not exist. It correctly returned no entry.

Thus, the verification channel does not simply confirm whatever we ask it. The same pass also marked four bare names as invented: EigentSearch-Q+, TopoCurate, DeepVerifier, and GraphCue. We did not check those four again, and we do not cite them. Thus, we make no claim about them in either direction.

Two lessons follow. (a) A verification pass that uses a language model can make false negatives. It can say, with confidence, that real work is invented. These errors damage more than false positives, because they delete true prior art and they inflate a claim of novelty.

The asymmetry is worth a statement. Any reader who follows an invented citation catches it. A citation that a pass retracts incorrectly removes the exact competitor that a reviewer wants to compare. (b) A retraction needs the same standard of evidence as a claim: a fetch of the primary source, with the URL recorded. An absence of recall is not evidence.

Concretely, this correction changes the posture of the related work substantially. jh is not new ground (§2.5). Other work corroborates its thesis independently, in [38] and [41]. Its defensible increment becomes narrower: the obligatory mechanical check, the tier framework, and the laws.

9.5 Overhead, complexity, and the risks that remain

• Overhead. Many calls for each task make the system slow, and they use many tokens. This is acceptable only with free local inference. The policy that removes the scaffold is the control that exchanges intensity for speed.
• Complexity. The design has three parts: an artifact store that names artifacts from their content, the computation of the closure, and the validation of law 7. Together, they are much more than a function that builds a prompt. This is a genuine exchange between robustness and complexity. It is not a free win.
• The floor. Decomposition needs some planning from the model. A scaffold cannot help a model that cannot propose a sane split. The bar is far below the solo horizon, but the bar exists.
• Honest atoms. The problem "the model says atomic, and the Step is not atomic" is smaller now. §3.2 gives a structural control before execution, and the observed difficulty gives an empirical control after execution. But the problem is not solved. Also, consumes and produces are two more fields that a weak model can fill incorrectly.
• No formal guarantees. We use the vocabulary of control theory. We do not use its proofs. jh has no known dynamics, no explicit cost function, and no guarantee of stability or convergence. We do not prove that progress is monotone. We measure it, with the retry count, the depth of the decomposition, and the pass rate of the Verifier. Then we fail safe, into a clean report that says "blocked".

10. Conclusion

Small local models fail as agents because they have too little horizon. They do not fail because they have too little knowledge. Horizon can move.

Put it in the harness. The harness is a deterministic controller. It owns the plan tree, the context of each step, the checks, and the recovery. It lets the model do one thing only: emit one tool call, or emit one decomposition. Then a quantized model of the 35B class completes work that it otherwise abandons.

We measured this against three acceptance bars. Tier 1 gave 10 of 10 on tasks for a build environment. Tier 2 gave 9 of 9 on a second tier of checks, with an engine that was identical, byte for byte. Tier 3 gave 5 of 6 on prose with a judge, after the size of the atomic Step matched the retry granularity of the domain.

The negative results have the same value. Ten measured waves of correct and validated changes never moved the per-try rate of a hard task above approximately 1 in 6. A time limit that was 37% longer moved it not at all. The most reliable pattern of the whole program is this: advice engages, and only mechanisms convert.

The corollary for the field is uncomfortable, and we think that it is correct. On small models, the harness is the dominant experimental variable. Thus, a capability number for a small model, reported with no harness, measures the harness at least as much as it measures the model. We know this because we made that same error two times, against our own model, inside this program.

Availability. The engine, the rigs, the measurement ledgers of each wave, and the failure taxonomy accompany this work.

Appendix A — The failure taxonomy, abridged

These classes accumulated over the program. Each failure of a test group gets one class. We designed families A to E before the measurements. The measurements forced families F to K. That fact is itself the pattern of law L1 in miniature.

Two entries are worth a read as design artifacts. I6 shows that the harness can break a verified primitive silently, and that nothing forces the suite green before the next phase. It produced law L7. K6 shows that the model computes a wrong expected value by hand, and then fights its own correct code. Suspicion machinery that needs an earlier pass cannot see this. It produced law L4.

Appendix B — The verification status of the citations

On 2026-07-26, we verified every arXiv identifier below against the arXiv API. Where a claim carried weight, we also verified it against the abstract page. We verified the non-arXiv sources against their primary pages. A control query with an identifier that does not exist returned no entry. Thus, the channel does not confirm indiscriminately. §9.4 explains why this appendix exists.

References

[1] Dziri et al. Faith and Fate: Limits of Transformers on Compositionality. arXiv:2305.18654 (NeurIPS 2023).
[2] Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoning. arXiv:2601.02902.
[3] Exploring State Tracking Capabilities of Large Language Models. arXiv:2511.10457.
[4] The Topological Trouble With Transformers. arXiv:2604.17121.
[5] World-Model Collapse as a Phase Transition. arXiv:2606.31399.
[6] Liu et al. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172.
[7] Shinn et al. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366.
[8] Madaan et al. Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651.
[9] Kadavath et al. Language Models (Mostly) Know What They Know. arXiv:2207.05221.
[10] Do Large Language Models Know What They Are Capable Of? arXiv:2512.24661.
[11] Prasad et al. ADaPT: As-Needed Decomposition and Planning with Language Models. arXiv:2311.05772 (NAACL 2024 Findings).
[12] Gao et al. PAL: Program-aided Language Models. arXiv:2211.10435.
[13] Chen et al. Program of Thoughts Prompting. arXiv:2211.12588.
[14] Wang et al. Executable Code Actions Elicit Better LLM Agents. arXiv:2402.01030.
[15] Kim et al. An LLM Compiler for Parallel Function Calling. arXiv:2312.04511.
[16] Khattab et al. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. arXiv:2310.03714.
[17] Tam et al. Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models. arXiv:2408.02442 (EMNLP 2024).
[18] JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models. arXiv:2501.10868.
[19] Yuan et al. Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inference. arXiv:2509.21791.
[20] Stop Comparing LLM Agents Without Disclosing the Harness. arXiv:2605.23950.
[21] LLMPC: Large Language Model Predictive Control. arXiv:2501.02486.
[22] Grounding LLMs For Robot Task Planning Using Closed-loop State Feedback. arXiv:2402.08546.
[23] The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents. arXiv:2511.03690.
[24] ESAA: Event Sourcing for Autonomous Agents in LLM-Based Software Engineering. arXiv:2602.23193.
[25] Fowler, M. Event Sourcing. martinfowler.com, 2005.
[26] CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning. arXiv:2603.08659.
[27] Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents. arXiv:2605.22166.
[28] LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science. arXiv:2510.01285.
[29] Levitt, H. Transformed Up-Down Methods in Psychoacoustics. Journal of the Acoustical Society of America 49(2):467–477, 1971.
[30] Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks (ATLAS). arXiv:2511.04689.
[31] Polo et al. tinyBenchmarks: evaluating LLMs with fewer examples. arXiv:2402.14992.
[32] Lambert et al. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv:2411.15124.
[33] DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.
[34] Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewards. arXiv:2506.00103.
[35] RLPR: Extrapolating RLVR to General Domains without Verifiers. arXiv:2506.18254.
[36] Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains. arXiv:2503.23829.
[37] EffGen: Enabling Small Language Models as Capable Autonomous Agents. arXiv:2602.00887 (ICML 2026).
[38] McClendon et al. Three Roles, One Model: Role Orchestration at Inference Time to Close the Performance Gap Between Small and Large Agents. arXiv:2604.11465.
[39] Wong et al. Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases. arXiv:2512.10398.
[40] SmallCode — AI coding agent optimized for small LLMs. github.com/Doorman11991/smallcode.
[41] little-coder — a harness optimized to smaller LLMs. github.com/itayinbarr/little-coder.
[42] Aider. Linting and testing. aider.chat/docs/usage/lint-test.html.
[43] Zhou et al. Language Agent Tree Search Unifies Reasoning, Acting and Planning in Language Models. arXiv:2310.04406.
[44] Huang et al. AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation. arXiv:2312.13010.
[45] Islam et al. MapCoder: Multi-Agent Code Generation for Competitive Problem Solving. arXiv:2405.11403.
[46] Xia et al. Agentless: Demystifying LLM-based Software Engineering Agents. arXiv:2407.01489.
[47] Kwa et al. Measuring AI Ability to Complete Long Software Tasks. arXiv:2503.14499 (METR).
[48] Temporal. Side effects and deterministic replay. docs.temporal.io.
[49] LangGraph. State, channels, and typed graph state. langchain-ai.github.io/langgraph.