FABERTry the beta
⟵ Back to the journal

Research note

The harness.

AI changed how software gets written and changed nothing about how parts get designed. The difference is not the models — it is that software already had the environment, and mechanical engineering has to build one.

· Faber · 1 430 words · about 6 min

In 2023, the best language model in the world could resolve 1.96% of real GitHub issues[1]. Three years later, frontier systems resolve around 80% of them[2]. That is one of the fastest capability jumps the field has produced, and it did not happen because the models got bigger. It happened because software engineering was, by accident, the perfect place for it.

Writing software already had everything an agent needs: a test suite that is green or red, and GitHub — version control, so a wrong attempt costs nothing, and every solved problem already written down somewhere. Neither was built for machines, and both turned out to be exactly what a model needs: a verdict on every attempt, and nothing lost by being wrong.

Now look next door, at mechanical engineering. Every physical object — a bracket, an engine block, an aircraft — is designed, manufactured and validated through software too: CAD for the shape, CAM for the machining, simulation for whether it survives. A CAD program even compiles: it runs in the kernel or it errors. But no test goes red when a wall is too thin to cast, nothing makes a wrong attempt free, and no surface lets a model act.

The reasons are structural, not temporary. The interface was built for human eyes and hands: decades of GUI over a proprietary kernel. The reasoning is physical, not linguistic — geometry, tolerances, materials, process logic. And the know-how is not on the internet: it lives in engineers' heads and behind firewalls. No amount of scale fixes any of the three. What is missing has to be built rather than scraped: the surface a model acts through, the checks it gets back, and the loop between them — a harness.

Coding is where models are strongest and the labs have optimised hardest. Even there the harness decides: the same GPT‑5.5, in the same week, scored 61.5% functional correctness inside Codex and 87.2% inside Cursor[6] — twenty-six points, no change of model. It is worth more still where the model is weak and there is no harness at all. A companion note makes the case that the thing to build one on is code; this note is about the three things that have to exist around that program.

MODELfrontier today, our LMM nextPROGRAMDSL of engineering intentKERNELexecutes the codeVERIFYmass · asserts · criticDELIVERABLEmodel · drawing · toolpathemitsrunsgeometrypassesACI — every face, its normal, area, centroidfails — back to the model with the measured reason
The loop: the agent reads the state of the part, writes a program, the kernel runs it, and nothing reaches the deliverable until the checks pass. A failure comes back as a measurement rather than as a stack trace.

The ACI

Start with how you read a part. Below is a real solid: click and drag, turn it over, look underneath, come round to the back. You did not read that part so much as interrogate it — changing your point of view until the shape stopped being ambiguous.

A part built by this loop. Click it, then drag — turn it over, look underneath, come round to the back. Whatever you wanted to know about it, you found out by moving.

A model gets pictures and text instead. The limit is not the viewpoint — it is that a render carries no numbers. You cannot read 12.0 against 12.4 off one, or a wall thickness, or whether two holes are concentric or merely close. The engineer at a workstation reads a model through pixels, every dimension one click away. Take the numbers out and a CAD viewer is a photograph.

So perception has to be rebuilt in a form a model can use. The name for that surface comes from software agents: an agent-computer interface, or ACI, designed for an agent rather than a person[3]. The ACI is to the agent what the GUI is to the engineer.

Ours hands the agent a measured account of the part before every step, and lets it ask the kernel whatever that account leaves out. The part stops being a picture to interpret and becomes an object to question. One rule matters more than the contents: this has to be pushed, not offered. A capability the model must remember to ask for is not a capability, it is a probability.

The reward

Give a capable model a goal it can measure itself against and it will hillclimb on it: try, measure, see which way the number moved, go again. What makes a model good at it is training: reinforcement learning from verifiable rewards — RLVR — selects for exactly this behaviour[4]. It is also most of why coding agents work: a compiler and a test suite hand them a verdict on every attempt.

Mechanical engineering hands them the compiler and nothing else. So the unglamorous work is not making the agent cleverer — it is building the hill it can climb: measurements it can take by itself, cheap enough to take at every step, that move when the part gets closer to right.

Concretely, the harness closes the loop before anything is accepted. Every build asserts its own mass against what the material and the geometry imply; a vision critic compares rendered views against the request; execution errors are matched against known failure modes and turned into a targeted fix. A failure comes back as a measurement rather than a stack trace, which is what lets the agent recover in an attempt or two instead of looping.

One example, and the most ordinary task in the field: a drawing goes in, a solid comes out.

A technical drawing of the flanged plate: a section A-A on the left, a face view on the right, with diameters and overall sizes called out.Five renders of the built solid — an isometric view, three orthographic elevations, and a section.
What goes in, and what comes out. The sheet states what must be true of the part: a Ø34 boss over a Ø26 counterbore and a Ø18 bore, four Ø6.4 holes on a 66 by 40 pattern, 90 by 58 by 28 overall. Every one of those is a question the kernel can answer about the solid below. Drawn from the model by our own sheet generator; recoloured for this page.

A drawing is not a description of a part, it is a specification of one: every callout on it is a claim that either holds on the solid or does not. So the test suite writes itself. The agent declares what it read on the sheet — a Ø34 boss, four Ø6.4 holes on a 66 by 40 pattern — and the kernel measures the solid against each claim in turn: the boss is there, at 34.0; the pattern has three holes where the sheet asks for four; the counterbore is not on the part at all.

Getting this right is harder than it looks, and the failure mode is specific: a check that can be satisfied by the thing it checks.

The DSL

The third piece is what the agent is allowed to write. The obvious answer is to drive a CAD system's own scripting API, and it is the one we moved away from. Those APIs are verbose, vendor-specific and built around a particular kernel — exactly the long low-level plumbing we found models least reliable at — and a part written against one is worthless against another.

The alternative is a compact language of engineering intent — pad, pocket, fillet, revolve, sweep, loft, joints, machining operations — that sits above the API doing the building rather than being it. It is short enough that a whole part fits in the model's attention instead of its plumbing, and explicit enough that what was written can be read back as intent rather than as a sequence of calls. And there is exactly one language to train a model on, not one dialect per vendor.

That language also has to solve the topological naming problem — persistent naming, in the literature — the deepest unsolved problem in the paradigm. Kernel indices are renumbered whenever geometry changes, so fillet edge 47 does not fail loudly when something moves upstream — it runs, it succeeds, and it puts the fillet on a different face. How a part names its own geometry, so that a reference still points at what it meant after the shape underneath it has moved, is the question the language has to answer.

What comes next

The LMM

Everything above works with a frontier model in the slot, and none of it requires training anything. But it builds something else as a side effect: an environment where an attempt can be graded automatically. That is the one ingredient RLVR needs[4], and the checks above already supply it.

Which matters, because the model and the harness are converging. OpenAI trained o3 and o4-mini to decide when and how to reach for a tool, inside the reasoning itself[7]; Anthropic trains Claude explicitly for agentic, tool-using loops[8]. The scaffold is no longer something you wrap a model in — agentic RL folds it into the weights[9], and across the labs the harness has become the product[10].

Traditionally, academic work on CAD generation has sat on the other side of that divide[11][12][13] — standalone networks mapping an input to a program, with no tools, no feedback, no environment in the loop. The question is what a model learns when it is trained inside the environment it will be deployed in: acting through the same ACI, writing the same DSL, rewarded by the same checks. That model is what we call an LMM, a large manufacturing model, and it is what we build next.

It already pays on the labs' own ground: Cursor's Composer, post-trained with RL on an open base, matches the frontier models on coding benchmarks at a tenth of the cost per task[5].

Which is the argument for building in this order. A clever prompt over a general model is precisely what the next general model absorbs. An environment that can say whether a physical part is correct is absorbed by nothing — it has to be built, and once it exists it is both what makes today's models useful here and what trains tomorrow's.

References

  1. Jimenez et al., SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (ICLR 2024) — « the best-performing model, Claude 2, is able to solve a mere 1.96% of the issues ».
  2. SWE-bench Verified leaderboard — frontier systems around 80% as of 2026.
  3. Yang et al., SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (NeurIPS 2024).
  4. DeepSeek-AI, DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning (Nature, 2025). The term RLVR is from Lambert et al., Tülu 3 (2024).
  5. The Decoder, Cursor's Composer 2.5 matches Opus 4.7 and GPT‑5.5 benchmarks at a fraction of the cost (2026) — post-trained on an open base, ≈ $0.50 against ≈ $7 per task.
  6. Endor Labs, GPT-5.5 sets a new code security record with Cursor, not Codex (Agent Security League, April 2026) — the same model, the same week, 61.5% functional correctness in one harness and 87.2% in the other.
  7. OpenAI, Introducing OpenAI o3 and o4-mini (2025) — trained to decide when and how to use tools.
  8. Anthropic, Claude Sonnet 4.5 system card (2025).
  9. Zhang et al., The Landscape of Agentic Reinforcement Learning for LLMs (2025).
  10. The New Stack, The harness is the product (2026).
  11. Wu, Xiao, Zheng, DeepCAD (ICCV 2021).
  12. Khan et al., Text2CAD (NeurIPS 2024).
  13. Rukhovich et al., CAD-Recode (ICCV 2025) and Kolodiazhnyi et al., cadrille (ICLR 2026).

Geometry as Code — the companion note on why an agent's only real handle on a part is a program.