Overview

Armature is a multi-tenant agent platform. Each workspace writes its own tools, runs them against real data, and keeps only the ones that pass. When the same kind of task comes back, the workspace runs a saved tool instead of asking a model to write one again.

The problem

Agents that write code usually throw it away after one use, so every repeat task pays full price again. And the model's own tests make a weak gate: in my experiments, tools passed their self-tests while returning garbage. I logged that failure as selftest_but_wrong.

So the question became: can a workspace build up a library of tools it can trust, without taking the model's word for any of them?

How it works

Every request runs through the same loop:

1. 1

   Route

   Decide whether a kept tool already covers the task or something new has to be built. A superset guard stops a narrow tool from being reused for a broader ask.

2. 2

   Synthesize

   The frontier model writes the tool. Routing, rendering, critique and compaction run on a cheaper tier, and embeddings and reranking run locally.

3. 3

   Verify by execution

   The harness runs the tool in a Firecracker microVM and checks exit status, results, failure sentinels and the output envelope against independent oracles. The model's self-test is only a pre-filter. Failures get up to 3 repair attempts driven by the execution error.

4. 4

   Keep and reuse

   Tools that pass are stored and versioned. They're verified again on reuse, and a stale tool is rebuilt as v+1.

What it measured

On the same task shape, the cold run took 115.1s and 8,866 tokens. The warm path finished in 9.1s on 964 tokens: 12x faster, 89% fewer tokens, and no frontier-model calls.

The real test was whether verified tools hold up on work with a known right answer. Each capstone is graded by an oracle the agent never saw:

1. 01

   Settlement month

   2,568 orders rolled into 22 payouts, with 6 faults injected. All 6 were caught, with zero false positives.

   Graded against

   The injected-fault ledger.

2. 02

   IBM Telco churn

   Churn scoring on the genuine 7,043-row dataset, trained on a live T4 GPU: ROC-AUC 0.8345, and 25 of 25 anti-fabrication row checks passed.

   Graded against

   Held-out rows.

3. 03

   USGS seismic

   Earthquake analysis on live USGS data.

   Graded against

   USGS's own count API.

Keeping tenants apart

• Generated code runs in a per-workspace Firecracker microVM with default-deny egress.
• Secrets are brokered per execution and never written to disk or logs.
• Every model call is metered against an up-front USD budget per workspace, enforced before compute.
• Hostile uploads (ELF binaries, zip bombs, malicious pickles) are identified by content, handled as data, and never executed.
• Kept tools export as signed bundles, and the importing tenant re-executes them before trusting them.

Selected figures