What Belongs in a Harness — Curating Tools Without Drowning in Them

There is a moment that arrives in every engineering org adopting agents. The harness works. The tests gate, the diffs are clean, the loop runs. And then a new tool ships — a slick MCP server, a community framework, a clever evaluator someone posted on a Friday — and the question lands on a technical leader's desk: should we add this?

The reflexive answer is yes. New capability looks like progress; saying no feels like falling behind. So tools accrete, one defensible decision at a time, until the harness is a junk drawer of half-used integrations nobody is willing to remove because nobody can prove it's safe to. This is how a clean system rots — not through one bad decision, but through a hundred reasonable additions with no countervailing discipline.

This piece is about that discipline. It is the companion to two ideas we have written about before: what a harness is (the structured environment a codebase lives in), and what's in our stack (the layers and the bets behind them). This is the third question, and the one most teams never formalize: how do you decide what earns a place in the harness, and keep deciding it as the ecosystem floods you with new options? The answer is not a tool. It is a curation methodology — a bar for inclusion, a repeatable intake process, and the nerve to prune.

The Counterintuitive Rule: Abundance in the Pipeline, Scarcity in the Toolbox

Start with the distinction that governs everything else, because getting it backwards is the single most common mistake we see.

A good harness wants an abundance of measured checks and a scarcity of tools. These are not the same thing, and conflating them is what produces sprawl.

A check is a determination of success — a test, a lint rule, a type constraint, an eval threshold, a coverage floor, a security scan. Checks live in the CI/CD pipeline. Every check you add narrows the gap between "the agent thinks it's done" and "it's actually correct." More checks mean more determined, measurable gates between a change and production, and more places where a defect is caught automatically before it reaches a human. You want a lot of these. The richer the gate, the more you can trust the throughput. A pipeline with code-coverage floors, vulnerability scans, performance thresholds, and passing unit tests is a multi-layered defense that moves quality verification out of late-stage manual review and into the automated pipeline. Abundance here is strength.

A tool is a capability the agent reaches for to do work — an MCP server, an integration, a command, a framework. Every tool you add does the opposite of a check: it widens the space the agent has to reason over. It is one more schema in the context window, one more thing the model has to choose correctly among, one more dependency to secure, monitor, and upgrade. Tools are a tax, and the tax compounds.

So the rule is asymmetric on purpose. Be greedy with checks. Be miserly with tools. Checks measure success; tools create surface area. One you want in abundance; the other you want curated to the minimum that does the job. A harness that gets this backwards — thin on gates, fat on tools — is precisely the harness that produces fast, confident, wrong output.

Why Tool Sprawl Is a Reliability Problem, Not Just a Tidiness One

It would be easy to treat tool sprawl as an aesthetic complaint — clutter, untidiness, a thing that offends engineers who like clean systems. It is not aesthetic. It is a measurable degradation in how well the agent works, and the research on this got sharp in the last year.

The mechanism is the context window. Every tool you connect ships its full schema — name, description, parameters — into the model's context on every turn, whether the agent uses it or not. This adds up faster than intuition suggests. A single GitHub MCP server can run roughly 35 tools consuming around 26,000 tokens; stack five common servers and you can burn around 55,000 tokens on tool definitions before the conversation even begins. Anthropic's own engineering has reported tool definitions consuming on the order of 134,000 tokens before optimization. That is context spent before the agent has read a line of your actual problem.

Spent context would be merely wasteful if models degraded gracefully as it filled. They don't. The phenomenon now widely called context rot is that as the input grows, LLMs don't fade evenly — they become unreliable, hallucinating parameters, calling the wrong tools, and missing instructions. Transformer attention is not uniform; signal from the actual task gets diluted by noise from tool definitions the agent will never use on this run.

And the effect on the one thing you most need from an agent — picking the right action — is stark. The RAG-MCP study measured tool-selection accuracy collapsing to 13.62% under a bloated tool set, versus 43.13% when the agent was shown only the relevant tools — more than a threefold difference driven entirely by how many tools were in front of the model. Anthropic's work on on-demand tool loading points the same direction from the other side: surfacing only the needed tools moved Opus 4's accuracy on a large-library benchmark from 49% to 74%. Same model, same task — the difference was tool count in context.

There is a behavioral failure mode on top of the accuracy one. Faced with too many options, agents don't just choose worse — they sometimes invent plausible-sounding tools that don't exist, or decline to act at all. Every tool you add to the harness raises the odds of all three: wrong selection, hallucinated capability, and paralysis.

This is the whole argument compressed: tool sprawl is not untidiness. It is a direct, measured tax on the agent's reliability and its context budget. A leader who would never ship code with a falling test-pass rate should treat a falling tool-selection rate with the same seriousness — because they are the same kind of regression.

The Inclusion Bar: What Earns a Place

If tools are a tax, then admission has to be earned, not granted. Over our own harness work we have converged on a small set of questions a candidate tool must answer before it gets in. None is sufficient alone; a tool worth adding clears all of them.

1. Does it add or strengthen a measured check? This is the highest-value reason to add anything. A tool that introduces a new gate — a security scanner, an eval, a validation step — is buying you abundance in the half of the harness where abundance is good. Favor these heavily. A tool that merely adds a capability the agent could already approximate has a much higher bar to clear.
2. Does it reduce variance or cost? A good addition makes outcomes more consistent or makes the same outcome cheaper — fewer retries, fewer tokens, less human cleanup. If you can't articulate the variance or cost it removes, you are adding surface area for a feeling.
3. Does it compose? The tool has to speak the same language as the rest of the harness — the repository, the gates, the version-control surface, the existing protocol layer. A tool that only works inside its own walled UI cannot be scripted, scheduled, or chained, and so it cannot participate in the loop. Composability is a hard gate, not a nice-to-have.
4. What is its context and token cost? This is the question the old build-versus-buy calculus never had to ask, and it is now central. Every tool's schema lives in the context budget. A tool that adds 8,000 tokens of definitions the agent rarely needs is competing for attention with your actual problem. Account for the token cost explicitly, the way you'd account for a dependency's binary size or a query's row count.
5. What is its failure and maintenance surface? Every tool is a thing that can break, a dependency to upgrade, a permission to scope, a credential to rotate, and a concept a new engineer must learn. Multiply that by the tool's actual usage. A capability used once a quarter rarely justifies a permanent seat in the context window and the maintenance ledger.

Notice the shape of the bar: it is biased toward checks and against bare capability. That is deliberate. The default answer for a new tool is no, and the burden of proof is on inclusion — the inverse of how most teams operate, where the default is yes and the burden is on whoever wants to remove something later.

The Intake Process: Trial, Measure, Adopt or Reject

A bar is only as good as the process that applies it. The community ships compelling new tools faster than any team can evaluate by gut feel, so the evaluation itself has to be a repeatable loop — the same disciplined intake every candidate runs, regardless of how exciting it looks in the demo.

Trial in isolation. A candidate tool enters on a branch, in a sandbox, never straight into the shared harness. The branch is the trial environment — wired in, scoped to minimum permissions, exercised against real tasks, and trivially revertible because nothing downstream depends on it yet. New is not a credential. The demo is marketing; the trial is evidence.

Measure against a baseline. This is the step teams skip, and skipping it is why sprawl wins. Before adopting, run the harness with and without the tool on a representative task set and compare on the dimensions the inclusion bar names: did a gate get stronger, did variance or cost drop, did context cost rise, did tool-selection accuracy hold? You already have the instrument for this — the same evals and tracing that gate your normal work measure a candidate tool just as well. A tool that can't show a measured improvement against the baseline has not earned a seat, however good it looks.

Adopt or reject, and write down why. The decision is binary and recorded. Adopt: the tool joins the harness with its scope, its owner, and the measured reason it's there written down — so a future engineer (or agent) inherits the rationale, not just the dependency. Reject: it comes off the branch and the reason is logged too, so the next person who finds the same shiny tool doesn't re-run the whole evaluation from scratch. A rejection is institutional knowledge, not a dead end.

The discipline this enforces is subtle but decisive: no tool enters the harness on enthusiasm. It enters on a measured delta against a baseline, or it doesn't enter. That single rule is what separates a curated harness from a junk drawer.

Pruning: The Discipline Nobody Schedules

Intake controls what comes in. It does nothing about what's already there, slowly going stale. So the other half of curation — the half teams almost universally neglect — is deliberate removal.

Pruning is hard for a human reason: removing a tool feels like loss and admitting a past decision didn't pan out, while leaving it feels free. It is not free. The dead tool still ships its schema into every context window, still needs its credential rotated, still has to be reasoned about during the next security review, still has to be upgraded when it breaks the build. The cost is just invisible until you total it.

So we make pruning a scheduled, non-optional pass rather than a thing that happens when someone finally snaps. On a regular cadence, every tool in the harness re-faces the inclusion bar as if it were a new candidate:

• Is it still earning its place? Usage data answers this. A tool the agent reaches for monthly is not paying for the context it occupies daily. Tracing tells you actual usage — use it.
• Has a better-composing or cheaper option replaced its job? The protocol-layer standardization of the last two years means capabilities consolidate. Two tools often collapse into one; the older one should go, not linger.
• Did the check it backed get absorbed elsewhere? Sometimes a gate that a tool once provided is now covered by the pipeline natively. The tool is then pure overhead.
• Is its failure or maintenance surface now disproportionate to its value? A dependency that breaks every upgrade and is used twice a quarter is a negative-value seat.

The ecosystem reinforces why this matters. The industry response to tool overload — Anthropic's on-demand tool search, retrieval-based tool selection, gateway-level filtering — exists precisely because unbounded tool sets degrade agents. Those techniques help you scale the toolbox when you genuinely must. But the first-order discipline is cheaper and more durable than any of them: don't accumulate what you'll have to engineer around later. Pruning is how you keep the toolbox small enough that you rarely need the clever workaround at all.

Facet's Point of View

We did not arrive at this by reading about it. We run our own delivery — and the content engine that produced this article — inside a harness we curate by exactly this discipline, and we got here the way most teams will: by feeling the tax before we named it.

Our position is three sentences long. Abundance of measured checks is the goal; abundance of tools is a failure mode wearing the costume of progress. Every tool earns its seat against a bar — does it add a gate, reduce variance or cost, compose, and justify its context and maintenance cost — or it doesn't get one. And what is true at intake is true forever: a tool that stops earning its place gets pruned, because the context window is a budget and the maintenance ledger is real.

This is the same instinct that has run through fifteen-plus years of our systems-integration work, long before agents made it acute. We have watched point-to-point integration sprawl turn nimble systems into ones only their original author could safely change. We have seen "more tools" sold as sophistication when it was really the absence of anyone owning the question is this still earning its place? The agentic era didn't invent that failure mode. It just made it faster, and put a measurable number — your tool-selection accuracy, your context budget — on the cost of getting it wrong.

For a CTO benchmarking against faster competitors, or an engineering leader trying to convert scattered AI experimentation into a systematic practice, the takeaway is precise and a little contrarian: your harness should be hungry for checks and stingy with tools. The discipline that gets you there is not a purchase. It is a bar for inclusion, an intake process that admits nothing on enthusiasm, and the recurring nerve to remove what has stopped paying its way.

If your harness has quietly filled with tools nobody is willing to remove, that is a fixable condition — and a familiar one. Talk to us about a harness curation review. We will map what's actually earning its place against what's just taxing your context budget, and help you build the intake-and-prune discipline inside your own walls — so the harness stays a harness instead of becoming a junk drawer with good intentions.

Sources

RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation (arXiv 2505.03275) — tool-selection accuracy 43.13% vs 13.62%, >50% prompt-token reduction: https://arxiv.org/abs/2505.03275

MarkTechPost — Tool Search for MCP, Anthropic evals (Opus 4 49%→74%; tool definitions ~134K tokens before optimization): https://www.marktechpost.com/2026/05/29/hermes-agent-ships-tool-search-for-mcp-anthropic-evals-show-49-to-74-accuracy-gain-on-opus-4/

AgentPMT — token cost of MCP tool definitions (GitHub ~35 tools / ~26K tokens; five-server ~55K tokens): https://www.agentpmt.com/articles/thousands-of-mcp-tools-zero-context-left-the-bloat-tax-breaking-ai-agents

Writer (engineering) — context rot, hallucinated/declined tool calls under overload: https://writer.com/engineering/rag-mcp/

DevOps Training Institute — 10 CI/CD Quality Gates for Production-Level Reliability: https://www.devopstraininginstitute.com/blog/10-cicd-quality-gates-for-production-level-reliability

testRigor — Software Quality Gates: What They Are & Why They Matter: https://testrigor.com/blog/software-quality-gates/

OneUptime — How to Implement Deployment Gates (2026): https://oneuptime.com/blog/post/2026-01-30-deployment-gates/view