The Carrying Cost of AI Agent Tools

I have a bad habit with coding agents. I install a skill or connect an MCP server for one task, finish the task, and leave the capability in my global setup. The next session inherits something I may never call again.

Software makes accumulation feel free. Installation takes seconds, while removal asks me to remember why I added the tool, whether another workflow depends on it, and how I would prove that disabling it caused no regression. The same habit fills applications with libraries that nobody reads and fills agent clients with capabilities that nobody measures.

The papers I collected over the last two weeks gave this habit a clearer shape. An agent should have access to a broad capability registry, while each task should receive a small working set. The registry preserves discoverability. The working set limits context, authority, and verification work to the current task.

Installation hides several runtime states

I used to describe a tool as installed or absent. That vocabulary collapses decisions that affect different parts of the system.

1. A discovered capability exists in a registry or marketplace.
2. An installed capability has files, packages, or configuration on the machine.
3. An enabled capability can start or register with the client.
4. An exposed capability places instructions, names, or schemas in front of a router or model.
5. A selected capability enters the plan for the current task.
6. An invoked capability executes code, uses authority, or returns data that may remain in context.

These states create different costs. An installed server can add dependency and update risk without adding prompt tokens. An exposed schema can consume attention without executing code. An invoked tool can mutate state even if the model drops its output from the next turn.

Some clients retrieve tool definitions on demand. Others expose a larger catalog before the model knows which capability it needs. I should measure the client path instead of claiming that each installation consumes context in the same way.

Selection belongs inside the context budget

Optimal Skill Selection for LLM Agents treats skill loading as an optimization problem under a hard token budget. The authors report 0.73 task success on a contamination-controlled BigCodeBench variant, compared with 0.20 to 0.52 for their evaluated baselines, while using 28 percent fewer tokens than the strongest released router. The paper appeared this week and needs replication, but its objective matches the engineering problem I see: the selected set changes both capability and cost.

How Many Tools Should an LLM Agent See? reaches a related conclusion from adaptive retrieval. Its selector showed Claude Sonnet 4.6 seven tools on average and reported 93.1 percent correct selection, compared with 87.1 percent for a fixed top-five list. On ToolBench, the fixed list still performed better on aggregate coverage, and it missed difficult queries whose correct tool ranked below fifth. The useful policy is adaptive depth with a budget, rather than one global number of tools.

General AgentBench reports average degradation of about 10 to 30 percent when ten agents move from specialized environments into a shared MCP tool pool. That setup changes routing, tool-schema volume, and task mix together, so it cannot assign the loss to one cause. It still warns against equating permanent exposure with generality.

Context pruning needs its own measurement. What Does Context Compression Cost an Agent? found no significant completion change at its prespecified 5x compression point, while retrieval calls increased in all six model-regime comparisons. For GPT-5.5, completion moved from 80 to 85 percent while retrieval calls rose from 21.0 to 63.9. The second environment did not show the same surge, which keeps the result narrow. Compression can move work into reacquisition, and completion rate can miss that transfer.

Tool descriptions participate in routing

A tool description looks like documentation, but the model reads it as a ranking input. Tool Preferences in Agentic LLMs are Unreliable changed natural-language descriptions and produced more than ten times the usage for selected tools in tests with GPT-4.1 and Qwen2.5-7B. The experiment does not estimate manipulation in deployed registries. It shows that publishers can influence selection without changing executable behavior.

Model Context Protocol Tool Descriptions Are Smelly! found another tradeoff. The authors report that richer description augmentation improved median task success by 5.85 percentage points, increased execution steps by 67.46 percent, and regressed 16.67 percent of cases. More explanation helped the model choose in some tasks and gave it more material to process in others.

I want tool metadata to carry an information budget. A useful description should state the contract, permission scope, and failure behavior with enough detail for selection. The client should keep marketing language and unrelated examples out of the active prompt.

A scanner cannot certify a capability graph

I also need to resist another shortcut: installing a security scanner and treating its verdict as trust. MaliciousSkillBench reports Macro-F1 scores from 0.882 to 0.932 on random splits for learned detectors, then 0.653 to 0.665 on source-disjoint evaluation. Its strongest TF-IDF SVM recovered 95.6 percent of malicious examples from held-out sources while flagging 62.4 percent of benign examples in that evaluation. The dataset does not tell us the prevalence of malicious skills in a marketplace. It tests whether detectors survive source shift, and the tested detectors struggled.

CompoSkill moves the unit of analysis from a skill to a path. Its constructed benchmark links outputs, capabilities, and side effects across skills that pass individual scans. The authors report risky chain formation rates up to 83.3 percent in the white-box setting and 80.6 percent in the black-box setting. Those rates belong to the paper's attack construction, OpenClaw and Nanobot workflows, and benchmark scenarios. They do not estimate the chance that a normal skill chain compromises a user.

The design lesson survives that boundary. Each node can pass a local check while the composed path crosses a policy boundary. A client needs to evaluate the route, shared state, and combined authority before execution.

Task-Conditioned Least-Privilege Learning offers one complementary layer. After post-training a Qwen3.5-4B model, the authors report 98.48 percent safe success across 2,896 evaluation episodes on 500 held-out tasks, compared with 64.36 percent for the base policy. Excess-authority events fell from 4.56 to 0.79 percent in their environment. The model learned restraint under task-specific authority envelopes and deterministic verifiers. The authors still keep permission gates and sandboxing in the architecture, which is the right boundary for a learned policy.

A valid tool call can produce the wrong state

Tool-call syntax gives the weakest form of confidence. A model can choose an allowed function, send valid JSON, receive a successful response, and leave the system in the wrong state.

Outcome Monitors checks observed effects against task contracts and names recovery tools when it detects a violation. The authors report completion rising from 10.9 to 28.1 percent on ToolMaze across four models and two provider families. Detection fell to 46 percent outside the failure vocabulary used to build the monitor, so the monitor cannot cover arbitrary failure modes. The result still supports effect-level verification over function-call validation.

Thinkingbox makes the reliability gap easier to see. The strongest evaluated agent reached 65.36 percent pass@1 across stateful business workflows and 25.25 percent pass^20. Many failed runs reached a normal termination state after valid state-changing actions. A polished response and valid tool calls did not establish the correct terminal state.

This result changes how I want to evaluate a tool registry. I need repeat-run success, policy fidelity, and terminal-state checks. Tool selection accuracy covers one link in that chain.

Libraries create the same ownership problem

Agent skills feel new, but software teams know the lifecycle problem from package managers. A developer imports a library to avoid rebuilding a wheel, then inherits defaults, transitive dependencies, release decisions, and failure behavior. The team may use the code for years without reading the path that handles its input.

Implicit, Yet Impactful studies Java projects that use classes from transitive dependencies without declaring those libraries. Across its Maven dataset, the authors report implicit dependencies in 34.12 percent of analyzed modules. They found version-drift breaking changes in 48 percent of identified cases and 36 CVEs whose vulnerable methods were used by root projects. The study is listed for ASE 2026, and its Maven-specific rates should stay inside that ecosystem and dataset.

BreakGuard uses generated tests around client call sites to detect dependency breaking changes. Its best configuration detected 27 of 89 changes, or 30.3 percent, and handled crash-type changes better than behavioral changes. Generated tests added evidence and left most changes undetected. Test count does not tell me which contracts remain unobserved.

The human side deserves attention too. How AI Impacts Skill Formation studied developers learning one unfamiliar asynchronous programming library. The authors report weaker conceptual understanding, code-reading, and debugging performance under AI assistance without an average efficiency gain, while more engaged interaction patterns preserved learning. The experiment does not describe expert maintenance across mature codebases. It supports a practical habit: if I adopt a library through generated code, I should still inspect and explain the implementation path I now own.

I do not want to turn this into an argument against libraries. A focused local implementation creates fresh correctness and maintenance work. A mature library can offer the smaller lifetime responsibility when its boundary matches the problem and the team can verify its behavior. I want to choose between those responsibilities after reading the relevant code and failure path.

My capability policy

The papers point toward a maintenance policy I can apply to my own agent setup.

1. Each capability should record its source, version, owner, and reason for installation.
2. Global configuration should contain broad discovery metadata and a narrow default working set.
3. Each project should declare the skills and MCP servers it enables, along with the permissions each task receives.
4. The client should report prompt footprint, selection frequency, invocation frequency, and retained output cost.
5. Every invoked capability should produce an effect record that a mechanical check or reviewer can inspect.
6. Rare capabilities should enter a disable trial before deletion, with representative tasks and a rollback path.
7. Scanner results should include source coverage, false-positive behavior, and composition limits.
8. Updates should run contract tests against the old and new versions, with behavioral gaps recorded instead of hidden.

This policy keeps uncommon tools available through discovery without placing their full instructions and authority into each session. It also makes removal a routine experiment instead of a guess based on memory.

My takeaway

I want my agent setup to behave like a maintained working set rather than a drawer of tools I installed once. The registry can stay broad while each task loads a smaller set chosen for evidence, token cost, and authority.

The client should show which capabilities entered the prompt, which permissions they received, and what evidence keeps them enabled. I should be able to trace a tool from discovery through invocation and inspect the state it changed. That record would let me remove stale capabilities, restore a rare one when a task needs it, and compare the cost of carrying a tool against the work it performs.

My agent can remain capable without carrying every past experiment into the next task.