For anyone who has wrestled with rigid JSON schemas to make large language models call tools, this will spark curiosity: a new study suggests that switching to plain English for tool selection can materially boost accuracy and consistency—while cutting token cost. At AI Tech Inspire, this caught our eye because it challenges the default assumption that stricter structure always yields better control in agent pipelines.
Key findings at a glance
- Replacing JSON/XML schemas with natural language for tool selection improved tool-call accuracy by ~18 percentage points (from 69.1% to 87.5%) across 6,400 trials and 10 models.
- Variance dropped by about 70% (0.0411 → 0.0121), making outputs more stable across perturbations.
- DeepSeek-V3 rose from 78.4% to 94.7% accuracy; variance fell from 0.023 to 0.0016.
- Models without native tool-call support (e.g., DeepSeek-R1) still benefited (reported 94.1% accuracy).
- Token overhead decreased—reporting includes roughly 31% token overhead reduction and a separate claim of 47.4% fewer input tokens on average.
- Evaluation covered two domains (Mental Health and Customer Service), 16 inputs per domain, with perturbations and five repetitions per input.
- The proposed framework—Natural Language Tools (NLT)—separates tool selection from response generation via a simple “Selector → Parser → Output” pipeline.
- Limitations include added latency (minimum two model calls) and a single-turn, parameterless evaluation scope.
What NLT changes in the tool-call mental model
Most agent stacks instruct the model to emit a perfectly structured tool_calls array inside JSON, forcing it to nail syntax while also deciding whether to call a tool and how to respond. NLT breaks that coupling. It moves tool selection into a short natural-language step where the model explicitly considers each tool and marks it YES or NO. A parser then executes the chosen tools, and a separate output stage composes the final answer.
Thinking: (brief reasoning)
Tool A - YES/NO
Tool B - YES/NO
Tool C - YES/NO
Assessment finished.
That’s it. No nested braces to babysit. No schema drift. Just language—arguably what LLMs are trained on the most.
Why plain language can beat structured JSON
- Reduced format burden: Structured outputs push probability mass toward staying within syntax instead of making the best tool choice. As complexity grows, small formatting mistakes cascade into failed calls.
- Less task interference: Separating selection from generation reduces the cognitive overhead of juggling multiple jobs in one pass.
- Alignment with training: LLMs are heavily optimized to produce fluent text, not necessarily schemas. That alignment appears to translate to better tool-selection behavior.
- Explicit catalog sweep: Listing each tool by name combats positional bias and “tool amnesia,” nudging the model to consider the full set before deciding.
- Shorter prompts: Cutting JSON boilerplate shrinks context. Even modest context reductions can matter, especially under long tool catalogs or multi-agent settings.
For developers who default to JSON because it “feels” safer, these findings pose a useful challenge: is schema enforcement helping—or quietly harming—your tool selection?
A quick tour of the evaluation setup
The reported experiments span 6,400 trials across two domains (Mental Health and Customer Service) with both original and perturbed prompts to reduce prompt-engineering bias. Ten models were included, with noteworthy gains for open-weight systems. That last part is especially interesting: open-weight models often have less specialized tool-call training compared to commercial APIs, which may explain why natural-language alignment pays off more noticeably.
One stand-out mention: DeepSeek-V3 reportedly jumped from 78.4% to 94.7%, and its output variance plummeted, turning a previously jittery performer into one of the most consistent in the cohort.
How to try NLT today
You don’t need special SDKs or fine-tuning. The pattern is simple and works with any model that can follow instructions—whether a hosted GPT-style API or an open model running locally via Hugging Face.
- Stage 1: Selector
Prompt the model with your tool catalog and request a YES/NO decision for each tool. Keep it short, explicit, and list all tools by name. - Stage 2: Parser
Read the lines, collect the YES tools, and run them. This parser can be trivial: regex over names and flags, with guardrails for duplicates or missing lines. - Stage 3: Output
Feed tool results back into the model with a concise instruction to generate the final user-facing answer.
If your orchestration lives in Python (hello, PyTorch shops) or on GPU-heavy infra with CUDA, integrating the selector-parser-output loop is straightforward—just treat selector and generator as separate calls. For local dev, popping open a REPL and running the selector step with Ctrl+Enter can be a quick way to sanity check decisions before wiring the parser.
What might surprise practitioners
- Models without tool-call support still benefit: The study reports high accuracy even when the model can’t emit native tool-call JSON. The parser acts as the bridge.
- Fewer tokens can be a big deal: The authors cite both a ~31% token overhead reduction and a separate figure of 47.4% fewer input tokens. Less context not only saves cost—it may also improve reliability.
- Open-weight models shine: Gains were reportedly stronger for open models, possibly because natural-language selection aligns more closely with their training emphasis.
Where NLT fits—and where it might not
NLT looks particularly compelling for:
- Agentic systems that maintain a large catalog of tools and need robust selection (think retrieval, calculators, internal APIs).
- Safety-critical routing, where false negatives (failing to call the right tool) are costly.
- Edge deployments of open models where native tool calling isn’t available.
However, there are trade-offs:
- Latency: At least two model calls (selector + output). If your current system answers in one pass when no tool is needed, this adds overhead.
- Scope: The evaluation covered single-turn, parameterless selection. Multi-turn flows, argument construction, and complex tool chaining weren’t the focus.
- Parser trust: You’ll need to harden the parser and add checks for misformatted outputs or ambiguous YES/NO lines.
Implementation tips
- Keep the selector short: Overlong tool descriptions reintroduce context bloat. Favor crisp “when to use” lines.
- Be explicit about coverage: Require a line for every tool. That’s key to the “full-catalog consideration” effect.
- Return a terminal marker: End with a literal
Assessment finished.to simplify parsing. - Consider confidence tags: If needed, add a light confidence note per tool (e.g.,
YES (high)/YES (low))—but avoid drifting back into verbose schemas. - Log selector outputs: Auditing why a tool was or wasn’t selected can surface mismatched descriptions or unintended biases.
Open questions worth exploring
- Does NLT still outperform JSON when tools require structured arguments or complex parameter trees?
- How does NLT behave in multi-turn dialogues where tool relevance evolves across turns?
- Could lightweight constrained decoding (e.g., regex-guided) capture NLT’s benefits without extra latency?
- What happens when the tool catalog scales to hundreds of entries—does the explicit list remain tractable?
Why this matters for builders
If you maintain an agent system that routes through APIs, databases, or retrieval, you’ve likely accepted the JSON schema tax as “just how it is.” This work suggests another path: lean into the thing models are best at—language—and let a simple parser do the deterministic part. Swapping the tool-selection step could be one of those low-effort, high-leverage changes that improve both accuracy and predictability.
“Plain English might be the most reliable ‘API’ your model already knows.”
At AI Tech Inspire, the takeaway is practical: before investing weeks into stricter schemas or heavier finetuning for structured tool calls, try an NLT-style selector. If your dashboards show jumps in successful tool routing and fewer weird edge cases, you’ve found a lever worth pulling.
Recommended Resources
As an Amazon Associate, I earn from qualifying purchases.
