Skip to content
AI Learning Playground
LinkedInX
English
English

Tool Selection Failures and Fixes

About 5 minutes

Shiori
Target audience: Those whose AI picks the wrong tool, or fails to use one when it should
Prerequisites: Basic understanding of What Is Tool Use

Most tool-use problems come not from the tool implementation but from selection. Failing to use a tool, or confusing it with a similar one, is something design can reduce.

Three Failure Patterns

FailureSymptomMain cause
Not used when it should beThe AI answers from guessworkThe purpose is not legible in the description
Similar tools confusedThe result differs from expectationsThe boundary between purposes is vague
Used when unnecessaryExtra executions pile upNo statement of when not to use it
This table scrolls horizontally. Keyboard users can focus the table and use the left and right arrow keys.

Cause 1: Too Many Tools

As tool count grows, similar purposes blend together and judgment gets harder. A person facing twenty similarly named functions would misfire too.

  • Merge tools with overlapping purposes
  • Remove tools that go unused
  • Narrow the set passed in based on the kind of task

There is no requirement to expose every tool at all times. Passing only what the current work needs improves both accuracy and cost.

Cause 2: Thin Descriptions

A tool description is the only evidence the AI has. Cover not just what it does but when to use it and when not to.

# Insufficient
Runs a search

# Sufficient
Full-text search over internal documentation.
Do not use for public information or general knowledge.
Search terms work best at three words or fewer.

Stating when not to use a tool does the most to reduce mix-ups.

Cause 3: Vague Boundaries

When overlapping tools sit side by side, either choice looks wrong.

# Vague boundary
- search_docs: Searches documents
- find_content: Finds content

# Explicit boundary
- search_docs: Full-text search over published documents
- find_files: Locates targets by filename and path, without searching contents

Keep responsibilities non-overlapping in both the name and the description.

Order of Investigation

When tool selection misbehaves, check in this order.

  1. Is the tool actually being passed to the AI?
  2. Does the description convey both its purpose and when to avoid it?
  3. Does another tool overlap in purpose?
  4. Is the total tool count too high?

Summary

  • Most tool-use failures happen in selection, not implementation
  • Too many tools blur similar purposes and reduce accuracy
  • Descriptions should state both when to use and when not to use a tool
  • Merge overlapping tools, or make the boundary explicit
Quiz