tiltmetertiltmeter
Launch suitesuiteSpecHash 9981576

mcp-tool-selection

SPEC §11 launch suite 2 — real tool schemas from three PUBLIC MCP servers actually configured on this machine (context7, github, playwright/agent-browser), vendored with attribution (repo+commit+blobSha, byte-verified by CI re-fetch). Includes genuinely confusable pairs: resolve-library-id vs query-docs (both take a free-text 'query'), search_issues vs search_code vs list_issues (search-vs-structured-list, issues-vs-code), browser_snapshot vs browser_take_screenshot (the upstream README explicitly warns about this exact confusion). 20 positive + 8 negative. Nothing from any private employer- or client-specific world appears here — every artifact is public and byte-verified against a commit.

presentation
tool-select@1
sampling
k=3 · t=1 · 512 tok
metrics
overall, triggerRate, falsePositiveRate
retired
0

Composition

A suite that only ever tests the happy path cannot tell you when it starts firing on everything. Every item here is an tool-selection probe, registered 2026-08-09.

28
active items
8
negative
28.6% negativequota met
  • pos-resolve-library-id-1
  • pos-resolve-library-id-2
  • pos-query-docs-1
  • pos-query-docs-2
  • pos-search_issues-1
  • pos-search_issues-2
  • pos-search_code-1
  • pos-search_code-2
  • pos-list_issues-1
  • pos-list_issues-2
  • pos-browser_navigate-1
  • pos-browser_navigate-2
  • pos-browser_snapshot-1
  • pos-browser_snapshot-2
  • pos-browser_take_screenshot-1
  • pos-browser_take_screenshot-2
  • pos-browser_click-1
  • pos-browser_click-2
  • pos-browser_type-1
  • pos-browser_type-2
  • neg-1
  • neg-2
  • neg-3
  • neg-4
  • neg-5
  • neg-6
  • neg-7
  • neg-8

amber = negative · quiet = positive

Artifacts

10 probed

Provenance is stated per artifact: a public artifact names the repo and commit it was captured from; a private one is vendored-only and not independently re-verifiable.

  • tool.resolve-library-id
    resolve-library-id
    tool-schema
    upstash/context7@8d52608e4e27
  • tool.query-docs
    query-docs
    tool-schema
    upstash/context7@8d52608e4e27
  • tool.search_issues
    search_issues
    tool-schema
    github/github-mcp-server@e7f7bb8b31bd
  • tool.search_code
    search_code
    tool-schema
    github/github-mcp-server@e7f7bb8b31bd
  • tool.list_issues
    list_issues
    tool-schema
    github/github-mcp-server@e7f7bb8b31bd
  • tool.browser_navigate
    browser_navigate
    tool-schema
    microsoft/playwright-mcp@4ed35bf87133
  • tool.browser_snapshot
    browser_snapshot
    tool-schema
    microsoft/playwright-mcp@4ed35bf87133
  • tool.browser_take_screenshot
    browser_take_screenshot
    tool-schema
    microsoft/playwright-mcp@4ed35bf87133
  • tool.browser_click
    browser_click
    tool-schema
    microsoft/playwright-mcp@4ed35bf87133
  • tool.browser_type
    browser_type
    tool-schema
    microsoft/playwright-mcp@4ed35bf87133