AI not using the MCP as much as it should

What problem does this solve?

I am using github co-pilot, with claude_sonnet 4.6. I am trying to do some benchmarking. and not seeing a big different between with it and without it. I asked the AI why and here is its answer.

I ask: so when I asked a question sometimes you use grep and other command instead of the MCP why?
it answers:
Honest answer — a few reasons:
Deferred tools require loading first. The MCP tools (mcp_codebase-memo_*) are in the availableDeferredTools list, meaning I must call tool_search to load them before I can use them. If I skip that step, they're unavailable and I fall back to grep_search, file_search, etc.
I don't always follow the instructions perfectly. The rules say to use MCP first, but I may shortcut to familiar tools out of habit, especially for seemingly simple queries.
The MCP may not be indexed. If the project hasn't been indexed yet, list_projects returns nothing and I have no choice but to use file-based tools.
What you can do: If you notice me using grep/file search instead of MCP, just remind me — e.g. "use codebase-memo" — and I'll load the tools properly via tool_search and follow the correct workflow.

this is with the standard vscode install files. is there something missing as to how to get the AI to use the tool as a first choice?

Proposed solution

a way to update the skill files to make it more direct for the AI to use the tool

Alternatives considered

No response

Confirmations

I searched existing issues and this is not a duplicate.

This is the story with harnesses that are not supported at a low level. Do you even mention the MCP in the AGENTS? Start there. If that fails, write SessionStart and if it is still failing, PreToolUse hooks.

making the skill files more direct helps the habit-shortcut case, but it doesn't touch the part the model already told you about: the memory tools are deferred, so at the moment it decides how to answer, grep is in its active toolset and the MCP isn't yet. no amount of wording wins a comparison against a tool that has to be pulled in by a separate tool_search step first. the lever is making those tools non-deferred (or auto-loading them plus the index check on session start), not stronger instructions.

I set codebase-memory-mcp to load mcp__codebase-memory-mcp__* at the start of the session, but this still didn’t increase its hit rate significantly. So I added the following instructions:

For ANY code exploration in this repo — finding functions/classes/routes, tracing call chains, understanding architecture, or locating where something lives — you MUST use the `mcp__codebase-memory-mcp__*` tools FIRST. Do NOT reach for Grep, Glob, Read, or Bash search for code comprehension until these tools return nothing useful. Grep/Glob/Read are for non-code files (configs, docs, logs) and for reading a file before editing it.

Core workflow:

- Resolve the project once per session via `mcp__codebase-memory-mcp__list_projects`
  and reuse that name verbatim for the rest of the session.
- To find symbols: `mcp__codebase-memory-mcp__search_graph`
  (name_pattern / label / qn_pattern).
- To trace call chains or data flow: `mcp__codebase-memory-mcp__trace_path`.
- To get exact source for a symbol: `mcp__codebase-memory-mcp__get_code_snippet`
  (qualified_name). Confirm any material finding against source with this BEFORE
  acting on it.
- For project structure: `mcp__codebase-memory-mcp__get_architecture`.
- Graph-augmented text search: `mcp__codebase-memory-mcp__search_code`.
- If a project is not indexed yet, run `mcp__codebase-memory-mcp__index_repository` FIRST
- BEFORE modifying code, check the blast radius of your pending changes with
  `mcp__codebase-memory-mcp__detect_changes` — it maps your uncommitted git diff
  to the affected symbols and their downstream impact, with risk classification.
  Review what a change touches before editing, especially for shared functions,
  and re-check after editing to confirm the impact matches expectations.

Two known limitations — when you hit these, supplement, don't abandon the graph:

- Dynamic dispatch (registry- or reflection-based calls) under-reports callers.
  Supplement with `mcp__codebase-memory-mcp__search_code` or
  `mcp__codebase-memory-mcp__ingest_traces`.
- Untyped code with common method names may yield mis-resolved edges. Verify at
  the call site with `mcp__codebase-memory-mcp__get_code_snippet`.

to ~/.claude/CLAUD.md. Although the number of calls increased noticeably, it still wasn’t as high as expected.

Since post-training, it should only be familiar with basic commands like grep/read, so this isn’t the MCP’s fault.

I’m not sure what I might be missing or if you have any better solutions—please share.

Thanks. I’ve linked this under #396 as an adoption / agent-guidance UX issue. The durable fix is likely a combination of better install-time reminders, clearer docs, and client-specific context hooks where available, rather than relying on users to manually remind the agent every session.

I did a little bit of digging with Claude’s help, and it might be that some harnesses, like the Claude Code VS Code extension, does not expose grep or glob directly, and instead, they use grep and glob indirectly through Bash?

If that's the case, the PreToolUse hook does not get triggered and the MCP may not be used. But this probably needs to be verified further.

Ran a small A/B on this with a Claude Code SessionStart reminder, same discovery prompts ("where is X handled", "what calls Y") each time. A generic "if your client defers MCP tools, load them in one batch first" sentence went 0/2 — the model still reached straight for rg. A line naming the exact deferred tool names in one ToolSearch call, plus "use these before grep/Read", went 3/3 graph-first with zero rg — and a Codex session with the installed AGENTS.md reminder behaved the same. Tiny sample, but the same prompt flipped on wording alone.

The generic version can't name the tools because deferred names are client-prefixed, and that seems to be exactly why it doesn't stick. The installer already owns per-client surfaces though (Claude reminder script, .codex/AGENTS.md) and knows each client's exact tool names, so it could emit the specific one-call loader line per client. Would also cover the exact-name retry from #330. Happy to PR it if that direction sounds right.

From my personal experience, I previously used CodeGraph and codebase-memory-mcp at the same time, and I noticed that the agent clearly preferred CodeGraph much more often.

CodeGraph has an explanation for this design choice:

“Measured agent behavior showed that one well-aimed tool steers agents to a direct answer better than a menu of narrower ones — fewer mis-picks — and agents reach for it both when answering questions and while editing code.” -- https://github.com/colbymchenry/codegraph/blob/main/site/src/content/docs/reference/mcp-server.md

I think this may also apply here.

codebase-memory-mcp has many useful capabilities, but exposing many specialized entry points also means the agent first has to decide which codebase-memory-mcp tool it should use. In practice, that extra decision may make it more likely to fall back to grep/read or another tool it already knows.

Following the same idea as CodeGraph, perhaps codebase-memory-mcp could provide one clear default entry point, something like codebase_explore, and internally route the request to search, tracing, architecture, cross-repo analysis, etc.

The existing specialized tools could still remain available for advanced or explicit use, but agents would have a much simpler default path.

Thanks for the original report, everyone’s follow-up experiments and suggestions, and your patience while this has remained open.

I checked current main at cb7d9376. Two distinctions help interpret the reports:

VS Code Copilot and standalone Copilot CLI have separate installation paths. The installer also writes shared Copilot skill, profile and lifecycle-hook files; this alone does not establish which files a particular VS Code session loaded. (Installer source)
The current Claude Code search hook registers Grep, Glob and Bash. Selected shell searches can receive graph context while the original search still runs, so an rg call alone does not show that graph integration was unused. This addresses the later Claude-specific hypothesis; it does not establish what happened in the original Copilot session. (Registration, graph context)

The inspected guidance and profile templates do not implement the proposed single-call ToolSearch bootstrap. These source observations do not establish the cause of the reported tool choices, validate an alternative entry point, or demonstrate a fix in a published release.

This remains open as maintainer-owned adoption and integration follow-up. Another reporter benchmark is not needed before responding to the existing proposals.

feat(mcp): load the structural tools upfront and say what the graph a…

feat(mcp): state the structural questions in the scout and analysis p…

feat(hooks): name the graph follow-up tools in every search augmentation

feat(cli): give Codex shell searches graph context through a PreToolU…

Thank you for the client-specific experiments and the proposed simpler entry point. These remain maintainer-owned integration and product decisions, as described in the October update. I have no new measured result to report and am not asking for another benchmark. Sorry for the delay—this is waiting on maintainer follow-up, and I’ll follow up soon.

fix(cli): render the Copilot manifest through the memory core

feat(cli): give Copilot searches graph context through a PostToolUse …

docs(readme): name the Codex PreToolUse and Copilot PostToolUse searc…

test(smoke): pin the Claude PostToolUse matcher that covers the graph…

feat(cli): note when a Claude Code session keeps searching without th…

Navigation Menu

Repository navigation

AI not using the MCP as much as it should #509

What problem does this solve?

Proposed solution

Alternatives considered

Confirmations

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

AI not using the MCP as much as it should #509

Description

What problem does this solve?

Proposed solution

Alternatives considered

Confirmations

Activity

esafak commented on Jun 20, 2026

m13v commented on Jun 26, 2026

bwnjnOEI commented on Jun 27, 2026

DeusData commented on Jun 29, 2026

superical commented on Jul 1, 2026

dergachoff commented on Aug 1, 2026

flyinghail commented on Sep 15, 2026

DeusData commented on Oct 2, 2026

DeusData commented on Oct 25, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions