Skip to content

feat: add a telemetry hook to the Claude Code plugin - #60

Merged
knihit merged 12 commits into
mainfrom
feat/claude-plugin-telemetry
Oct 22, 2026
Merged

knihit merged 12 commits into
mainfrom
feat/claude-plugin-telemetry

Conversation

@knihit

@knihit knihit commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds usage telemetry to the Claude Code marketplace plugin. It reports the same two events the CLI already sends for Claude Code, in the same format.

Plugin hook

  • Added scripts/claude-plugin-telemetry-hook.js, shipped as hooks/konductor-telemetry.js. Plugin installs have no konductor binary, so this is a plain Node port of konductor __telemetry-hook. It has no npm dependencies and runs on macOS, Linux and Windows.
  • SessionStart (startup|clear) sends agent_invocation, and SubagentStart sends subagent_invocation, as in the CLI's TELEMETRY_HOOK_ENTRIES. A SessionStart with no agent_type sends nothing.
  • Only Konductor's own agents are reported, and only in the plugin form konductor:<agent>. targetName is the bare name. A bare agent_type comes from a CLI install, whose own hook reports it, so an agent is never counted twice. parentAgentName comes from CLAUDE_CODE_AGENT under the same rule, so a user's own agent is never named.
  • The body matches the CLI's:
    • outer Solution/Version/UUID/TimeStamp
    • every schema-required Data field
    • sessionId = sha256(UUID + raw session id)
    • harness is claude-plugin, so plugin events can be told apart from CLI agent events, which send null
    • errorCode and agentVersion are null
    • the outer Version is the plugin version
  • docs/telemetry-schema.json's harness description is updated to say so, and its agentVersion description now says plugin events carry the plugin version in the outer Version.
  • Machine ID: the hook shares ~/.konductor/telemetry.json with the CLI. It creates the file the way identity.rs does: mode 0600, a temp file plus link, and on a race the loser reads the winner's file. A short write fails before anything is published. It never overwrites a record, and it refuses a symlinked ~/.konductor.
  • Opt-outs (any one stops sending):
    • KONDUCTOR_TELEMETRY=off
    • telemetry.enabled: false in .konductor/config.yml in cwd, any parent directory or $HOME, as the CLI searches. A missing enabled leaves telemetry on, as in the CLI; an enabled set to anything but plain true turns it off. Comments are ignored. If the file mentions both telemetry and enabled but the hook finds no plain top-level telemetry key (an indented root, a ? telemetry key), it treats telemetry as off.
    • telemetry_consent: false in the machine record
    • the plugin's new telemetry_enabled option
  • The agent filter runs before any config.yml is read, so a non-Konductor session reads no config files. All opt-outs are checked before the machine record is created or any request is made.
  • File reads: config.yml, telemetry.json and plugin.json are opened non-blocking, must be regular files, and are read up to 64 KiB. A FIFO can't hang the hook. A non-regular or oversized file counts as unreadable: a config.yml is skipped, a telemetry.json fails closed, and a plugin.json sends nothing.
  • Endpoint: always the CLI's default URL. The plugin ignores KONDUCTOR_METRICS_ENDPOINT and telemetry.endpoint, so there's no endpoint to validate. POST JSON, 3 s send timeout, no retries.
  • The hook never writes to stdout or stderr and always exits 0. A backstop timer ends it if stdin never closes.

Manifest and assembly

  • scripts/claude-plugin.template.json: added the telemetry_enabled boolean option, default true. Claude Code passes it to hooks as CLAUDE_PLUGIN_OPTION_TELEMETRY_ENABLED.
  • Added scripts/render-claude-plugin-hooks-json.py. It writes hooks/hooks.json with the SubagentStart matcher ^konductor:(<agents>)$, built from the assembled agents/*.md so a new agent can't be missed. It fails if an agent's name differs from its file name.
  • scripts/assemble-claude-plugin-branch.sh copies the hook and docs/telemetry-schema.json into the plugin tree and renders hooks/hooks.json.

Docs

  • docs/user-guide/tasks/install-claude-plugin-marketplace.md: new Telemetry section. It says telemetry is on by default, lists what is sent and that the machine ID is shared with the CLI, and gives each opt-out and where config.yml is read. It also notes that the plugin ignores telemetry.endpoint.
  • README.md: one paragraph in Data Collection. The published plugin's README is a copy of it.
  • generated/claude-plugin/README.md: lists the new files.
  • CHANGELOG.md: [Unreleased] / Added entry.

Closes #58

Type of Change

  • Infrastructure / pipeline
  • Documentation

Testing

  • node --test tests/scripts/claude-plugin-telemetry-hook.test.js: 41/41. These cover:
    • Drift: schema field types and patterns read from docs/telemetry-schema.json. The endpoint, solution ID, nil sentinel, instance schema version and payload limit are read from the Rust sources.
    • Opt-outs: each one, including config.yml in a parent directory and $HOME, and eight YAML forms (quoted keys, JSON, BOM, !!bool, anchors and others).
    • Config parsing: a missing enabled and comments don't turn telemetry off. Forms with no plain top-level telemetry key, and content on a leading --- line, read as off.
    • File reads: a FIFO config.yml, telemetry.json or plugin.json exits 0 promptly. A config.yml over 64 KiB is skipped and one at the cap is read. An oversized telemetry.json fails closed. A non-Konductor session opens no config.yml.
    • Hang guards: a request that never responds is dropped after the 3 s send timeout, and a stdin that never closes exits 0 through the backstop. Each test fails with its guard removed.
    • Endpoint: with KONDUCTOR_METRICS_ENDPOINT and telemetry.endpoint both set, the request still goes to the default URL.
    • Machine ID: create, never overwrite, invalid record, symlinked dir, dir mode 0700, short write.
    • Output: silent exit 0 on bad input. Tests use a stub, so nothing reaches the network.
  • Full npm test: 17 failures in other test files, all from scripts/brand-config/lib/constants.js, which isn't in the repo. Those files fail on main too, and none of the failures are in this PR's tests.
  • python3 -m pytest tests/ -q: 153 passed. make claude-plugin-check passes, and it runs the hook tests first.
  • Claude Code session with the plugin installed (local directory marketplace, https.request stubbed, other network calls blocked):
    • konductor:konductor delegating sends one agent_invocation and one subagent_invocation. Both validate against the schema and carry harness: "claude-plugin".
    • Built-in agents and a bare k-developer send nothing.
    • Each opt-out sends nothing.
    • The hook's output is empty.
  • CLI install plus the plugin in one config, with https.request and curl stubbed:
    • A bare konductor session sends one event from the CLI (harness: null) and none from the plugin.
    • A konductor:konductor session sends from the plugin only.
    • Both use the same machine UUID.
    • telemetry.enabled: false in $HOME/.konductor/config.yml stops both.
  • cd cli && make test: no cli/ changes
  • New files carry the SPDX header

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own changes
  • I have commented my code where necessary
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added or updated tests that prove my change works

Notes for Reviewers

Marketplace plugin installs had no konductor binary and no hooks, so they
sent no usage telemetry. The plugin now ships hooks/konductor-telemetry.js,
a dependency-free Node port of `konductor __telemetry-hook` for Claude Code:

- SessionStart (startup|clear) sends agent_invocation and SubagentStart
  sends subagent_invocation, only for Konductor's own agents. Plugin agents
  arrive as `konductor:<agent>`; the bare name is reported.
- Same envelope, Data fields, session hash and endpoint as the CLI, and
  the same machine record in ~/.konductor/telemetry.json, created with the
  CLI's temp file plus link pattern and never overwritten.
- Opt-outs: KONDUCTOR_TELEMETRY=off, telemetry.enabled: false in
  .konductor/config.yml, telemetry_consent: false in the machine record,
  and a new telemetry_enabled plugin userConfig option.
- Prints nothing and always exits 0.

Assembly copies the hook and docs/telemetry-schema.json into the plugin
tree and renders hooks/hooks.json, with the SubagentStart matcher built
from the assembled agents/. make claude-plugin-check runs the new
node --test suite. Telemetry is disclosed in the marketplace install guide,
the plugin build README, and the root README.

Closes #58
…laude-plugin

Bare agent names come from a CLI install, whose own hook already reports
them, so matching them here counted those events twice. Plugin events now
send harness "claude-plugin", so they can be told apart from the CLI's,
which send null on agent events.
…d write

- Look for .konductor/config.yml in cwd, each parent directory, and $HOME.
  Any of them setting telemetry.enabled: false turns telemetry off.
- Parse the config more fully: strip a BOM, accept quoted keys, and read
  JSON and flow mappings. If the file mentions telemetry and enabled but
  does not plainly set enabled to true, treat telemetry as off. Tags,
  anchors, aliases and merge keys count as unclear. An unreadable file is
  skipped, as the CLI does.
- Pass a lookup to https.request that resolves every address, refuses the
  connection if any is loopback, private or link-local, and pins the
  checked address. DNS gets 500 ms, the send 3 s, as in the CLI.
- Reject localhost with a trailing dot and *.localhost.
- Loop on short writes of the machine record; a write that makes no
  progress fails closed before the record is published.
… Rust code

- A symlinked ~/.konductor sends nothing and leaves the target empty; a
  new ~/.konductor gets mode 0700.
- Read NIL_UUID_SENTINEL, INSTANCE_SCHEMA_VERSION, MAX_HOOK_PAYLOAD_BYTES
  and the host denylist ranges from the Rust sources and compare.
- Check each event field's JSON type and pattern against
  docs/telemetry-schema.json.
… read

The telemetry_enabled option now names the delegating agent and the
claude-plugin harness tag. The install guide says config.yml is read from
the project, its parents, and $HOME, and that only
KONDUCTOR_METRICS_ENDPOINT changes the plugin's endpoint.
Comment thread scripts/claude-plugin-telemetry-hook.js Outdated
Comment thread scripts/claude-plugin-telemetry-hook.js Outdated
Resolve the README conflict with the refreshed Data Collection section from
#55: add the plugin paragraph and an opt-out row in the new layout.
…ents

A config.yml with a telemetry block but no enabled key turned plugin
telemetry off, while the CLI treats it as on. Comments also affected the
reading, so 'enabled: true  # see **docs**' counted as off. Only an
enabled key set to something other than plain true now turns it off.
The plugin ignores telemetry.endpoint, so KONDUCTOR_METRICS_ENDPOINT was
its only way to change the endpoint, and nothing needs that. Dropping it
removes the host allowlist, address parsers, DNS check and escape hatch
that guarded it, along with their tests. Tests still stub https.request.
…onfigs

- Read config.yml, telemetry.json and plugin.json with a non-blocking open,
  regular files only, capped at 64 KiB. A FIFO no longer hangs the hook,
  and a non-regular or oversized file counts as unreadable.
- Check the agent filter before any config.yml read, so non-Konductor
  sessions read no config files. Every opt-out is still checked before
  the machine record is created.
- Treat config.yml as off when it mentions both telemetry and enabled but
  has no plain top-level telemetry key, and accept content on a leading
  `---` line. A telemetry block without enabled still means on.
- Add a 10 second timeout to both hooks.json handlers.
- Test the send timeout and the stdin backstop with a stub that never
  responds and a stdin that never closes, and test that the endpoint
  ignores KONDUCTOR_METRICS_ENDPOINT and telemetry.endpoint.
- Narrow the hook header to the wire format, rename a stale test section,
  and say in the schema that plugin events carry the plugin version.
Claude Code doesn't enforce timeout on async command hooks, so the field
had no effect. The hook's own send timeout and backstop bound its run time.
@knihit
knihit requested a review from ihmaws October 21, 2026 15:07
ihmaws
ihmaws previously approved these changes Oct 22, 2026
simonkrol
simonkrol previously approved these changes Oct 22, 2026
knihit added a commit that referenced this pull request Oct 22, 2026
… delegation (#57)

Fixes found while preparing the first Claude Code marketplace release. Each one broke a plugin install or the plugin release itself.

Release: the candidate commit shipped build output

    scripts/build-plugin-candidate-commit.sh used to commit from the job's own checkout. git rm -rq . also removed the checked-out .gitignore, so untracked build output (cli/konductor-rs/target/, build/, dist/, generated/) was committed. That is the ~1,383 extra files in Publish Claude Code plugin v1.1.0 #56.
    The script now builds the tree from --assemble-dir alone, through a scratch index (git add -A -f, write-tree, commit-tree). It never reads the checkout.
    Before the no-op check, it compares the tree with --assemble-dir: path, mode and blob ID (git hash-object --no-filters). An ignore rule or line-ending filter can't drop or change a file silently.
    update-ref passes an empty old value, so it never overwrites an existing branch.
    New scripts/check-claude-plugin-tree.sh fails on any top-level entry outside .claude-plugin, .mcp.json, LICENSE.txt, README.md, agents, docs, hooks, skills, including dangling symlinks. docs and hooks are for feat: add a telemetry hook to the Claude Code plugin #60. It runs in the release script and in make claude-plugin-check.
    make claude-plugin-check now fails when plugin assembly fails. Before, a failed assembly still ended "validated successfully".
    release.yml is unchanged.

Plugin agents: delegation was blocked

    Claude Code names plugin agents konductor:<agent>. Agent(...) grants and subagent_type don't resolve bare names, so Agent(k-developer, ...) matched nothing.
    New scripts/rewrite-claude-plugin-agent-grants.py runs during plugin assembly and qualifies each grant with the plugin name (12 names across konductor, k-architect, k-quality-assurance). It fails if a grant names an agent the plugin doesn't ship. Source specs keep bare names, so CLI installs are unchanged.
    It also fails, before writing anything, on an agent file with no frontmatter, since that file's grants can't be checked. Both rewrites end the frontmatter only at a column 0 ---, so an indented --- in a block scalar no longer cuts it short.
    k-architect's grant also carried a stale package prefix. It is now Agent(k-developer, k-quality-assurance) in the source.
    Both grant rewrites (rewrite-claude-plugin-agent-grants.py, and the existing rewrite-claude-plugin-mcp-tool-names.py) dropped the newline after the closing ---. The blank line after the fence was lost, and a body starting right after the fence would have merged into it. They now write the file back unchanged apart from the grants.
    delegation-protocol and claude-teams-behavior examples now show konductor:k-* for plugin installs and k-* for .claude/agents/ installs.

Dispatch: mux and cmux refused to dispatch in plugin installs

    tmux-dispatch.sh, zellij-dispatch.sh and cmux-dispatch.sh only found agents under ~/.claude/agents/. A plugin install has none there.
    The existing lookup is unchanged for plain names: local-<pkg>-<agent>, then a registry <pkg>-<agent>, then the bare file.
    If nothing matches and the skill is running inside the plugin, they launch the bare name. claude --agent k-developer resolves to konductor:k-developer by itself.
    A qualified --agent konductor:<agent> is accepted only when it names this plugin and an agent it ships. Anything else gets the not-installed error.
    cmux now rejects an agent-file prefix with shell characters before pasting it into the pane command. Before, a file named local-x$(cmd)-k-developer.md ran cmd.

Skills: bundled scripts weren't found, and a repo could plant one

    mux-dispatch, cmux-dispatch, persistent-memory and legacy-to-agentic-estimate looked for their scripts relative to the working directory or the git root.
    They now try, in order:
        ${CLAUDE_SKILL_DIR}, which Claude Code substitutes with the skill's own directory
        ${SKILLS_HOME:-$HOME/.konductor/skills}/<skill>
        <git root>/.konductor/skills/<skill>, only if .konductor/manifest exists
        <git root>/skills/<skill>, only if the repo's .claude-plugin/marketplace.json names konductor
    If none matches, the snippet stops with an error. A cloned repo can no longer supply dispatch.sh or the memory validator.
    cmux-dispatch.sh finds its sibling files from its own location, like dispatch.sh.
    The orchestrator agents defer to the skills for these paths.

Fixes #56's root causes. #56 was closed without merging.
Resolve conflicts with #57: keep both assembly steps, the Agent() grant
rewrite before the telemetry hook, and keep both [Unreleased] sections.
@knihit
knihit dismissed stale reviews from simonkrol and ihmaws via 57c7af4 October 22, 2026 18:57
@knihit
knihit merged commit 15cd19b into main Oct 22, 2026
5 checks passed
@knihit
knihit deleted the feat/claude-plugin-telemetry branch October 22, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telemetry for Claude Code marketplace plugin installs via plugin hooks

3 participants