Skip to content

Prune/rework AWS skills in favor of the official AWS Agent Toolkit MCP server #39

Open

Description

@knihit

Konductor ships a set of AWS-specific skills (aws-service-validator, find-aws-skills, dynamodb-design, dynamodb-validation, iam-policy-design, policy-validation, infra-validation, cost-estimation, architecture-diagram-generation, threat-modeling) that predate the official AWS Agent Toolkit and its AWS MCP server (aws-mcp.us-east-1.api.aws/mcp). Now that the toolkit exists and k-architect/k-developer can already reach it, several of these skills likely duplicate — or are now outclassed by — what the official server and its 114 bundled skills provide.

A pass through the current set, cross-checked against the live AWS MCP server, found:

  • dynamodb-design — the toolkit's amazon-dynamodb skill is deeper (axioms, cost calculator, live validation/benchmark scripts) and is itself served live via retrieve_skill. Likely candidate to replace outright.
  • dynamodb-validation — same toolkit skill covers review mode; our checker-style scoring contract isn't duplicated anywhere, so this may be worth slimming rather than removing.
  • infra-validation — currently LLM-only (no cfn-lint/cfn-guard/cdk-nag). The toolkit's aws-cloudformation and aws-cdk skills prescribe exactly those tools. Worth reworking to run the deterministic tools first.
  • find-aws-skills — thin wrapper over search_documentation/retrieve_skill that the toolkit's own rules file already documents. Likely retireable, with its "verbose search phrase" guidance folded into agent prompts.
  • iam-policy-design / policy-validation — different in kind from the toolkit's aws-iam (a short "common pitfalls" skill); ours covers design-level security architecture and compliance checklists the toolkit doesn't. Keep, but should absorb pitfalls the toolkit's aws-iam flags that ours currently misses (iam:PassRole escalation, confused-deputy protection, aws:SourceArn/aws:SourceAccount).
  • cost-estimation — calls get_pricing/analyze_cdk_project, which belong to a separate Pricing MCP server that no konductor agent currently declares. Needs its data-source dependency fixed regardless of toolkit overlap.
  • aws-service-validator, architecture-diagram-generation, threat-modeling — no toolkit equivalent found; keep as-is (aside from the parameter-shape fix in Fix stale AWS MCP tool references in agents and skills #38).

None of the toolkit's 114 skills cover draw.io diagram generation, STRIDE threat modeling, or document fact-checking, so those three stay regardless of outcome here.

Proposed follow-up work

  1. Decide, per skill above, whether to replace, rework, slim down, or retire.
  2. Where keeping a skill, close the specific coverage gaps the toolkit's equivalent flags as common agent mistakes (e.g. IAM pitfalls above).
  3. Fix cost-estimation's missing Pricing MCP server dependency.
  4. Consider an opt-in konductor install --with aws flag that configures the AWS MCP server entry (mcp-proxy-for-aws-cli) for agents that need it, since the public CLI currently declares aws-mcp as a dependency on k-architect/k-developer but never writes a server entry for it.

This is tracking/discussion for now — happy to split into separate issues per skill once there's agreement on direction. See #38 for a related, narrower fix (stale AWS MCP tool-name references in the affected skills/agents).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions