Prepare for tinycloud 0.3.21: bulk connector metadata imports (skill floor → 0.3.21)

CLAUDE.md

@@ -319,6 +319,32 @@ client-side filter masked it). Because the skill teaches the new
 flag values and probe semantics, the floor was raised to 0.3.20 (same
 merge-after-CDN gate — the dist PR merges only after CDN `channels.stable` =
 0.3.20).
+0.3.21 picks up SDK 0.7.25 (spec v0.7.17) and adds **bulk connector metadata
+imports** (features 45→46: `library.collections.imports.v1`; verbs stay 17 —
+`imports` is a `library` subcommand family, not a verb): `library imports
+create <col> --name <n> --connector <id>` saves an immutable definition that
+lists a data connector server-side (google-drive, dropbox, zoom, gong,
+recall, grain, iconik — not S3/GCS) and imports each matching file's
+source_metadata into a METADATA collection as collection files — thousands
+to hundreds of thousands of records per run, no media processing, runs
+consume no credits. Filters are listing passes reusing the `connectors
+files` flags (or `--filters '<json-array>'` for several passes — Dropbox
+listing is non-recursive, one pass per folder; unsupported keys are a 400
+at create, never dropped; Zoom/Gong pin their 6-month lookback into the
+stored filters at create; iconik caps 10k rows per set). Modes: `append`
+(default) imports new files + retries failures; `refresh` re-imports
+everything matched and alone honors `--delete-missing` (sweeps ONLY files
+this import brought in; a `--max-files`-capped run never sweeps).
+`--rate-limit` (clamped per connector) and `--thumbnails` (Grain/iconik
+posters) tune runs; `--no-start` saves without running. `imports
+list`/`show` page run history; `create`/`run`/`show` report envelope status
+`pending` while a run executes (with an `imports show` next-hint, mirroring
+`query --background`) and `ready` once settled; one run may be active per
+collection (busy trigger = rewritten retryable 409 error); `cancel`
+(defaults to the latest run) and `delete` never remove imported files.
+Because the skill teaches the subcommand family, the floor was raised to
+0.3.21 (same merge-after-CDN gate — the dist PR merges only after CDN
+`channels.stable` = 0.3.21).
 The host-level `profile` verb and the leading global flags `--home`/`--profile`
 (also `$TINYCLOUD_HOME`; 0.3.3+) relocate state and are intentionally absent
 from `commands --json` — like the launcher's install/update, they're CLI/host
@@ -371,9 +397,9 @@ of printing JSON. Any script invoking the binary must redirect `</dev/null`
   metadata sync) vs live-CDN jobs (`Install + smoke` matrix, npx-against-CDN)
   which run only on push to main or manual dispatch — never on PRs, because a
   CDN gap would fail every PR.
-- The live CDN serves 0.3.20 (latest aliases + v-prefixed pinned tarballs
-  for 0.3.0 through 0.3.20, with `manifest.json` + `.sha256`
-  sidecars; `channels.stable` = 0.3.20); all smoke legs are required.
+- The live CDN serves 0.3.21 (latest aliases + v-prefixed pinned tarballs
+  for 0.3.0 through 0.3.21, with `manifest.json` + `.sha256`
+  sidecars; `channels.stable` = 0.3.21); all smoke legs are required.
 - `publish-npm.yml` (tag `v*`): asserts tag == package.json version → gates
   on `generate-manifest.mjs --check` against the live CDN → publishes via
   npm trusted publishing (OIDC, `id-token: write`, npm ≥ 11.5.1 — no token

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cloudglue/tinycloud",
-  "version": "0.3.20",
+  "version": "0.3.21",
   "description": "Agent CLI for deep video and image work, by Cloudglue. Downloads the tinycloud binary on first run.",
   "bin": {
     "tinycloud": "bin/tinycloud.js"

skills/tinycloud/SKILL.md

@@ -144,6 +144,9 @@ tinycloud library collections delete col_desc --json
 tinycloud library collections create meetings --type metadata --json
 tinycloud library collections add zoom://uuid/<uuid> --to col_meta --metadata '{"deal":"acme"}' --json
 tinycloud library connectors refresh cloudglue://files/<id> --json   # re-fetch stale source_metadata, re-index (free)
+# Bulk metadata imports (0.3.21+) — load a WHOLE connector corpus in one free run (don't add files one at a time):
+tinycloud library imports create col_meta --name "drive backfill" --connector <connector-id> --json   # → pending; first run starts now
+tinycloud library imports show col_meta <import-id> --json           # poll: pending while running, ready when settled → probe/ask
 
 # Publish an HTML artifact to Cloudglue Sites (manage with list / unpublish)
 tinycloud publish ./tinycloud-output/html/report.html --name report --visibility private --json
@@ -231,6 +234,13 @@ Authoring your own recipes: [reference/workflow-authoring.md](reference/workflow
 - `probe --filter` works only with a collection scope (`--in
   collection:col_…`). `source_metadata.*` filters are file-level facts, so
   pair them with `--scope file`.
+- `library imports` (0.3.21+): to load a WHOLE connector corpus into a
+  metadata collection, create a bulk import instead of `add`/`sync`ing files
+  one at a time — free, no media processing. `imports create`/`imports run`
+  return `pending` while the run executes; poll `imports show <col>
+  <import-id>` until the envelope is `ready` before querying. One run may be
+  active per collection at a time; definitions are immutable (delete +
+  recreate to change filters); cancel/delete never remove imported files.
 - `query` (0.3.17+) is for analytics, not search: when the task is to COUNT,
   GROUP, rank, or join across a collection ("how many…", "which … most",
   "total hours per host"), reach for `query`, not `probe`/`ask`. Run `query

skills/tinycloud/reference/verbs.md

@@ -336,7 +336,7 @@ finds the query face across one or more collections (`--min-score`,
 `--group-by file`). `total` reports the server-available count across all modes
 (never rewritten by client `--min-*`/`--limit` filters).
 
-### library — collections and connectors
+### library — collections, connectors, and bulk metadata imports
 
 ```bash
 tinycloud library collections list --json
@@ -353,6 +353,16 @@ tinycloud library connectors files <connector-id> [--limit 25] [--page-token <t>
 tinycloud library connectors inspect [<connector-id>] <uri-or-share-link> --json   # metadata peek, no file created (0.3.11+)
 tinycloud library connectors refresh <file-id|cloudglue-uri|connector-url> --json  # re-fetch stored source_metadata (0.3.15+)
 tinycloud library connectors sync [<connector-id>] <uri-share-link-or-public-url> --json
+# Bulk metadata imports (0.3.21+) — load a whole connector corpus into a METADATA collection, free:
+tinycloud library imports create <col_id> --name <name> --connector <connector-id> \
+  [--from <date> --to <date> --title-search <t> --folder-id <id> --path <p> --team <t> --meeting-type <t> \
+   | --filters '<json-array|file.json>'] \
+  [--mode append|refresh] [--delete-missing] [--rate-limit <n>] [--max-files <n>] [--thumbnails] [--no-start] --json
+tinycloud library imports list <col_id> [--limit <n>] [--offset <n>] --json
+tinycloud library imports show <col_id> <import-id> [--limit <n>] [--offset <n>] --json   # run history; PENDING while a run executes
+tinycloud library imports run <col_id> <import-id> [--mode append|refresh] [--delete-missing|--no-delete-missing] [--max-files <n>] [--thumbnails|--no-thumbnails] --json
+tinycloud library imports cancel <col_id> <import-id> [<run-id>] --json   # no run-id = the latest (only possibly-active) run
+tinycloud library imports delete <col_id> <import-id> --json              # imported files always stay
 ```
 
 `collections create|add|remove|delete` are the only writes in an otherwise
@@ -433,6 +443,42 @@ or `ask`, using `--filter` on
 connector library (titles, participants, dates, tags) before paying for full
 processing in a `media-descriptions` collection.
 
+**Bulk metadata imports** (0.3.21+, feature `library.collections.imports.v1`)
+are the bulk path INTO a metadata collection — never `add`/`sync` files one
+at a time when a whole corpus is wanted. `library imports create <col> --name
+<n> --connector <id>` saves a definition that lists the connector server-side
+(google-drive, dropbox, zoom, gong, recall, grain, iconik — not S3/GCS) and
+imports every matching file's `source_metadata` as collection files:
+thousands to hundreds of thousands of records per run, **free — runs consume
+no credits**. The first run starts immediately (`--no-start` saves the
+definition only), and `create`/`imports run` return **`pending`** while a run
+executes — poll `imports show <col> <import-id>` until the envelope goes
+`ready` (progress counters inline: listed, created, updated, skipped,
+indexed, failed, removed), then query with `probe`/`ask`.
+
+Filters are listing passes reusing the `connectors files` flags (`--from`/
+`--to` YYYY-MM-DD UTC, `--title-search`, `--folder-id` Drive, `--path`
+Dropbox — non-recursive, so cover a tree with one pass per folder via
+`--filters '[{"path":"/a"},{"path":"/b"}]'` — `--team`/`--meeting-type`
+Grain); a key the connector can't honor is a clear 400 at create, never
+silently dropped, and no filters means one unfiltered pass over everything.
+Zoom/Gong pin their 6-month default lookback into the stored filters at
+create (pass `--from` to control the window); iconik caps 10k results per
+filter set (split by date windows). Modes: `append` (default) imports new
+files and retries previously-failed ones; `refresh` re-imports everything
+matched, and only refresh runs honor `--delete-missing` (removes ONLY files
+this import brought in that the source stopped returning; a
+`--max-files`-capped run never sweeps). `--rate-limit` tunes upstream
+listing requests/sec (per-connector safe defaults, clamped);
+`--thumbnails` copies Grain/iconik poster images (off by default — slows
+large iconik imports). Definitions are **immutable** (delete + recreate to
+change one); **one run may be active per collection** — triggering while
+busy errors with a clear retryable message; `imports cancel` stops the
+active run (already-imported files stay) and `imports delete` removes the
+definition + run history but never the imported files. On pre-0.3.21
+binaries every `imports` subcommand fails with an unknown-command error —
+fall back to per-file `collections add` / `connectors sync`.
+
 `connectors sync` materializes its argument into a Cloudglue file without
 starting analysis (idempotent). The connector id is optional — with just a
 URI or link, sync routes through the matching connector type. Connector URIs

skills/tinycloud/scripts/preflight.sh

@@ -9,11 +9,11 @@ set -u
 
 # Mirror tinycloud-skill.json: min_version / supported_range upper bound
 # (CI diffs these against the manifest).
-MIN_VERSION="0.3.20"
+MIN_VERSION="0.3.21"
 MAX_VERSION_EXCLUSIVE="0.4.0"
 INSTALL_CMD='curl -fsSL https://app.cloudglue.dev/tinycloud.sh | bash'
 # Kept in sync with ../tinycloud-skill.json required_features (CI diffs them).
-REQUIRED_FEATURES="envelope.v1 watch.v1 watch.speech.v1 see.v1 extract.v1 extract.images.v1 caption.v1 search.v1 probe.v1 probe.filters.v1 probe.entities.v1 probe.scope.auto.v1 ask.v1 query.v1 query.export.v1 clip.v1 grab.v1 face.v1 jobs.v1 library.collections.v1 library.collections.create.v1 library.collections.mutate.v1 library.collections.entities.v1 library.collections.metadata.v1 library.collections.describe.v1 library.collections.pagination.v1 library.sync.url.v1 workflow.v1 publish.v1 publish.manage.v1 publish.video.v1 publish.link.preview.v1 publish.link.preview.player.v1 setup.v1"
+REQUIRED_FEATURES="envelope.v1 watch.v1 watch.speech.v1 see.v1 extract.v1 extract.images.v1 caption.v1 search.v1 probe.v1 probe.filters.v1 probe.entities.v1 probe.scope.auto.v1 ask.v1 query.v1 query.export.v1 clip.v1 grab.v1 face.v1 jobs.v1 library.collections.v1 library.collections.create.v1 library.collections.mutate.v1 library.collections.entities.v1 library.collections.metadata.v1 library.collections.describe.v1 library.collections.pagination.v1 library.collections.imports.v1 library.sync.url.v1 workflow.v1 publish.v1 publish.manage.v1 publish.video.v1 publish.link.preview.v1 publish.link.preview.player.v1 setup.v1"
 
 # 1) Binary present and responsive?
 if ! command -v tinycloud >/dev/null 2>&1; then

skills/tinycloud/tinycloud-skill.json

@@ -1,8 +1,8 @@
 {
-  "skill_version": "0.3.20",
+  "skill_version": "0.3.21",
   "tinycloud": {
-    "min_version": "0.3.20",
-    "supported_range": ">=0.3.20 <0.4.0",
+    "min_version": "0.3.21",
+    "supported_range": ">=0.3.21 <0.4.0",
     "required_features": [
       "envelope.v1",
       "watch.v1",
@@ -30,6 +30,7 @@
       "library.collections.metadata.v1",
       "library.collections.describe.v1",
       "library.collections.pagination.v1",
+      "library.collections.imports.v1",
       "library.sync.url.v1",
       "workflow.v1",
       "publish.v1",