Skip to content

feat: enrich_metadata + Dropbox recursive listing (sync spec v0.7.20) - #76

Merged
kdr merged 1 commit into
mainfrom
kdr-enrich-meta
Aug 17, 2026
Merged

feat: enrich_metadata + Dropbox recursive listing (sync spec v0.7.20)#76
kdr merged 1 commit into
mainfrom
kdr-enrich-meta

Conversation

@kdr

@kdr kdr commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Syncs the Python SDK to spec v0.7.20 (cloudglue/cloudglue-api-spec#111) and bumps the package to 0.7.23.

Two new API surfaces, both additive:

  • recursive — Dropbox-only subtree listing on GET /data-connectors/{id}/files
  • enrich_metadata — opt-in source-metadata backfill on metadata imports, plus a files_enriched progress counter

Generated (cloudglue/sdk/)

The regen touches 312 files, but nearly all of it is the The version of the OpenAPI document: 0.7.17 → 0.7.20 header comment. The substantive changes are eight files:

  • api/data_connectors_api.pyrecursive query param plumbed through all three call variants
  • models/metadata_import_filter_set.pyrecursive
  • models/metadata_import_run_progress.pyfiles_enriched
  • models/{metadata_import,metadata_import_detail,metadata_import_run,create_metadata_import_request,create_metadata_import_run_request}.pyenrich_metadata

Wrapper (cloudglue/client/)

Unlike the JS wrapper (where request bodies are inferred), the Python wrapper passes explicit kwargs, so every new param needs hand-plumbing:

  • data_connectors.list_files — new recursive kwarg, forwarded to the generated API and documented
  • metadata_imports.create — new enrich_metadata kwarg, forwarded into CreateMetadataImportRequest
  • metadata_imports.run — new enrich_metadata kwarg, forwarded into CreateMetadataImportRunRequest as a per-run override

Docstrings refreshed alongside: the filter-set docs described path as "non-recursive, direct children" (now covers recursive), and include_thumbnails still claimed "Grain and iconik today" with the old "materially slows large imports" caveat — v0.7.20 widens it to Grain, iconik, Google Drive, and Dropbox and decouples poster copying from indexing.

No changes needed in cloudglue/__init__.py — this spec bump adds no new models, only fields on models already re-exported there.

Test plan

import cloudglue clean; wrapper signatures confirmed to carry the new kwargs in order; MetadataImportFilterSet.from_dict({'recursive': 'true'}), MetadataImportRunProgress.from_dict({'files_enriched': 3}), and CreateMetadataImportRequest(..., enrich_metadata=True) all round-trip.

Live prod battery: 27/27 passed, mirroring the JS battery check-for-check. Script and full results markdown are saved outside the repo; the substance:

  • recursive (8 checks) — against a real nested Dropbox tree: non-recursive root returns 3 folders and 0 files, recursive="true" returns 17 entries / 11 files reaching path depth 5. recursive="false" reproduces the default exactly. Scoped to path="/test": 3 direct children vs 11 recursive, recursive being a strict superset. Ignored (not an error) on a google-drive connector. An out-of-enum value is rejected with 400.
  • enrich_metadata (13 checks) — full lifecycle on a throwaway metadata collection against Dropbox, capped at max_files=3, with a {"path": "/test/subfolder", "recursive": "true"} filter set (which also proves recursive round-trips inside MetadataImportFilterSet). The definition persists enrich_metadata=True across create/get/list; a run with no override inherits it; the run completed with files_enriched: 3 of 3 indexed. The payoff is verified, not just the flag: enriched files carry Dropbox media_info{'duration_ms': 16166, 'width': 4094, 'height': 2014} — which Dropbox's list endpoint has not returned since 2019. A per-run override to False on a refresh pass is honored and reports files_enriched: 0. Import and collection were deleted afterward.
  • Clip-window regression (6 checks)clip_start/clip_end on share.get are not new, but they were absent from the internal spec snapshot and are restored in the spec PR, so they were re-verified rather than assumed: the window lands in the Mux playback-token claims as asset_start_time=3.0 / asset_end_time=9.0, an unclipped read carries no claims, and both invalid forms (half a pair, end <= start) return 400.

Gong enrichment — additionally verified, 9/9. Confirms metadata_imports.create(enrich_metadata=True) and .run() drive the Gong path through this wrapper, with Call Spotlight content modelling correctly through pydantic (parties[0].to_dict() round-trips, brief and topics present). Refresh run reported files_enriched: 1 with all five Spotlight fields populated.

One correction during that run is worth recording as real API behavior: the first attempt used append mode and reported files_enriched: 0 while the Spotlight fields still looked populated. That was not a bug and not a pass — the underlying Cloudglue file for a Gong call outlives the collection it was imported into, so an earlier run's already-enriched file was still present and append mode correctly declined to re-spend Gong API budget on it (files_created: 0, files_updated: 0). The fields on display were stale residue rather than that run's work. Re-running in refresh mode gave a genuine signal (files_updated: 1, files_enriched: 1), and the battery now asserts files_updated >= 1 so stale enrichment cannot masquerade as a pass. Capped at max_files: 1.

The deeper before/after — including proof that enriched documents are re-embedded (same query scoring 0.2580 → 0.4844) — is covered in the JS PR, cloudglue/cloudglue-js#155, since that behavior is server-side and identical for both SDKs.

Merge order

The spec PR (cloudglue/cloudglue-api-spec#111) is already merged, and this branch's spec submodule points at the squashed main commit 23b7b7f — verified byte-identical to the spec these SDKs were generated and live-tested against. No re-pointing needed; this is mergeable as-is.

Regenerates cloudglue/sdk against spec v0.7.20 and plumbs the new params
through the wrapper layer:

- data_connectors.list_files gains `recursive`, Dropbox-only subtree listing
- metadata_imports.create and .run gain `enrich_metadata`; the
  `files_enriched` progress counter flows through the generated model
- refreshed docstrings for recursive filter sets, the per-run enrich
  override, and the widened include_thumbnails connector support
@cursor

cursor Bot commented Aug 17, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Additive OpenAPI sync and wrapper kwargs with no auth or breaking signature removals; runtime impact depends on callers opting into enrich_metadata or recursive Dropbox listing.

Overview
Bumps the package to 0.7.23 and regenerates cloudglue/sdk/ from OpenAPI 0.7.20. Most of the diff is version header churn; the meaningful API additions are recursive on data-connector file listing and enrich_metadata on metadata imports (with files_enriched on run progress).

The hand-written client in cloudglue/client/ wires those through explicitly: data_connectors.list_files accepts recursive and forwards it to the generated API; metadata_imports.create and metadata_imports.run accept enrich_metadata on the create/run request models. Docstrings were updated for Dropbox recursive in filter sets, wider include_thumbnails sources (Google Drive and Dropbox), and async poster copying vs indexing.

Reviewed by Cursor Bugbot for commit 7c6db93. Bugbot is set up for automated code reviews on this repo. Configure here.

@kdr
kdr requested a review from amyxst August 17, 2026 19:15
@kdr
kdr merged commit d3285c8 into main Aug 17, 2026
1 check passed
@kdr
kdr deleted the kdr-enrich-meta branch August 17, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants