feat: enrich_metadata + Dropbox recursive listing (sync spec v0.7.20) - #76
Conversation
Regenerates cloudglue/sdk against spec v0.7.20 and plumbs the new params through the wrapper layer: - data_connectors.list_files gains `recursive`, Dropbox-only subtree listing - metadata_imports.create and .run gain `enrich_metadata`; the `files_enriched` progress counter flows through the generated model - refreshed docstrings for recursive filter sets, the per-run enrich override, and the widened include_thumbnails connector support
PR SummaryLow Risk Overview The hand-written client in Reviewed by Cursor Bugbot for commit 7c6db93. Bugbot is set up for automated code reviews on this repo. Configure here. |
Summary
Syncs the Python SDK to spec v0.7.20 (cloudglue/cloudglue-api-spec#111) and bumps the package to 0.7.23.
Two new API surfaces, both additive:
recursive— Dropbox-only subtree listing onGET /data-connectors/{id}/filesenrich_metadata— opt-in source-metadata backfill on metadata imports, plus afiles_enrichedprogress counterGenerated (
cloudglue/sdk/)The regen touches 312 files, but nearly all of it is the
The version of the OpenAPI document: 0.7.17 → 0.7.20header comment. The substantive changes are eight files:api/data_connectors_api.py—recursivequery param plumbed through all three call variantsmodels/metadata_import_filter_set.py—recursivemodels/metadata_import_run_progress.py—files_enrichedmodels/{metadata_import,metadata_import_detail,metadata_import_run,create_metadata_import_request,create_metadata_import_run_request}.py—enrich_metadataWrapper (
cloudglue/client/)Unlike the JS wrapper (where request bodies are inferred), the Python wrapper passes explicit kwargs, so every new param needs hand-plumbing:
data_connectors.list_files— newrecursivekwarg, forwarded to the generated API and documentedmetadata_imports.create— newenrich_metadatakwarg, forwarded intoCreateMetadataImportRequestmetadata_imports.run— newenrich_metadatakwarg, forwarded intoCreateMetadataImportRunRequestas a per-run overrideDocstrings refreshed alongside: the filter-set docs described
pathas "non-recursive, direct children" (now coversrecursive), andinclude_thumbnailsstill claimed "Grain and iconik today" with the old "materially slows large imports" caveat — v0.7.20 widens it to Grain, iconik, Google Drive, and Dropbox and decouples poster copying from indexing.No changes needed in
cloudglue/__init__.py— this spec bump adds no new models, only fields on models already re-exported there.Test plan
import cloudglueclean; wrapper signatures confirmed to carry the new kwargs in order;MetadataImportFilterSet.from_dict({'recursive': 'true'}),MetadataImportRunProgress.from_dict({'files_enriched': 3}), andCreateMetadataImportRequest(..., enrich_metadata=True)all round-trip.Live prod battery: 27/27 passed, mirroring the JS battery check-for-check. Script and full results markdown are saved outside the repo; the substance:
recursive(8 checks) — against a real nested Dropbox tree: non-recursive root returns 3 folders and 0 files,recursive="true"returns 17 entries / 11 files reaching path depth 5.recursive="false"reproduces the default exactly. Scoped topath="/test": 3 direct children vs 11 recursive, recursive being a strict superset. Ignored (not an error) on a google-drive connector. An out-of-enum value is rejected with 400.enrich_metadata(13 checks) — full lifecycle on a throwaway metadata collection against Dropbox, capped atmax_files=3, with a{"path": "/test/subfolder", "recursive": "true"}filter set (which also provesrecursiveround-trips insideMetadataImportFilterSet). The definition persistsenrich_metadata=Trueacross create/get/list; a run with no override inherits it; the run completed withfiles_enriched: 3of 3 indexed. The payoff is verified, not just the flag: enriched files carry Dropboxmedia_info—{'duration_ms': 16166, 'width': 4094, 'height': 2014}— which Dropbox's list endpoint has not returned since 2019. A per-run override toFalseon a refresh pass is honored and reportsfiles_enriched: 0. Import and collection were deleted afterward.clip_start/clip_endonshare.getare not new, but they were absent from the internal spec snapshot and are restored in the spec PR, so they were re-verified rather than assumed: the window lands in the Mux playback-token claims asasset_start_time=3.0/asset_end_time=9.0, an unclipped read carries no claims, and both invalid forms (half a pair,end <= start) return 400.Gong enrichment — additionally verified, 9/9. Confirms
metadata_imports.create(enrich_metadata=True)and.run()drive the Gong path through this wrapper, with Call Spotlight content modelling correctly through pydantic (parties[0].to_dict()round-trips, brief and topics present). Refresh run reportedfiles_enriched: 1with all five Spotlight fields populated.One correction during that run is worth recording as real API behavior: the first attempt used append mode and reported
files_enriched: 0while the Spotlight fields still looked populated. That was not a bug and not a pass — the underlying Cloudglue file for a Gong call outlives the collection it was imported into, so an earlier run's already-enriched file was still present and append mode correctly declined to re-spend Gong API budget on it (files_created: 0, files_updated: 0). The fields on display were stale residue rather than that run's work. Re-running in refresh mode gave a genuine signal (files_updated: 1,files_enriched: 1), and the battery now assertsfiles_updated >= 1so stale enrichment cannot masquerade as a pass. Capped atmax_files: 1.The deeper before/after — including proof that enriched documents are re-embedded (same query scoring 0.2580 → 0.4844) — is covered in the JS PR, cloudglue/cloudglue-js#155, since that behavior is server-side and identical for both SDKs.
Merge order
The spec PR (cloudglue/cloudglue-api-spec#111) is already merged, and this branch's
specsubmodule points at the squashedmaincommit23b7b7f— verified byte-identical to the spec these SDKs were generated and live-tested against. No re-pointing needed; this is mergeable as-is.