You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Collection Profile 0.3.0: say which parent keys DETAIL_COVERAGE required_keys must cover #631
Section 5.6 of the Collection Profile (docs/spec/collection-profile.md, around lines 1353-1370) defines a DETAIL_COVERAGE report's required_keys as the connector's own complete key accounting for one detail stream and parent boundary. It does not tie that set to the parent records the runtime accepted in the same run.
So a connector can emit parent records and then report required_keys: [], or omit a key, and the parent cursor commits with that key's detail never accounted for. The runtime cannot reject this without stalling a legitimate pattern. A connector emits required_keys: [] beside accepted parents when their detail is already held, which is what the Amazon orders and order_items streams do.
Data Connect accepts this as a known limit in 0.2.0 (data-connect PR #636 pins the behaviour in a test).
Proposal for 0.3.0: state which parent keys must appear in required_keys. One option is every parent key accepted for that boundary in the run, unless the report marks the key's detail as already held. A runtime can then check coverage without relying on the connector's own set.
Activity