Skip to content

Validate ingested data — trust-minimize the backfill source #773

Description

@tamirms

TL;DR

Backfill currently requires the data lake to be a trusted source. Validate that the data we ingest is actually correct, so a faulty or malicious source can't silently feed us bad data — removing the trusted-source requirement.

Scope

  • Determine how ingested data can be verified as correct without trusting the source, and put that verification in place.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions