3.1.2

What's Changed

• fix(cli): allow BigQuery to determine location if unset by @Ford544 in #2396
• refactor(core): rename property graph entities/relationships to nodes/edges by @rafal-hawrylak in #2387
• Bump DF_VERSION from 3.1.1 to 3.1.2 by @rafal-hawrylak in #2397

Full Changelog: 3.1.1...3.1.2

3.1.1

What's Changed

• refactor(cli): migrate BigQuery integration tests from KMS-encrypted test key to ADC by @mbiernacik in #2362
• fix(cli): filter out non-existent tables in BigQueryDbAdapter.search by @rafal-hawrylak in #2388
• test: Make bigquery search integration test ignore leftover datasets by @apilaskowski in #2390
• fix(tests): Ensure concurrent test runs do not operate on the same BQ datasets by @Ford544 in #2385
• feat(core): add support for 4-part Lakehouse Catalog targets by @SergikL in #2384
• fix(cli): Use defaultLocation for BigQuery jobs and datasets in dataform run by @apilaskowski in #2389
• Canary test (with GCP bucket) by @Ford544 in #2354
• fix(cli): remove unused sql and errorMessage OpenLineage facets by @rafal-hawrylak in #2391
• test(cli): add location resolution e2e tests by @Ford544 in #2394
• Bump DF_VERSION from 3.1.0 to 3.1.1 by @SergikL in #2395

New Contributors

• @SergikL made their first contribution in #2384

Full Changelog: 3.1.0...3.1.1

3.1.0

What's Changed

• Bump fast-uri from 3.1.4 to 3.1.6 by @dependabot[bot] in #2270
• Update KMS test credentials by @rafal-hawrylak in #2346
• Pin fast-uri to 3.1.7 by @Ford544 in #2349
• Publishing Dataform security patches by @dataform-team in #2353
• Bump @grpc/grpc-js from 1.14.4 to 1.14.5 by @dependabot[bot] in #2355
• Bump vm2 from 3.11.6 to 3.11.7 by @dependabot[bot] in #2358
• Bump vm2 from 3.11.7 to 3.12.2 by @dependabot[bot] in #2363
• docs(cli): add comprehensive CLI reference README (#2329) by @rafal-hawrylak in #2364
• Bazel toolchain rewrite by @Ford544 in #2352
• Add a binary run test by @Ford544 in #2356
• Update TypeScript, ts-eslint and rollup by @Ford544 in #2365
• refactor(cli): replace deprecated vm2 with native node:vm runner by @mbiernacik in #2310
• feat(cli): add Windows support and automated CI test coverage by @mbiernacik in #2350
• Update pnpm to 9.7.1 by @Ford544 in #2366
• Bump baseline-browser-mapping from 2.9.19 to 2.11.25 by @apilaskowski in #2370
• Bump browserslist from 4.28.1 to 4.29.0 by @apilaskowski in #2371
• feat(cli): emit lineage for assertions and property graphs by @rafal-hawrylak in #2368
• fix(cli): stop rejecting older @dataform/core versions after a CLI version bump by @mbiernacik in #2376
• Add proto fields for defaultJobLabels and action-level jobLabels by @zaptot in #2351
• Upgrade chokidar to v4 to drop vulnerable braces dependency by @apilaskowski in #2377
• chore(cli): address review comments from PR #2368 by @rafal-hawrylak in #2373
• Bump DF_VERSION from 3.0.72 to 3.1.0 by @rafal-hawrylak in #2375
• Bump serialize-javascript override to ^7.1.2 by @apilaskowski in #2379
• docs(tooling): remove outdated line from contributing.md by @Ford544 in #2381
• Bump brace-expansion override to 5.0.12 by @apilaskowski in #2378
• fix(tooling): fix publish script by @Ford544 in #2380
• test(core): cover lineage settings in workflow_settings.yaml, including an empty lineage block by @rafal-hawrylak in #2382

Full Changelog: 3.0.71...3.1.0

3.0.71

What's Changed

• fix(security): upgrade linkify-it to 5.0.2 by @mbiernacik in #2324
• Update minimatch by @Ford544 in #2325
• Remove rimraf by @Ford544 in #2326
• chore(deps): upgrade braces to 3.0.3 by @mbiernacik in #2327
• chore(deps): upgrade serialize-javascript to 7.1.1 by @mbiernacik in #2331
• Update parse-duration by @Ford544 in #2332
• Minimatch update by @Ford544 in #2330
• Pin uuid version to 1.14.2 by @Ford544 in #2333
• Update markdown-it by @Ford544 in #2335
• chore(deps): remove unused ts-loader dependency by @mbiernacik in #2336
• Update debug by @Ford544 in #2337
• Split devDependencies off dependencies by @Ford544 in #2338
• hoist DECLARE statements to start of onSchemaChange procedure (#2307) by @apilaskowski in #2319
• Remove unused git workflow by @Ford544 in #2341
• Format entire repo with prettier by @rafal-hawrylak in #2342
• Sqlx comment parsing backslashes by @Ford544 in #2339
• chore(core): remove unused extractJsBlocks to resolve CodeQL ReDoS alert by @mbiernacik in #2340
• Bump DF_VERSION from 3.0.70 to 3.0.71 by @apilaskowski in #2334

Full Changelog: 3.0.70...3.0.71

3.0.70

What's Changed

• refactor(cli): decompose monolithic cli/index.ts into modular command modules by @mbiernacik in #2286
• Eliminate hardcoded project names for integration tests by @Ford544 in #2285
• Mark tests that require valid GCP project credentials as integration tests by @Ford544 in #2289
• refactor(cli): clean up help command handling and encapsulate yargs configuration by @mbiernacik in #2290
• refactor(cli): scope quiet compilation option to compile command by @mbiernacik in #2291
• Refactor cli tests by @Ford544 in #2287
• build(bazel): update Go modules versions and speed up local run script by @mbiernacik in #2293
• Split tests/integration/bigquery.spec.ts by @Ford544 in #2296
• Split the large test in tests/api/projects.spec.ts by @Ford544 in #2295
• Split tests/api/api.spec.ts by @Ford544 in #2294
• chore(lint): migrate from tslint to eslint by @mbiernacik in #2297
• Split core/main_test.ts by @Ford544 in #2301
• Split core/actions/table_test and core/actions/incremental_table_test by @Ford544 in #2302
• Split core/actions/assertion_test.ts by @Ford544 in #2305
• refactor(cli): strongly type yargs commands by @mbiernacik in #2306
• Refactor core/main_property_graphs_test.ts by @Ford544 in #2304
• Mitigate core/index_compile_test timeouts by @Ford544 in #2308
• Remove redundant request dependency by @Ford544 in #2311
• Bump js-yaml from 4.3.0 to 4.3.2 by @dependabot[bot] in #2299
• chore(deps): remove typedoc dependency to resolve marked vulnerability by @mbiernacik in #2313
• Upgrade prettier to 3.9.4 by @Ford544 in #2315
• fix: cli compile time increased by ~4x by @ashish10alex in #2300
• Upgrade jsdoc to ^4.0.5 by @Ford544 in #2316
• Update form-data to 2.5.6 by @Ford544 in #2317
• Enforce prettier formatting standards on PRs by @Ford544 in #2312
• Preserve Governance Controls in Incremental Tables. by @mohamed-saad in #2314
• Bump DF_VERSION from 3.0.69 to 3.0.70 by @mohamed-saad in #2318

New Contributors

• @mbiernacik made their first contribution in #2286
• @Ford544 made their first contribution in #2285

Full Changelog: 3.0.69...3.0.70

3.0.69

🕸️ BigQuery Graph Integration (Property Graphs)

With this release, Dataform CLI delivers full support and native integration for BigQuery Graph (Property Graphs). You can now declaratively define, orchestrate, and manage BigQuery Property Graphs directly in your Dataform projects via YAML.
Key Integration Capabilities:

• Declarative YAML Definitions: Define property graph schemas using definitions/graph.yaml to specify node tables (entities), edge tables (relationships), source/destination endpoints, labels, and properties.
• Automated Node & Edge DAG Dependencies: Reference upstream Dataform models in entities/relationships using ref data sources (dataSourceRef), automatically resolving dataset targets and ensuring correct execution ordering after underlying tables.
• Tagging & Selective Execution: Full tags support in graph.yaml to filter and run property graphs during selective executions (e.g., dataform run --tags graph).
• Assertion Propagation Control: Explicitly control whether dependent assertions run before property graph creation (dependOnDependencyAssertions: false / includeDependentAssertions: false).
• Automated BigQuery DDL Lifecycle: Compiles declarative YAML definitions into BigQuery CREATE OR REPLACE PROPERTY GRAPH DDL statements and executes them in BigQuery.

What's Changed

• Fix View.verifyConfig mutating caller-shared bigquery configs by @rafal-hawrylak in #2273
• Support tags on PropertyGraph actions by @rafal-hawrylak in #2275
• Bump fast-xml-parser from 5.5.6 to 5.7.3 by @dependabot[bot] in #2163
• Preserve explicit includeDependentAssertions=false on PropertyGraph refs by @rafal-hawrylak in #2281
• Fix Session.declare mutating caller-shared configs (#2280) by @rafal-hawrylak in #2283
• Bump DF_VERSION from 3.0.68 to 3.0.69 by @rafal-hawrylak in #2284

Full Changelog: 3.0.68...3.0.69

3.0.68

What's Changed

• Support ref() and DAG dependencies in PropertyGraph by @rafal-hawrylak in #2258
• Fix plumb filename by @rafal-hawrylak in #2271
• Add CLI runtime coverage and E2E integration test for PropertyGraph by @rafal-hawrylak in #2268
• Bump DF_VERSION from 3.0.67 to 3.0.68 by @rafal-hawrylak in #2276

Full Changelog: 3.0.67...3.0.68

3.0.67

🌟 Knowledge Catalog Integration (Data Lineage)

Starting with this release, Dataform CLI fully delivers native end-to-end integration with Google Cloud Knowledge Catalog (Dataplex Data Lineage). Executions now automatically emit OpenLineage-compliant operational metadata and table-level lineage graphs directly into Knowledge Catalog.
Key Integration Capabilities:

• Automated Table-Level Lineage: Automatically maps dependency graphs across tables, views, incremental tables, and custom operations.
• OpenLineage-Standard Lifecycle Events: Emits START, COMPLETE, and FAIL events mapping source and target BigQuery dataset/table relationships.
• BigQuery Job Correlation: Links Dataform actions to BigQuery Job IDs for unified auditability and lineage tracking in the GCP Console.
• Regional Endpoint Routing (REP): Direct emission to regional endpoints (datalineage.${location}.rep.googleapis.com) with automated global fallback.
• Resilient Non-Blocking Emission: Non-blocking background dispatch with transient retries and graceful silencing on missing IAM roles.
• Flexible Controls & Diagnostics: Workflow-level opt-out (lineage.enabled: false) in workflow_settings.yaml and verbose tracing with DATAFORM_LINEAGE_DEBUG.

What's Changed

• feat(lineage): harden CLI lineage emission (endpoint routing, retry) by @rafal-hawrylak in #2259
• feat(lineage): add DATAFORM_LINEAGE_DEBUG env var by @rafal-hawrylak in #2261
• test(lineage): add positive emit test for operation-type actions by @rafal-hawrylak in #2262
• feat(lineage): skip emission on UNAUTHENTICATED like PERMISSION_DENIED by @rafal-hawrylak in #2264
• test(lineage): cover multi-task errorMessage aggregation by @rafal-hawrylak in #2263
• Fix LegacyConfigConverter mutating caller-shared configs by @rafal-hawrylak in #2260
• Publishing Dataform security patches by @dataform-team in #2265

Full Changelog: 3.0.66...3.0.67

3.0.66

What's Changed

• Fix malformed dot output for operations in --dot compile output by @alpaca-tc in #2252
• Wire PropertyGraph action into compile pipeline by @rafal-hawrylak in #2251
• feat(lineage): emit OpenLineage RunEvents from CLI to Knowledge Catalog Lineage by @rafal-hawrylak in #2248
• Fix incremental table compilation failing on metadata.extraProperties by @fuzmish in #2236
• Bump vm2 from 3.11.4 to 3.11.6 by @dependabot[bot] in #2245
• Follow-up to #2251: address PropertyGraph review comments by @rafal-hawrylak in #2253
• Restore --timeout as compile-only; add --execution-timeout (#2247) by @rafal-hawrylak in #2256
• Wire PropertyGraph into CLI build and DDL execution by @rafal-hawrylak in #2255
• feat(lineage): emit GCP-oriented OpenLineage facets for Knowledge Catalog UI enrichment by @rafal-hawrylak in #2254
• Bump DF_VERSION from 3.0.65 to 3.0.66 by @rafal-hawrylak in #2257

New Contributors

• @alpaca-tc made their first contribution in #2252
• @fuzmish made their first contribution in #2236

Full Changelog: 3.0.65...3.0.66

3.0.65

What's Changed

• Add ignore js flag to format subcommand by @andres-lowrie in #2227
• Add proto surface for PropertyGraph action by @rafal-hawrylak in #2243
• JiT test suite by @rafal-hawrylak in #2185
• Add PropertyGraph action class by @rafal-hawrylak in #2246
• Assertion JiT integration by @rafal-hawrylak in #2223
• Follow-up to #2246: address PropertyGraph review comments by @rafal-hawrylak in #2249
• Bump DF_VERSION from 3.0.64 to 3.0.65 by @rafal-hawrylak in #2250

Full Changelog: 3.0.64...3.0.65