Repository navigation
RECURSIVE CTE no longer compiles correctly for INCREMENTAL insert #2083
Description
Recursive CTE like the example below now compiles to non-working code due to proxy-procedure being used. Worked before on dataform/core=2.4.2.
Code:
EXECUTE IMMEDIATE """
CREATE OR REPLACE PROCEDURE project.schema.df_09289b69b6be2dab9e8ed4e01adaf8caa53b57ceca86ea247a2494e585e21160() OPTIONS(strict_mode=false)
BEGIN
INSERT INTO project.schema.table
(""" || dataform_columns_list || """)
SELECT """ || dataform_columns_list || """
FROM (
WITH RECURSIVE -- materializes output of non-deterministic functions
cte AS (
SELECT *
FROM project.schema.some_table
)
...
)
Error:
Query error: WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.
Issue didn't occur in "non-procedural inserts" of versions 2.x.
Activity
kolina commentedon Feb 6, 2026
Can you show your config block? Do you use GCP Dataform?
bartekelm commentedon Feb 23, 2026
Yes, it's GCP DF.
kolina commentedon Mar 2, 2026
I think the reason is not in a generated procedure, but in additional
SELECTquery wrapping yourSELECTwith recursive CTE.To fix it, the most feasible way is to create an intermediate temporary table with results of the incremental query and generate the insert statement separately. I've created an internal bug to track it. Because this is not related to open-source, can you please file an issue in our Public Tracker?
As a workaround (somewhat), you can enable
onSchemaChangevalue different fromIGNORE. In this case we're already generating SQL with putting incremental results into a temporary table, so you can check if this works for your case (obviously if you're fine with enabling automatic incremental table changes).