Skip to content

Cannot disable XCom push in SnowflakeOperator #29593

Closed

Description

@fritz-astronomer

Apache Airflow Provider(s)

snowflake

Versions of Apache Airflow Providers

4.0.3

Apache Airflow version

2.5.0

Operating System

docker/linux

Deployment

Astronomer

Deployment details

Normal Astro CLI

What happened

>>> SnowflakeOperator(
    ...,
    do_xcom_push=False
).execute()


ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/providers/common/sql/operators/sql.py", line 272, in execute
    return self._process_output([output], hook.descriptions)[-1]
  File "/usr/local/lib/python3.9/site-packages/airflow/providers/snowflake/operators/snowflake.py", line 118, in _process_output
    for row in result_list:
TypeError: 'NoneType' object is not iterable

What you think should happen instead

XCom's should be able to be turned off

How to reproduce

astro dev init
  1. dags/snowflake_test.py
import os
from datetime import datetime

from airflow import DAG
from airflow.providers.snowflake.operators.snowflake import SnowflakeOperator

os.environ["AIRFLOW_CONN_SNOWFLAKE"] = "snowflake://......."

with DAG('snowflake_test', schedule=None, start_date=datetime(2023, 1, 1)):
    SnowflakeOperator(
        task_id='snowflake_test',
        snowflake_conn_id="snowflake",
        sql="select 1;",
        do_xcom_push=False
    )
astro run -d dags/snowflake_test.py snowflake_test
Loading DAGs...
Running snowflake_test... [2023-02-17 18:45:33,537] {connection.py:280} INFO - Snowflake Connector for Python Version: 2.9.0, Python Version: 3.9.16, Platform: Linux-5.15.49-linuxkit-aarch64-with-glibc2.31
...
[2023-02-17 18:45:34,608] {cursor.py:727} INFO - query: [select 1]
[2023-02-17 18:45:34,698] {cursor.py:740} INFO - query execution done
...
[2023-02-17 18:45:34,785] {connection.py:581} INFO - closed
[2023-02-17 18:45:34,841] {connection.py:584} INFO - No async queries seem to be running, deleting session
FAILED
  'NoneType' object is not iterable

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. josh-fell commented on Feb 17, 2023

    @josh-fell
    Contributor

    @fritz-astronomer Would you mind adding a repro case here? I'd hate to assume anything about what's causing this issue and make the wrong conclusion.

  2. fritz-astronomer commented on Feb 17, 2023

    @fritz-astronomer
    ContributorAuthor

    Added a full DAG example, @josh-fell

  3. changed the title [-]SnowflakeOperator and XCOM[/-] [+]Cannot disable XCom push in SnowflakeOperator[/+] on Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions