Skip to content

[BUG] inconsistent timestamp in FAIL events in airflow OL #2858

Closed

Description

@rahul-madaan

OpenLineage integration/client

Airflow integration

OpenLineage version

apache-airflow-providers-openlineage==1.9.1

Technology and package versions

Any version above 2.7.0

Environment configuration

default - send events to marquez

Deployment details

Local Astronomer, but any Airflow will work.

Problem details

eventTime for START and COMPLETE events are containing timezone offset
but FAIL event does not have timezone offset

eg. 2024-07-02T12:23:27.571842+00:00 - From COMPLETE event
2024-07-02T12:23:22.829392 - From FAIL event

What you think should happen instead

we should get timezone offset in FAIL events as well.

How to reproduce

run any DAG in your airflow which would have a task which always fails.
check OL events
Check Task FAIL and DAG FAIL events
eventTime does not have timezoneoffset

Additional details

It occurs everytime, although any metadata consumer service can handle this irregularity but it would be best if it can be fixed in the package itself

DAG START

Screenshot 2024-07-17 at 2 42 26 PM

TASK FAIL

Screenshot 2024-07-17 at 2 42 58 PM

Do you plan to make this contribution yourself?

  • I am interested in doing this work

Activity

  1. boring-cyborg commented on Jul 17, 2024

    @boring-cyborg

    Thanks for creating your first OpenLineage issue! Your feedback is valuable and improves the project. If you haven't already, please be sure to follow the issue template!

  2. self-assigned this
    on Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions