Skip to content

Unable to retrieve logs for nested task group when parent is mapped #34535

Closed
#34587
Closed
Unable to retrieve logs for nested task group when parent is mapped#34535
#34587
Labels
affected_version:main_branchIssues Reported for main brancharea:webserverWebserver related Issueskind:bugThis is a clearly a bug

Description

@fritz-astronomer
fritz-astronomer
opened on Sep 21, 2023

Apache Airflow version

2.7.1

What happened

Unable to retrieve logs for task inside task group inside mapped task group.
Got 404 "TaskInstance not found" in network requests

What you think should happen instead

No response

How to reproduce

from datetime import datetime
from airflow import DAG
from airflow.decorators import task, task_group
from airflow.operators.bash import BashOperator
from airflow.utils.task_group import TaskGroup

with DAG("mapped_task_group_bug", schedule=None, start_date=datetime(1970, 1, 1)):
    @task
    def foo():
        return ["a", "b", "c"]


    @task_group
    def bar(x):
        with TaskGroup("baz"):
            # If child task group exists, logs 404
            #  "TaskInstance not found"
            # http://localhost:8080/api/v1/dags/mapped_task_group_bug/dagRuns/manual__2023-09-21T22:31:56.863704+00:00/taskInstances/bar.baz.bop/logs/2?full_content=false
            # if it is removed, logs appear
            BashOperator(task_id="bop", bash_command="echo hi $x", env={"x": x})


    bar.partial().expand(x=foo())

Operating System

debian 11 / astro dev start

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. fritz-astronomer
    added
    area:core
    kind:bugThis is a clearly a bug
    needs-triagelabel for new issues that we didn't triage yet
    on Sep 21, 2023

  2. fritz-astronomer commented on Sep 21, 2023

    @fritz-astronomer
    fritz-astronomer
    on Sep 21, 2023
    ContributorAuthor

    If this is known/expected behavior, I don't see mention of it here:
    https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html#taskgroups
    or here:
    https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/dynamic-task-mapping.html#mapping-over-a-task-group

  3. hussein-awala
    added
    affected_version:main_branchIssues Reported for main branch
    area:webserverWebserver related Issues
    and removed
    needs-triagelabel for new issues that we didn't triage yet
    area:core
    on Sep 24, 2023

  4. hussein-awala commented on Sep 24, 2023

    @hussein-awala
    hussein-awala
    on Sep 24, 2023
    Member

    I found that the log exists and available in the TaskInstance view, but the problem is in the GridView which should consider the TI as mapped TI and try to get the log for the selected index.

    I tested #34587 and it fixed the issue on the dag your provided, and I tested with a more complex case (5 nested levels) and everything looks good, could you check it?

  5. bbovenzi
    closed this as completedin #34587on Sep 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    affected_version:main_branchIssues Reported for main brancharea:webserverWebserver related Issueskind:bugThis is a clearly a bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions