Display entire KubernetesExecutor worker pod logs when task is running instead of only the last 100 lines

[karenbraganz]

Description

When a task is in the running state, only the last 100 lines are streamed from the worker pod (with KubernetesExecutor). The entire logs appear once the task is completed. The 100 line limit is hardcoded here.

We should allow this limit to be configurable or display all logs that have been generated so far instead of only the last 100 lines.

Use case/motivation

Users might need the entire log history or more context to ensure that long running tasks are executing as expected.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

Feel free to implement it if you would like to, otherwise it will have to wait for someone to pick it.

[insomnes]

I would be happy to implement this.

display all logs that have been generated so far

I think that setting defaults this way may lead to OOMs in the executor, it loads all logs in memory, I believe. But I can double-check it during the implementation

But giving the ability to increase or even disable the limit if the user knows what they do, I am 100% on it.

[gdivya6028]

hi @potiuk , I would like to work on this issue.

[karenbraganz]

I am assigning this to @insomnes for now since they had already requested this.

[insomnes]

Sorry for disappearing. A number of unpleasant things happened. However, I'm back now and will try to finish the issue.
I'm rebasing and refreshing my PR and will try to request the review for it after the green pipelines.

[potiuk]

@insomnes We are unassigning you from this issue as part of our updated assignment policy.

This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.