Manually triggered dags should have default logical_date and run_after of "right now" #48412

Three scenarios we need to account for are:

1. Manually triggering DAG Run via UI
2. Triggering Dag Run via API
3. Trigger Dag Run Operator usage

@vatsrahul1001 talk to me / constance to confirm requirements i was going to review the slack thread again

@dstandish / @cmarteepants Currently, logical_date is a required field in the API request; however, the user can pass it as null.

Do we want to make logical_date optional, defaulting to datetime.now() if not provided, while respecting any value explicitly passed?

Triggering Dag Run via API
try to make it so if not supplied then it will be timezone.now() but if supplied we respect it. if this causes trouble and is more or less impossible to support (1), then we can keep it required

Currently, for TriggerDagRunOperator, logical_date defaults to None if not provided. Do we want to change this to datetime.now() as the default?

Just started a lazy consensus vote: https://lists.apache.org/list.html?dev@airflow.apache.org

I think it makes the most sense for TriggerDagRunOperator to default to now() if nothing is explicitly supplied. This is the current behaviour, and anyone using the operator is likely already OK with it. If the possibility of supplying None benefits some, they can opt into it explicitly.

“Not supplying a value” does not technically apply to the UI since the date field is always there and always included in the request by the frontend. A more technically correct way to put it is what initial value we present to the user in the form. As long as there is an easy way to set the value to “now” or “empty” (instead of needing to manually click through the datetimepicker, which is a lot of work), either default should be fine. Personally I lean a little toward None, but defaulting to now is not a big deal either since the possibility of a conflict should be next to none (pun unintended).

Personally I feel the API should not have a default at all, but this opinion was not received well when I proposed in Slack. If we must have a default, it should be now to match TriggerDagRunOperator. I want to state again that I still think not having a default is the correct way to go since the number of users relying on the API should be relatively low, and supplying a “now” value in the API is comparatively easy. Defaulting to now is very unintuitive and confusing to new users IMO, but it is the correct way to go if the goal is to break as few existing users as possible at any cost to potential new users,.

I feel strongly about TriggerDagRunOperator and the UI, and glad we're on the same page. If that's all we update, I can live with that.

For the API, I'm relying on feedback from DevRel that we are underestimating API usage and concern about the quantity of things that need to be changed (even if most of the changes are easy). I agree with you that if you are starting fresh, it's awkward to replace "now" with None though, if you fall in that category. TLDR: I'm personally on the fence re API, but erring towards trusting feedback from DevRel for this.

Is it too early in the major version lifetime to have a config for this? 😛

UI: #48690

UI: #48690

Thanks @bbovenzi

Bumping up to high now that lazy consensus has passed. Now that UI is done, priority is DagRunTrigger Operator. We can leave the API with the current None default.

Hello,

It seems logical_date is still a required field in the API, None is accepted but not the default.

I'm working on migration from Airflow 2 --> 3 for a large org & this change (logical_date mandatory) has become quite an issue for API migration.

If there is consensus for the default to be now() (similar to UI & TriggerDagRunOperator), can we contribute code to make logical_date optional with default behaviour now() via API?