Summary
pgflow.start_tasks() computes effective per-task visibility delays and calls pgflow.set_vt_batch() in an unreferenced SELECT CTE. PostgreSQL may skip an unreferenced SELECT CTE, so a claimed task can keep only the shorter visibility set by the initial PGMQ read.
The message may become visible while its step_tasks row remains started. Current workers repeatedly read that message and return fewer tasks than messages. Future per-step workers must not mistake this healthy duplicate visibility for unsupported work and stop.
Current path
pkgs/core/schemas/0120_function_start_tasks.sql contains:
timeouts as (
select
task.message_id,
task.flow_slug,
coalesce(step.opt_timeout, flow.opt_timeout) + 2 as vt_delay
from tasks task
...
),
set_vt_batch as (
select pgflow.set_vt_batch(...)
from timeouts
)
select ...
from tasks ...
The final query does not reference set_vt_batch. This does not guarantee that pgflow.set_vt_batch() runs.
Required behavior
For every task that changes from queued to started:
- compute the same effective timeout used by stalled recovery;
- extend that message's visibility before returning the task to a handler;
- increment the attempt exactly once;
- return no task whose visibility extension failed;
- preserve atomic behavior for the claimed batch.
A message that becomes visible while its exact task remains started is a benign duplicate. Reading it must consume no new attempt and must not stop the worker.
Implementation boundary
Make the visibility side effect structurally required. Use PL/pgSQL PERFORM, a referenced CTE whose result the final statement consumes, or another statement form whose execution does not depend on demand for an unreferenced SELECT CTE.
Keep timeout semantics aligned across:
initial PGMQ read visibility
start_tasks() effective visibility extension
#621 stalled-task threshold
requeue_stalled_tasks() visibility reset
Do not fold queue-per-step routing into this fix. #650 will later add canonical queue_name and group visibility updates by queue.
Tests
Relationship to other work
Summary
pgflow.start_tasks()computes effective per-task visibility delays and callspgflow.set_vt_batch()in an unreferencedSELECTCTE. PostgreSQL may skip an unreferencedSELECTCTE, so a claimed task can keep only the shorter visibility set by the initial PGMQ read.The message may become visible while its
step_tasksrow remainsstarted. Current workers repeatedly read that message and return fewer tasks than messages. Future per-step workers must not mistake this healthy duplicate visibility for unsupported work and stop.Current path
pkgs/core/schemas/0120_function_start_tasks.sqlcontains:The final query does not reference
set_vt_batch. This does not guarantee thatpgflow.set_vt_batch()runs.Required behavior
For every task that changes from
queuedtostarted:A message that becomes visible while its exact task remains
startedis a benign duplicate. Reading it must consume no new attempt and must not stop the worker.Implementation boundary
Make the visibility side effect structurally required. Use PL/pgSQL
PERFORM, a referenced CTE whose result the final statement consumes, or another statement form whose execution does not depend on demand for an unreferencedSELECTCTE.Keep timeout semantics aligned across:
Do not fold queue-per-step routing into this fix. #650 will later add canonical
queue_nameand group visibility updates by queue.Tests
coalesce(step.opt_timeout, flow.opt_timeout) + 2beforestart_tasks()returns.startedtask consumes no additional attempt.Relationship to other work
startedmessages as benign duplicates rather than unsupported work.