OpenAI client intercepts Celery SoftTimeLimitExceeded exception if it happens during API call #2737

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

Issue description

When using the OpenAI Python SDK inside a Celery task with a soft time limit, if the task exceeds its time limit while the client is making a request, Celery raises a SoftTimeLimitExceeded exception.
However, the OpenAI client catches this exception in _base_client.py under a broad except Exception clause (since SoftTimeLimitExceeded is an instance of Exception), causing the error to be handled internally as a retryable connection error instead of propagating back to user code.

This prevents the Celery task from ever receiving the SoftTimeLimitExceeded exception, meaning any cleanup or graceful shutdown logic in the task is skipped.

Proposed fix

Option 1

Restrict retry logic to network-related exceptions (e.g., httpx.RequestError) rather than all Exceptions.

Option 2

Explicitly re-raise termination exceptions (SoftTimeLimitExceeded, asyncio.CancelledError, KeyboardInterrupt, SystemExit).

To Reproduce

Option 1

1.. Write a Celery task that has a very short soft time limit (5 - 10 seconds)
2. Write cleanup logic for SoftTimeLimitExceeded exception (you can just print "cleanup")
3. In that task spam requests towards OpenAI API using the openai-python client library
4. Set max retries as something like 5
5. When SoftTimeLimitExceeded exception is thrown (when soft time limit is reached) the OpenAI client will retry the request and "cleannup" will never be printed

Option 2

1 - 3 stay the same
4. Set max retries to 0
5. When SoftTimeLimitExceeded exception is thrown (when soft time limit is reached) the OpenAI client will raise APIConnectionError and "cleannup" will never be printed

Code snippets

openai-python/src/openai/_base_client.py
---------------------------------------------
except Exception as err:
log.debug("Encountered Exception", exc_info=True)

if remaining_retries > 0:
    self._sleep_for_retry(
        retries_taken=retries_taken,
        max_retries=max_retries,
        options=input_options,
        response=None,
    )
    continue

log.debug("Raising connection error")
raise APIConnectionError(request=request) from err

OS

Linux

Python version

Python v3.11

Library version

openai v2.7.1