Skip to content

[Bug]: Server emits ValueError: Unknown provider: cudo with gpuhunt >= 0.1.26 #4017

Description

@r4victor

Steps to reproduce

  1. Have a cudo backend configured
  2. Try to apply a run/fleet configuration or get offers.

You'll see ValueError: Unknown provider: cudo errors in server logs.

Actual behaviour

Latest gpuhunt release dropped cudo provider but dstack still allows configuring cudo backend and existing project may have cudo backend still.

Expected behaviour

Drop cudo backend from dstack. Existing project may have a cudo backend configured but it should not be functional.

dstack version

master

Server logs

ERROR    dstack._internal.server.services.backends:504 Got exception when
                    requesting offers from backend BackendType.CUDO
                    Traceback (most recent call last):
                      File
                    "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/utils/
                    common.py", line 51, in run_async
                        return await asyncio.get_running_loop().run_in_executor(None,
                    func_with_args)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    ^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/.local/share/uv/python/cpython-3.12.10-macos-aarch
                    64-none/lib/python3.12/concurrent/futures/thread.py", line 59, in
                    run
                        result = self.fn(*self.args, **self.kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/server
                    /services/backends/__init__.py", line 527, in get_offers_tracked
                        res = backend.compute().get_offers(requirements)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/core/b
                    ackends/base/compute.py", line 263, in get_offers
                        return iter(self._get_offers_cached(requirements))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/.venv/lib/python3.12/site-p
                    ackages/cachetools/_cachedmethod.py", line 350, in __call__
                        return wrapper(self._obj, *args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/.venv/lib/python3.12/site-p
                    ackages/cachetools/_cachedmethod.py", line 329, in wrapper
                        v = method(self, *args, **kwargs)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/core/b
                    ackends/base/compute.py", line 277, in _get_offers_cached
                        return self.get_offers_by_requirements(requirements)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/core/b
                    ackends/cudo/compute.py", line 49, in get_offers_by_requirements
                        offers = get_catalog_offers(
                                 ^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/core/b
                    ackends/base/offers.py", line 72, in get_catalog_offers
                        for item in catalog.query(**asdict(q)):
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "/Users/r4victor/Projects/dstack/dstack/.venv/lib/python3.12/site-p
                    ackages/gpuhunt/_internal/catalog.py", line 154, in query
                        raise ValueError(f"Unknown provider: {p}")
                    ValueError: Unknown provider: cudo

Additional information

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions