fix(trainer): validate HuggingFace storage_uri has both user and repo - #575
google-oss-prow[bot] merged 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR tightens early validation of HuggingFace storage_uri values used by Trainer initializers so malformed hf:// URIs (missing either the user or repo component) fail fast at construction time instead of later during download.
Changes:
- Added a shared
_validate_hf_storage_uri(storage_uri, entity)helper and used it from bothHuggingFaceModelInitializerandHuggingFaceDatasetInitializer. - Expanded unit tests to cover
HuggingFaceDatasetInitializervalidation and added additional invalidstorage_uricases (missing user or repo).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
kubeflow/trainer/types/types.py |
Consolidates HuggingFace storage_uri validation and enforces both user and repo components for model/dataset initializers. |
kubeflow/trainer/types/types_test.py |
Adds parametrized tests for both HuggingFace initializers, including newly rejected URI forms. |
The HuggingFace initializers only rejected a storage_uri with an empty path, so URIs missing either component, such as hf://user/ (no repo) or hf:///model (no user), were accepted and failed later at download time. Consolidate the validation into a shared _validate_hf_storage_uri helper used by both HuggingFaceModelInitializer and HuggingFaceDatasetInitializer, requiring a non-empty user and repo. Add tests for the dataset initializer, which had no prior coverage, and for the new rejected forms. Follow-up to kubeflow#564. Signed-off-by: Sahil Kumar Singh <sahilkumargreat12@gmail.com>
3a7595f to
da5414e
Compare
andreyvelich
left a comment
There was a problem hiding this comment.
/ok-to-test
/retest
/lgtm
/approve
Thanks @SahilKumar75!
/cc @akshaychitneni
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: andreyvelich The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
* chore!: upgrade to Python 3.10 (#282)
This upgrades the minimum Python version for the project from 3.9 to
3.10. Python 3.9 is past end-of-life and dependencies will likely
require a supported version soon.
Signed-off-by: Jon Burdo <jon@jonburdo.com>
* chore: Confirm that a public ConfigMap exists to check version (#250)
* Confirm that a public ConfigMap exists to check version
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* python 3.9 fix
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Surya Sameer Datta Vaddadi <137607947+sameerdattav@users.noreply.github.com>
* Exceptiom handling better
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Surya Sameer Datta Vaddadi <137607947+sameerdattav@users.noreply.github.com>
* Addressing comments
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* Update kubeflow/trainer/backends/kubernetes/backend.py
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Surya Sameer Datta Vaddadi <137607947+sameerdattav@users.noreply.github.com>
* Refactored tests into a single function and followed agents.md
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* CI friendly edit
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* pre-commit format checked
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* Modified according to new updates
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* Ran pre-commit locally to fix formatting
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* unix2dos CLAUDE.md
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
* Revert CLAUDE.md
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
---------
Signed-off-by: Surya Sameer Datta Vaddadi <f20220373@goa.bits-pilani.ac.in>
Signed-off-by: Surya Sameer Datta Vaddadi <137607947+sameerdattav@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: added sdk docs website to readme (#284)
* docs: added sdk docs website to readme
Signed-off-by: Akash Jaiswal <akashjaiswal3846@gmail.com>
* format: order of sdk docs
Signed-off-by: Akash Jaiswal <akashjaiswal3846@gmail.com>
---------
Signed-off-by: Akash Jaiswal <akashjaiswal3846@gmail.com>
* feat(trainer): add dataset and model initializer support to container backend (#188)
* feat(trainer): add dataset and model initializer support to container backend
Add support for dataset and model initializers in the container backend
to bring it to feature parity with the Kubernetes backend.
Changes:
- Add utility functions for building initializer commands and environment variables
- Implement _run_initializers() and _run_single_initializer() methods in ContainerBackend
- Run initializers sequentially before training containers start
- Download datasets to /workspace/dataset and models to /workspace/model
- Track initializer containers as separate steps in TrainJob
- Support all initializer types: HuggingFace, S3, and DataCache
- Add comprehensive unit tests for all initializer configurations
- Handle initializer failures with proper cleanup and error messages
Fixes #171
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
* feat(trainer): address reviewer feedback for initializer support
- Make initializer image configurable via ContainerBackendConfig
- Make initializer timeout configurable (default 600 seconds)
- Implement wait API in adapters instead of polling
- Clean up successful initializer containers after completion
- Clean up network on initializer failure
- Raise ValueError for unsupported initializer types (no datacache fallback)
All tests passing (173/173). Addresses all feedback from PR #188.
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
* chore(trainer): add cleanup helper to reduce duplication
Add _cleanup_container_resources() helper method to consolidate
duplicated cleanup logic for stopping/removing containers and
deleting networks. Refactor 5 locations across train(), initializer
handlers, and delete_job() to use this helper.
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
* fix(trainer): use correct initializer images and working directory
Address feedback for initializer support in container backend:
- Use separate images for dataset/model initializers:
- kubeflow/dataset-initializer:latest for datasets
- kubeflow/model-initializer:latest for models
(instead of kubeflow/training-operator:latest)
- Update python commands to use pkg.initializers module:
- python -m pkg.initializers.dataset (for dataset)
- python -m pkg.initializers.model (for model)
- Change initializer working_dir from /workspace to /app
per Dockerfile convention
Refs: https://github.com/kubeflow/trainer/tree/master/cmd/initializers
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
* fix(container): address PR review comments for initializer support
- Use GHCR images as default for dataset/model initializers
- Replace suppress with try-except blocks
- Refactor initializer utils with ContainerInitializer dataclass
- Add get_dataset_initializer and get_model_initializer functions
- Remove DataCache support (unsupported in container backend)
- Merge initializer tests into test_train() and test_get_job_logs()
- Remove duplicate test functions
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
* fix(container): add name field to ContainerInitializer and remove init_type
- Add name field to ContainerInitializer dataclass
- Set name='dataset-initializer' and name='model-initializer' in utils
- Remove init_type parameter from _run_single_initializer()
- Use container_init.name for labels and log messages
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
---------
Signed-off-by: HKanoje <hrithik.kanoje@gmail.com>
* feat: add SparkClient API for SparkConnect session management (#225)
* feat(spark): add core types, dataclasses, and constants
- Add SparkConnectInfo, SparkConnectState, Driver, Executor types
- Add type tests for validation
- Add Kubernetes backend constants (CRD group, version, defaults)
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* feat(spark): add backend base class and options pattern
- Add RuntimeBackend abstract base class with session lifecycle methods
- Add options pattern (Name, Image, Timeout, etc.) aligned with trainer SDK
- Add validation utilities for connect parameters
- Add comprehensive option tests
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* feat(spark): add KubernetesBackend for SparkConnect CRD operations
- Implement KubernetesBackend with create/get/list/delete session methods
- Add port-forward support for out-of-cluster connections
- Add CRD builder utilities and URL validation
- Add comprehensive backend and utils tests with parametrized patterns
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* feat(spark): add SparkClient API with KEP-107 compliant connect method
- Implement SparkClient as main user interface for SparkConnect sessions
- Support connect to existing server (base_url) or auto-create new session
- Add public exports for SparkClient, Driver, Executor, options
- Add SparkClient unit tests
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* chore(spark): add test infrastructure and package init files
- Add test common utilities and fixtures
- Add package __init__ files for test directories
- Setup test/e2e/spark structure
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* feat(spark): add example scripts demonstrating SparkClient usage
- Add spark_connect_simple.py with 3 usage levels (minimal, simple, advanced)
- Add spark_advanced_options.py with full configuration examples
- Add connect_existing_session.py for connecting to existing servers
- Add demo and test scripts for local development
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* docs(spark): add documentation for SparkClient and E2E testing
- Add examples/spark/README.md with usage guide
- Add local Spark Connect testing documentation
- Add E2E test README with CI/CD integration guide
- Update KEP-107 proposal documentation
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* test(spark): add E2E test framework with cluster watcher
- Add test_spark_examples.py with example validation tests
- Add cluster_watcher.py for monitoring SparkConnect and pods during tests
- Add run_in_cluster.py for executing examples as K8s Jobs
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* ci(spark): add GitHub Actions workflow and E2E cluster setup
- Add test-spark-examples.yaml workflow for E2E validation
- Add e2e-setup-cluster.sh for Kind cluster with Spark Operator
- Add SparkConnect CRD, Kind config, and E2E runner Dockerfile
- Update Makefile with E2E setup target
- Update PR title check for spark prefix
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* chore(spark): add pyspark[connect] dependency and update lock file
- Add spark extra with pyspark[connect]==3.4.1 for grpcio, pandas, pyarrow
- Update uv.lock with resolved dependencies
- Update .gitignore for spark-related files
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* Update kubeflow/spark/backends/base.py
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Shekhar Prasad Rajak <5774448+Shekharrajak@users.noreply.github.com>
* refactor(spark): rename backend.connect_session() to connect()
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
* refactor: move session creation flow from SparkClient to backend.create_and_connect()
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
---------
Signed-off-by: Shekhar Rajak <shekharrajak@live.com>
Signed-off-by: Shekhar Prasad Rajak <5774448+Shekharrajak@users.noreply.github.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: bump minimum model-registry version to 0.3.6 (#289)
Signed-off-by: Jon Burdo <jon@jonburdo.com>
* fix: Improve CVE workflow (#267)
* fix: Improve CVE workflow
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix: fix issue with bash compare
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* feat: Add workflow to cleanup overrides in pyproject.toml
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix: address review comments
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* chore: refactor to reduce size of cve related workflows
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
---------
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* chore: upgrade code style for python3.10 (#288)
* chore: update code style for Python 3.10
This disables a couple ruff rules in pyproject.toml:
```
"UP007", # Use X | Y instead of Union[X, Y] (requires Python 3.10+)
"UP045", # Use X | None instead of Optional[X] (requires Python 3.10+)
```
Then the code changes are made with:
```
uv run ruff check --fix
uv run ruff format
```
Signed-off-by: Jon Burdo <jon@jonburdo.com>
* fix: handle unions, bools in convert_value
The convert_value function didn't seems to be handling union types
properly and also needs to handle `T | None` similarly to
`Optional[None]` after the upgrade to Python 3.10. This fixes union
types, an issue with bool conversion, and adds tests for this function.
Signed-off-by: Jon Burdo <jon@jonburdo.com>
---------
Signed-off-by: Jon Burdo <jon@jonburdo.com>
* chore(ci): bump astral-sh/setup-uv from 5 to 7 (#276)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 5 to 7.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/v5...v7)
---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
dependency-version: '7'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump the python-minor group across 1 directory with 4 updates (#291)
Bumps the python-minor group with 4 updates in the / directory: [coverage](https://github.com/coveragepy/coveragepy), [ruff](https://github.com/astral-sh/ruff), [pre-commit](https://github.com/pre-commit/pre-commit) and [ty](https://github.com/astral-sh/ty).
Updates `coverage` from 7.10.7 to 7.13.4
- [Release notes](https://github.com/coveragepy/coveragepy/releases)
- [Changelog](https://github.com/coveragepy/coveragepy/blob/main/CHANGES.rst)
- [Commits](https://github.com/coveragepy/coveragepy/compare/7.10.7...7.13.4)
Updates `ruff` from 0.14.14 to 0.15.0
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.14.14...0.15.0)
Updates `pre-commit` from 4.3.0 to 4.5.1
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v4.3.0...v4.5.1)
Updates `ty` from 0.0.14 to 0.0.16
- [Release notes](https://github.com/astral-sh/ty/releases)
- [Changelog](https://github.com/astral-sh/ty/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ty/compare/0.0.14...0.0.16)
---
updated-dependencies:
- dependency-name: coverage
dependency-version: 7.13.4
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: python-minor
- dependency-name: ruff
dependency-version: 0.15.0
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: python-minor
- dependency-name: pre-commit
dependency-version: 4.5.1
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: python-minor
- dependency-name: ty
dependency-version: 0.0.16
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: python-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat: Added examples to the documentation demonstrating different ways to handle ports (#243)
* update docs and add test cases.
Signed-off-by: osamaahmed17 <osamaahmedtahir17@gmail.com>
* pre-commit error solved
Signed-off-by: osamaahmed17 <osamaahmedtahir17@gmail.com>
* Update kubeflow/hub/api/model_registry_client.py
Co-authored-by: Jon Burdo <jon@jonburdo.com>
Signed-off-by: Osama Tahir <31954609+osamaahmed17@users.noreply.github.com>
* readme updated
Signed-off-by: Osama Tahir <31954609+osamaahmed17@users.noreply.github.com>
* Refactor model registry client test cases for clarity
Signed-off-by: Osama Tahir <31954609+osamaahmed17@users.noreply.github.com>
---------
Signed-off-by: osamaahmed17 <osamaahmedtahir17@gmail.com>
Signed-off-by: Osama Tahir <31954609+osamaahmed17@users.noreply.github.com>
Co-authored-by: Jon Burdo <jon@jonburdo.com>
* chore(ci): bump peter-evans/create-pull-request from 6 to 8 (#277)
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 8.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6...v8)
---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
dependency-version: '8'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(ci): bump actions/checkout from 4 to 6 (#278)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v6)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: '6'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat: Adds a GitHub Actions workflow to check kubeflow/hub/OWNERS. (#280)
* Add OWNERS validation
Signed-off-by: muhammadjunaid8047 <muhammadjunaid8047@gmail.com>
* Update .github/workflows/check-owners.yaml
Co-authored-by: Jon Burdo <jon@jonburdo.com>
Signed-off-by: Muhammad Junaid <muhammadjunaid8047@gmail.com>
* Update OWNERS file check in workflow
Signed-off-by: Muhammad Junaid <muhammadjunaid8047@gmail.com>
* Update paths in check-owners workflow
Signed-off-by: Muhammad Junaid <muhammadjunaid8047@gmail.com>
---------
Signed-off-by: muhammadjunaid8047 <muhammadjunaid8047@gmail.com>
Signed-off-by: Muhammad Junaid <muhammadjunaid8047@gmail.com>
Co-authored-by: Jon Burdo <jon@jonburdo.com>
* fix: nightly security dependency updates (#296)
Co-authored-by: google-oss-prow <92114575+google-oss-prow@users.noreply.github.com>
* chore(ci): bump aquasecurity/trivy-action from 0.33.1 to 0.34.0 in the actions group (#297)
Bumps the actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).
Updates `aquasecurity/trivy-action` from 0.33.1 to 0.34.0
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/0.33.1...0.34.0)
---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
dependency-version: 0.34.0
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: actions
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump pytest from 8.4.2 to 9.0.2 (#301)
Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.4.2 to 9.0.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/8.4.2...9.0.2)
---
updated-dependencies:
- dependency-name: pytest
dependency-version: 9.0.2
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(trainer): Support namespaced TrainingRuntime in the SDK (#130)
* feat(backend): Support namespaced TrainingRuntime in the SDK
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Fixed bugs and validated current test cases
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Fixed pre-commit test failure
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Addressed comments
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Fixed no attribute 'DEFAULT_TIMEOUT' error
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Added namespace-scoped runtime to test cases
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Addressed fallback logic bugs
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Added scope field to Runtime
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Improved code
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Fixed copilot's comments
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Shadow duplicate runtimes, priority to ns
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Fixed bug
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Fixed copilot comments
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Improved test cases to validate all possible cases
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* small fix
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* lint fix
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* improved error message
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Moeed <shaikmoeed@gmail.com>
* refactored code
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* improve code
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* Removed RuntimeScope
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* removed scope references and improved error handling as per kubeflow standards
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
---------
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
Signed-off-by: Moeed <shaikmoeed@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: Fix runtime lookup fallback and test local SDK in E2E (#307)
* fix: Install SDK locally in E2E workflow and improve error handling for runtime fetching in Kubernetes backend.
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
* refactor: Explicitly return errors from and refine exception handling in .
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
* docs: update comment to clarify Kubeflow SDK installation from source in e2e workflow.
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
* feat: Enhance runtime retrieval tests to cover Kubernetes API 404/403 errors and partial success for list operations on timeout.
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
* refactor: Update runtime listing to immediately raise exceptions on failure instead of collecting partial results.
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
---------
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
* chore(ci): bump actions/setup-python from 5 to 6 (#298)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v5...v6)
---
updated-dependencies:
- dependency-name: actions/setup-python
dependency-version: '6'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump the python-minor group with 2 updates (#299)
Bumps the python-minor group with 2 updates: [ruff](https://github.com/astral-sh/ruff) and [ty](https://github.com/astral-sh/ty).
Updates `ruff` from 0.15.0 to 0.15.1
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.0...0.15.1)
Updates `ty` from 0.0.16 to 0.0.17
- [Release notes](https://github.com/astral-sh/ty/releases)
- [Changelog](https://github.com/astral-sh/ty/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ty/compare/0.0.16...0.0.17)
---
updated-dependencies:
- dependency-name: ruff
dependency-version: 0.15.1
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: python-minor
- dependency-name: ty
dependency-version: 0.0.17
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: python-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: improve logging around packages_to_install (#269)
* improve logging around packages_to_install
Signed-off-by: Brian Gallagher <briangal@gmail.com>
* exit when pip install fails, append errors from both attempts
Signed-off-by: Brian Gallagher <briangal@gmail.com>
* Add shlex to address command injection vulnerabilities. Write pip install logfile to cwd
Signed-off-by: Brian Gallagher <briangal@gmail.com>
---------
Signed-off-by: Brian Gallagher <briangal@gmail.com>
* feat: Add validate lockfile workflow to complement CVE scanning (#306)
* feat: Add validate lockfile workflow to complement CVE scanning
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix: make cve fix pr branch static
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
---------
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix(trainer): handle falsy values in get_args_from_peft_config (#328)
* fix(trainer): handle falsy values in get_args_from_peft_config
Signed-off-by: krishdef7 <gargkrish06@gmail.com>
* fix: apply pre-commit formatting
Signed-off-by: krishdef7 <gargkrish06@gmail.com>
* fix: also handle falsy train_on_input in dataset_preprocess_config
Signed-off-by: krishdef7 <gargkrish06@gmail.com>
* fix: add missing newline at end of utils_test.py
Signed-off-by: krishdef7 <gargkrish06@gmail.com>
* fix: pre-commit formatting
Signed-off-by: krishdef7 <gargkrish06@gmail.com>
---------
Signed-off-by: krishdef7 <gargkrish06@gmail.com>
* fix(optimizer): prevent input mutation in optimize() (#322)
* fix(optimizer): prevent input mutation in optimize()
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
* remove unnecessary things
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
* rename test
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
---------
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
* feat: add TrainerClient examples for local PyTorch distributed training (#312)
* docs: add TrainerClient examples for local PyTorch distributed training
- Add examples/trainer/pytorch_distributed_simple.py
- Add examples/trainer/README.md
- Demonstrates LocalProcessBackend usage without Kubernetes
- Fixes #218
Signed-off-by: Mansi Singh <singh.m1@northeastern.edu>
* docs: add training examples table to SDK website
Signed-off-by: Mansi Singh <singh.m1@northeastern.edu>
* docs: expand examples table with PyTorch, MLX, DeepSpeed, and TorchTune examples grouped by framework
Signed-off-by: Mansi Singh <singh.m1@northeastern.edu>
---------
Signed-off-by: Mansi Singh <singh.m1@northeastern.edu>
* chore: fix docstrings in TrainerClient (#333)
Signed-off-by: Transcendental-Programmer <priyena.programming@gmail.com>
* feat(spark): Refactor unit tests to sdk coding standards (#293)
* Refactored unit test
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* Changes made
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* Version
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* Restructured clien_test
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* reformated backend_test.py
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* revert pyproject.toml and uv.lock changes
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* Standarized spark backend tests
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* backend_tests
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
---------
Signed-off-by: digvijay-y <yewaredigvijay@gmail.com>
* fix(optimizer): add missing get_job_events() to RuntimeBackend base c… (#325)
* fix(optimizer): add missing get_job_events() to RuntimeBackend base class
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
* Update kubeflow/optimizer/backends/base.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Ruskaruma <154019945+ruskaruma@users.noreply.github.com>
* Update kubeflow/optimizer/backends/base.py
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Ruskaruma <154019945+ruskaruma@users.noreply.github.com>
* fix: add abstractmethod, remove docstrings
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
* make get_job_events abstract in RuntimeBackend
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
* Update kubeflow/trainer/backends/localprocess/backend.py
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Ruskaruma <154019945+ruskaruma@users.noreply.github.com>
* fix
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
---------
Signed-off-by: ruskaruma <ishaan.sinha10@gmail.com>
Signed-off-by: Ruskaruma <154019945+ruskaruma@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore(spark): migrate SDK to kubeflow_spark_api Pydantic models (#295)
* chore(spark): add kubeflow-spark-api dependency
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* chore(spark): migrate options to typed Pydantic models
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* chore(spark): migrate utils to typed Pydantic models
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* chore(spark): migrate backend to typed Pydantic models
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* chore(spark): refactor tests to use typed models and cleanup
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* chore(spark): rename build_spark_connect_crd to build_spark_connect_cr
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* fix(spark): use typed model helpers in mock handlers
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* chore(spark): bump kubeflow-spark-api to 2.4.0
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
---------
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* feat(docs): Update README with Spark Support (#349)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix(trainer): return TRAINJOB_COMPLETE when all steps are done (#340)
* fix(local): return TRAINJOB_COMPLETE when all steps are done (#338)
Signed-off-by: priyank <priyank8445@gmail.com>
* test(trainer): add test case for __get_job_status
Signed-off-by: priyank <priyank8445@gmail.com>
* fix(trainer): early return TRAINJOB_CREATED when job has no steps
Signed-off-by: priyank <priyank8445@gmail.com>
* test(trainer): refactor test_get_job_status with TestCase fixture
Signed-off-by: priyank <priyank8445@gmail.com>
---------
Signed-off-by: priyank <priyank8445@gmail.com>
* fix(trainer): adapt SDK to removal of numProcPerNode from TorchMLPolicySource (#360)
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* fix: Make validate-lockfile action non-blocking (#361)
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* chore(spark): change pyspark[connect] dependency (#357)
Change pyspark[connect] 3.4.1 dependency to pyspark-connect 4.0.1.
This matches the version of Spark in the spark-operator container image
(https://github.com/kubeflow/spark-operator/blob/master/Dockerfile#L17).
Signed-off-by: Ali Maredia <amaredia@redhat.com>
* chore(spark): remove SDK-side validation from SparkClient (#345)
Remove all SDK-side input validation from the spark module.
Validation will be handled server-side by the Spark Operator
admission webhooks (spark-operator#2862).
- Remove validation.py and validation_test.py
- Remove isinstance checks from _create_session()
- Remove ValidationError from public API
Closes: #272
Signed-off-by: Yassin Nouh <yassinnouh21@gmail.com>
Signed-off-by: yassinnouh21 <yassinnouh21@gmail.com>
* chore(spark): add Spark documentation and API reference (#364)
* docs(spark): add Spark documentation and API reference
Signed-off-by: Amir380-A <62997533+Amir380-A@users.noreply.github.com>
* chore(spark): add spark installation section
Signed-off-by: Amir380-A <62997533+Amir380-A@users.noreply.github.com>
* chore(docs): apply pre-commit formatting fixes
Signed-off-by: Amir380-A <62997533+Amir380-A@users.noreply.github.com>
* chore(docs): apply changes
Signed-off-by: Amir380-A <62997533+Amir380-A@users.noreply.github.com>
* Update docs/source/spark/index.rst
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Amir Ibrahim <62997533+Amir380-A@users.noreply.github.com>
---------
Signed-off-by: Amir380-A <62997533+Amir380-A@users.noreply.github.com>
Signed-off-by: Amir Ibrahim <62997533+Amir380-A@users.noreply.github.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix(trainer): ignore PEP 668 system python check (#384)
Signed-off-by: Rob Bell <robell@redhat.com>
* feat(trainer): replace PodTemplateOverrides with RuntimePatches API (#381)
* feat(trainer): replace PodTemplateOverrides with RuntimePatches API
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix: address review comments
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
---------
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix(trainer): Fix packages installation with extra notation (#385)
* fix(trainer): Fix packages installation with extra notation
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Update pip index URL format
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
---------
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* feat: Run dataset and model initializers in parallel (#313)
* feat(trainer): Run dataset and model initializers in parallel (#290)
Signed-off-by: priyank <priyank8445@gmail.com>
* feat(container): refactor initializer dispatch into a loop
Signed-off-by: priyank <priyank8445@gmail.com>
* style(container): fix ruff formatting
Signed-off-by: priyank <priyank8445@gmail.com>
---------
Signed-off-by: priyank <priyank8445@gmail.com>
* fix(trainer): add missing wildcard to .pt and .pth ignore patterns (#372)
* fix(trainer): add missing wildcard to .pt and .pth ignore patterns
Signed-off-by: Vahagn <vghazaryan@cloudlinux.com>
* docs(trainer): fix wildcard patterns in S3ModelInitializer docstring
Signed-off-by: ghazariann <vahagn.ghazayan@gmail.com>
---------
Signed-off-by: Vahagn <vghazaryan@cloudlinux.com>
Signed-off-by: ghazariann <vahagn.ghazayan@gmail.com>
Co-authored-by: Vahagn <vghazaryan@cloudlinux.com>
* chore(trainer): fix typos in TrainerClient docstrings (#394)
Fix grammatical error in list_jobs returns description ("TrainJob exist" -> "TrainJobs exist") and add missing period in get_job docstring.
Signed-off-by: andres75125 <starlord52mcu@gmail.com>
* chore(trainer): Add API reference docs for kubeflow.trainer.options classes (#396)
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix: support RC version format in Makefile release target (#398)
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* Kubeflow SDK Official Release 0.4.0rc0 (#397)
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* chore(ci): bump aquasecurity/trivy-action from 0.34.0 to 0.34.1 in the actions group (#319)
Bumps the actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).
Updates `aquasecurity/trivy-action` from 0.34.0 to 0.34.1
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/0.34.0...0.34.1)
---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
dependency-version: 0.34.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: actions
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump Kubeflow Trainer API to 2.2.0 (#406)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Kubeflow SDK Official Release 0.4.0 (#404)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore(docs): Add Apache Spark to the diagram (#425)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix: remove validate-lockfile workflow as it uses trivy (#427)
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* chore(ci): Remove Trivy Action (#426)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* feat(docs): Add Kubeflow SDK 0.4 blog to the latest news (#428)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix: change documentation typo (#449)
* fix: change documentation typo
Signed-off-by: Slowlybomb <hslyusar@redhat.com>
* fix: path in a comment
Signed-off-by: Slowlybomb <hslyusar@redhat.com>
---------
Signed-off-by: Slowlybomb <hslyusar@redhat.com>
* chore(trainer): add data and model initializers guide (#414)
* chore(trainer): add data and model initializers guide
Add docs/source/train/initializers.rst covering dataset and model
initializers for the container backend (added in #188, parallelised
in #313). Includes per-type code examples, combined usage, ContainerBackendConfig
options, and debugging via get_job_logs().
Signed-off-by: Ayush Petwal <ayushpetwal.0105@gmail.com>
* docs(trainer): address PR review comments on initializers guide
- Fix model output path from /workspace/model-weights to /workspace/model
to match the MODEL_PATH constant in constants.py
- Clarify DataCacheInitializer is Kubernetes-only in the backend note
and annotate the Available Initializers table row accordingly
- Add DataCacheInitializer usage example with required fields
(storage_uri, metadata_loc, num_data_nodes) and backend constraint note
Signed-off-by: 1Ayush-Petwal <ayushpetwal.0105@gmail.com>
* docs(trainer): use latest tag for initializer images in guide
Signed-off-by: 1Ayush-Petwal <ayushpetwal.0105@gmail.com>
---------
Signed-off-by: Ayush Petwal <ayushpetwal.0105@gmail.com>
Signed-off-by: 1Ayush-Petwal <ayushpetwal.0105@gmail.com>
* fix(trainer): improve error diagnostics and add retry logic (#451)
* fix(trainer): improve error diagnostics and add retry logic
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* chore: ensure consistent message, move test to util file
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
---------
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
* fix(hub): add params to register_model() (#440)
* fix(hub): add storage_key, storage_path, service_account_name to register_model
Signed-off-by: 1Ayush-Petwal <ayushpetwal.0105@gmail.com>
* refactor(hub): group storage args into StorageConfig dataclass
Replace the three flat kwargs (storage_key, storage_path,
service_account_name) on register_model() with a single
storage_config: StorageConfig | None = None parameter.
Introduces kubeflow/hub/types/types.py with the StorageConfig
dataclass (matching the trainer/optimizer/spark types/ pattern),
re-exports it from kubeflow.hub, updates the docstring, and adds
parametrized test cases verifying correct field forwarding.
Signed-off-by: 1Ayush-Petwal <ayushpetwal.0105@gmail.com>
---------
Signed-off-by: 1Ayush-Petwal <ayushpetwal.0105@gmail.com>
* fix(trainer): validate polling_interval is strictly less than timeout (#402)
* fix(trainer): validate polling_interval is strictly less than timeout
In wait_for_job_status(), the guard used strict greater-than (>),
allowing polling_interval == timeout to pass validation. This results
in round(timeout/polling_interval) == 1, meaning the job is polled
exactly once with no retry window — silently wrong behavior that
contradicts the documented constraint 'must be less than timeout'.
Fix: change > to >= in the validation guard.
Fixes #400
Signed-off-by: Prabinder Singh <prabindersinghh@gmail.com>
* fix(trainer): address review comments on polling_interval validation
- Add guard for zero/negative polling_interval to prevent ZeroDivisionError
at round(timeout / polling_interval); raise ValueError with clear message
- Improve ValueError message for polling_interval >= timeout to include
actual parameter values for easier debugging
- Replace hardcoded "basic-job" string with BASIC_TRAIN_JOB_NAME constant
in the new test case to match existing conventions
- Add test cases for zero and negative polling_interval values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Prabinder Singh <prabindersinghh@gmail.com>
* fix(trainer): extend polling_interval validation to all backends
Apply same polling_interval validation to localprocess and optimizer
backends as requested in PR review.
- Add polling_interval <= 0 guard to prevent ZeroDivisionError
- Change > to >= so equal values are also rejected (strictly less than)
- Improve ValueError messages to include actual parameter values
- Add parametrized test cases for all three validation scenarios
(greater than, equal to, zero, negative) in both backend test files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Prabinder Singh <prabindersinghh@gmail.com>
---------
Signed-off-by: Prabinder Singh <prabindersinghh@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: Kubeflow SDK ROADMAP for 2026 (#326)
Signed-off-by: kramaranya <kramaranya15@gmail.com>
* chore: update AGENTS.md with lint commands and current repo map (#484)
- Add single-file lint/format/type-check commands for faster agent feedback loops
- Add kubeflow/spark module to repo map (SparkClient, kubernetes backend)
- Add kubeflow/hub/types and kubeflow/trainer/test directories to repo map
- Condense development principles to concise rules (remove verbose code examples)
- Fix typo in context awareness section
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
* feat: add kfp-client wrapper proposal (#343)
* feat: add kfp-client wrapper proposal
Signed-off-by: MStokluska <mstoklus@redhat.com>
* fix: address copilot comments
Signed-off-by: MStokluska <mstoklus@redhat.com>
* docs(proposals): refine KEP-125 PipelinesClient story split and run semantics
Signed-off-by: MStokluska <mstoklus@redhat.com>
* docs(proposals): remove trailing whitespace in KEP-125 README
Signed-off-by: MStokluska <mstoklus@redhat.com>
* docs(proposals): shorten the proposal and add kep.yaml
Signed-off-by: MStokluska <mstoklus@redhat.com>
* docs(proposal): address comments and upstream agreements
Signed-off-by: MStokluska <mstoklus@redhat.com>
* docs(proposal): address further comments
Signed-off-by: MStokluska <mstoklus@redhat.com>
* docs(proposal): address further comments regarding get_pipeline_version
Signed-off-by: MStokluska <mstoklus@redhat.com>
---------
Signed-off-by: MStokluska <mstoklus@redhat.com>
* chore: rename lifecycle/needs-triage to needs-triage in issue templates (#487)
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* feat(docs): add AI policy reference to contributing guide (#488)
Direct contributors to review the Kubeflow AI Policy before using AI
agents, aligning with the same note added to kubeflow/trainer.
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(optimizer): add comprehensive unit tests for KubernetesBackend (#485)
Cover all public methods of the optimizer's KubernetesBackend with
parametrized unit tests mirroring the trainer backend test structure.
Tests added for: optimize, get_job, list_jobs, get_job_logs,
get_best_results, wait_for_job_status, delete_job, get_job_events.
Each method tested with success paths and error scenarios
(TimeoutError, RuntimeError, ValueError) using mock K8s APIs.
Address review feedback:
- Add payload verification for create_namespaced_custom_object
- Add test_get_job_status_conditions for all status-mapping branches
- Add get_job_logs branch coverage (trial_name, follow, empty, pending pod)
- Restore polling_interval validation cases (zero, equal-to-timeout, negative)
- Rework wait_for_job_status with CR-based mock and callback test
- Fix docstrings, move TypeVar near imports, add positional-arg comments
Signed-off-by: Slowlybomb <hslyusar@redhat.com>
* feat(trainer): Support NPU labels in TrainJob device (#336)
Implement support for NPU resource labels in resource limit validation,
resolving the existing TODO to support additional accelerator types.
Signed-off-by: Sujal Shah <sujalshah28092004@gmail.com>
* feat(ci): Replace Trivy CVE scanning with uv audit + OSV-Scanner (#495)
Replaces the Trivy-based workflows removed in #427 with two new tools:
- validate-lockfile.yaml: PR check using uv audit to diff-compare
vulnerabilities between PR and base branch. Non-blocking (informational
comments only). Note: uv audit is experimental; if its text output
format changes, the fallback is switching to osv-scanner JSON output
(both query the same OSV.dev database).
- osv-scanner.yaml: Nightly scan using OSV-Scanner CLI (v2.3.8) with
SARIF upload to the GitHub Security tab and auto-fix PRs. Integrates
with existing .github/scripts/ utilities (update_overrides.py,
compare_versions.py, extract_version.py) and the cleanup-overrides
workflow for the full fix lifecycle.
- osv-scanner.toml: Minimal config for suppressing false positives.
Closes #478
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): fix --preview-features flag usage in uv audit (#509)
uv audit requires `--preview-features audit` (with a value), not
`--preview-features` as a bare flag. The bare flag causes exit code 2
(CLI usage error) on every run. Without the flag at all, uv audit
works but emits a warning to stderr.
Signed-off-by: Fiona-Waters <fiwaters6@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(deps): exclude kubernetes 36.0.0 (multiple client regressions) (#507)
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* fix: nightly security dependency updates (#497)
Co-authored-by: google-oss-prow <92114575+google-oss-prow@users.noreply.github.com>
* fix(deps): pin below kubernetes 36.0.0 (multiple client regressions) (#511)
Signed-off-by: tariq-hasan <mmtariquehsn@gmail.com>
* fix(docs): Fix light mode text visibility in custom CSS (#332)
* fix(docs): Fix light mode text visibility in custom CSS
Furo sets data-theme on body, not :root. Custom CSS used
:root:not([data-theme="light"]) so dark overrides always applied.
Switched to body-based selectors and added explicit
body[data-theme="light"] block for correct light-mode colors.
Fixes #331
Signed-off-by: Sridhar1030 <sridharpillai75@gmail.com>
* docs: add --kf-accent to all theme blocks for consistency
Signed-off-by: Sridhar1030 <sridharpillai75@gmail.com>
* fix(docs): address Copilot review - remove redundant light block, add comments
Remove duplicate body[data-theme="light"] block since :root values are
inherited when the media-query selector stops matching. Add explanatory
comment documenting why two dark blocks are needed (system-dark auto vs
explicit dark selection).
Signed-off-by: Sridhar1030 <sridharpillai75@gmail.com>
---------
Signed-off-by: Sridhar1030 <sridharpillai75@gmail.com>
* chore(ci): fix header comment loss in update_overrides on repeated runs (#503)
* chore(ci): fix header comment loss in update_overrides on repeated runs
Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>
* chore(ci): fix header reinsertion edge cases in update_overrides
Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>
* chore(ci): handle CRLF line endings in override block insertion
Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>
---------
Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>
* feat(hub): add upload-artifact utility (#514)
* feat(hub): add upload-artifact utility for #475
Add standalone S3/OCI upload helper and types so artifacts can be uploaded
without coupling to the model registry client.
Signed-off-by: myesmin <myesmin415@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(hub): align OCIUploadParams and _upload_to_oci with model-registry API
Replace OCIUploadParams fields (model_uri, author, model_description,
model_title) with base_image and oci_ref to match the actual
save_to_oci_registry signature in the model-registry package.
Add type: ignore[call-arg] on intentionally invalid Pydantic
instantiations in tests that ty was flagging.
Signed-off-by: myesmin <myesmin415@gmail.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Signed-off-by: myesmin <myesmin415@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): Fix git-cliff to generate changelog for patch releases (#527)
* fix(ci): Fix git-cliff to generate changelog for patch releases
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Add final message
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Fetch only upstream tags
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Remove git-cliff from deps
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Fix commit msg
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Fix PR names and fetch upstream commit
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Update RELEASE docs to align with Trainer
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Update RELEASE.md
Co-authored-by: Anya Kramar <kramaranya15@gmail.com>
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
---------
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: Anya Kramar <kramaranya15@gmail.com>
* Kubeflow SDK Official Release 0.4.1 (#532)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore(docs): update wait_for_run_status timeout default in KEP-125 (#535)
Signed-off-by: MStokluska <mstoklus@redhat.com>
* chore(trainer): Align list_runtimes docstring with namespace-scoped TrainingRuntime behavior (#305)
* Align docstring with current behavior
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
* updated for get_runtime
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
---------
Signed-off-by: Moeed Shaik <shaikmoeed@gmail.com>
Signed-off-by: Moeed <shaikmoeed@gmail.com>
* chore(docs): Move KEPs to the proposals directory (#538)
KEPs lived under docs/ even though they are project governance
artifacts rather than user-facing documentation, and were never part
of the Sphinx build. Move them to a top-level proposals/ directory to
mirror kubeflow/trainer#3629, update the contributor guide and
repository map, and fix a now-stale KEP link in the trainer backend.
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(docs): update wait_for_run_status to allow for indefinite wait, remove ERROR state and add system CA bundle support in KEP-125 (#539)
Signed-off-by: MStokluska <mstoklus@redhat.com>
* fix: nightly security dependency updates (#519)
Co-authored-by: google-oss-prow <92114575+google-oss-prow@users.noreply.github.com>
* chore(docs): Update release doc with milestone applier (#543)
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: Add ADOPTERS.md for Kubeflow SDK (#545)
Signed-off-by: Eder Ignatowicz <ignatowicz@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
* chore: rename SparkConnectInfo pod_name to driver_pod_name (#546)
* refactor: rename SparkConnectInfo pod_name to driver_pod_name
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Sameer_Yadav <159073326+Goku2099@users.noreply.github.com>
* minor fix
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
---------
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
Signed-off-by: Sameer_Yadav <159073326+Goku2099@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* feat: [KEP-107] Add long-running job submission proposal for SparkClient (#524)
* docs: update KEP-107 with review feedback
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* revised
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* updated
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* docs: address final review feedback for SparkClient KEP
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
---------
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* chore: improve Spark SDK documentation with connection and session management (#537)
* docs: improve Spark SDK documentation with connection and session management examples
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* refinement
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* chore: address review feedback
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
---------
Signed-off-by: Sameer_yadav <159073326+Goku2099@users.noreply.github.com>
* fix(docs): use relative URLs for nav bar logo and examples links (#553)
* fix(docs): use relative URLs for nav bar logo and examples links
Signed-off-by: Dmytro Hryshchenko <dhryshch@redhat.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: fix linter
Signed-off-by: Dmytro Hryshchenko <dhryshch@redhat.com>
---------
Signed-off-by: Dmytro Hryshchenko <dhryshch@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Trainer control-plane ConfigMap warning message (#558)
* Improve Trainer control-plane warning message
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Sumitha J <jsumitha012@gmail.com>
---------
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
Signed-off-by: Sumitha J <jsumitha012@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(docs): correct broken tune cross-reference in Quickstart (#557)
The "What's Next?" list in the Quickstart linked to `../tune/index`, but no
`tune/` docs directory exists: the hyperparameter-tuning docs live under
`optimize/` (referenced correctly by the top-level toctree and the Katib link
in index.rst). The stale reference produced a Sphinx "unknown document:
'../tune/index'" warning and rendered a dead link on the docs site.
Point the reference at `../optimize/index` so the link resolves. The bullet's
description ("Automatically tune hyperparameters") still matches, since that
page is titled "Hyperparameter Tuning".
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
* fix(spark): correct stale spark_version default in build_spark_connect_cr docstring (#556)
The build_spark_connect_cr docstring documented the spark_version default
as 3.4.1, but the applied default is constants.DEFAULT_SPARK_VERSION, which
was bumped to 4.0.1 to match the spark-operator image. Update the docstring
to 4.0.1 so it reflects the actual default. Documentation-only; no behavior
change. The default is already covered by test_minimal_cr.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
* fix(trainer): serialize TorchTune dtype/loss enums by value, add unit tests for get_args_using_torchtune_config and get_trainer_cr_from_builtin_trainer (#416)
* chore(trainer): add unit tests for get_args_using_torchtune_config and get_trainer_cr_from_builtin_trainer
Add comprehensive unit tests for two previously untested utility functions
in kubeflow/trainer/backends/kubernetes/utils.py:
- test_get_args_using_torchtune_config: 8 parametrized test cases covering
empty config, dtype only, all scalar fields (dtype/batch_size/epochs/loss),
LoRA peft_config, dataset_preprocess_config, HuggingFace dataset URIs
(directory and file paths via initializer), and invalid dtype error.
- test_get_trainer_cr_from_builtin_trainer: 4 parametrized test cases
covering num_nodes with batch_size, resources_per_node, empty config,
and invalid config type error.
These functions had zero test coverage. The new tests follow existing
patterns in the file using TestCase dataclass and pytest.mark.parametrize.
Signed-off-by: Prabhaharan Velu <haranprabha.v@gmail.com>
* fix(trainer): serialize TorchTune dtype/loss enums by value; address review
get_args_using_torchtune_config interpolated the DataType/Loss enum members
directly (e.g. "dtype=DataType.BF16"), so torchtune received the enum repr
instead of the intended value ("dtype=bf16",
"loss=torchtune.modules.loss.CEWithChunkedOutputLoss"). Use .value for both.
Tests:
- Assert literal produced strings for dtype/loss instead of reusing the
implementation's f-string, which made the assertions tautological.
- Add a nested-directory dataset URI case covering the data_dir branch with
a non-trivial relative path.
- Add a num_nodes=0 case pinning the `if config.num_nodes:` boundary.
- Add a case threading an initializer through get_trainer_cr_from_builtin_trainer
and asserting the dataset arg lands in trainer_cr.args.
Signed-off-by: Prabhaharan Velu <haranprabha.v@gmail.com>
* test(trainer): update builtin trainer backend test for dtype/loss value serialization
test_train's "valid flow with built in trainer" case asserted the old enum
repr (loss=Loss.CEWithChunkedOutputLoss). Update it to the value-serialized
form now produced by get_args_using_torchtune_config.
Signed-off-by: Prabhaharan Velu <haranprabha.v@gmail.com>
---------
Signed-off-by: Prabhaharan Velu <haranprabha.v@gmail.com>
* fix(trainer): validate HuggingFaceModelInitializer storage_uri has a repo path (#564)
HuggingFaceModelInitializer only checked the hf:// prefix, while its
sibling HuggingFaceDatasetInitializer also rejects a storage_uri whose
urlparse path is empty. A model storage_uri such as hf://model was
silently accepted and only failed later at download time.
Add the same repo path guard to the model initializer so an invalid
storage_uri raises ValueError at construction with a clear message. Add
parametrized tests for HuggingFaceModelInitializer, which had no prior
coverage.
Fixes #563
Signed-off-by: Sahil Kumar Singh <sahilkumargreat12@gmail.com>
* fix(trainer): validate HuggingFace storage_uri has both user and repo (#575)
The HuggingFace initializers only rejected a storage_uri with an empty
path, so URIs missing either component, such as hf://user/ (no repo) or
hf:///model (no user), were accepted and failed later at download time.
Consolidate the validation into a shared _validate_hf_storage_uri helper
used by both HuggingFaceModelInitializer and HuggingFaceDatasetInitializer,
requiring a non-empty user and repo. Add tests for the dataset initializer,
which had no prior coverage, and for the new rejected forms.
Follow-up to #564.
Signed-off-by: Sahil Kumar Singh <sahilkumargreat12@gmail.com>
* feat: add update_trainjob_status utility for progress reporting (#368)
* feat: add update_runtime_status utility for progress reporting
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* test: add unit tests for update_runtime_status
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* docs: add documentation for update_runtime_status utility
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* fix: align env var names with kubeflow/trainer#3227
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* fix(trainer): harden update_runtime_status for thread safety and edge cases
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* fix(trainer): fix token_path type hint and update progress reporting docs
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* fix: address review feedback on update_runtime_status
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* fix: pyproject file
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* refactor: move status utility to kubernetes backend and rename
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* refactor: rename estimated_time_remaining to estimated_remaining_seconds
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* refactor: address review feedback on update_trainjob_status
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
---------
Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com>
* fix(trainer): don't report Complete when all container statuses are Unknown (#562)
aggregate_status_from_containers() ran all() over statuses filtered to
exclude UNKNOWN. When the list is empty or every status is UNKNOWN the
filtered sequence is empty, so all([]) returns True and the job is
reported Complete. This is reachable via aggregate_container_statuses()
when a container cannot be inspected (get_container_status returns
UNKNOWN), so an unreadable job wrongly reports success to get_job(),
list_jobs(), wait_for_job_status() and get_job_logs().
Require at least one known status and all known statuses to be Complete
before returning Complete; empty or all-UNKNOWN now aggregate to Unknown.
Add unit test coverage for aggregate_status_from_containers.
Fixes #561
Signed-off-by: Sahil Kumar Singh <sahilkumargreat12@gmail.com>
* fix(trainer): Move TrainJob cleanup to finally block in get_runtime_packages (#582)
* Move TrainJob cleanup to finally block in get_runtime_packages
Signed-off-by: kingabdulsalamsas111-lgtm <kingabdulsalamsas111@gmail.com>
* Apply ruff formatting fixes
Signed-off-by: kingabdulsalamsas111-lgtm <kingabdulsalamsas111@gmail.com>
* test: verify temporary TrainJob cleanup in get_runtime_packages
Signed-off-by: kingabdulsalamsas111-lgtm <kingabdulsalamsas111@gmail.com>
* Address review comments
Signed-off-by: kingabdulsalamsas111-lgtm <kingabdulsalamsas111@gmail.com>
---------
Signed-off-by: kingabdulsalamsas111-lgtm <kingabdulsalamsas111@gmail.com>
* feat: add runtime kind support for TrainingRuntime and ClusterTrainingRuntime (#549)
* Add runtime kind support for TrainingRuntime and ClusterTrainingRuntime
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* refactor(trainer): replace runtime kind constants with enum
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* refactor(trainer): replace runtime kind constants with enum
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* Add get_runtime test for namespaced runtime preference
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* Add test case for namespaced TrainingRuntime preference
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* refactor(tests): move namespaced TrainingRuntime test case to appropriate section
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
---------
Signed-off-by: Sumitha007 <jsumitha012@gmail.com>
* feat: add PipelinesClient with kfp 2.17.0 (#591)
* feat(pipelines): add PipelinesClient re-export from kfp (KEP-125)
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
* docs(pipelines): add documentation and update status to Available
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
* fix(pipelines): simplify re-export and fix ImportError handling
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
* fix(pipelines): address coderabbit comments
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
* fix(pipelines): address comments
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
* fix(pipelines): fix pre-commit check
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
* fix(pipelines): use released kfp 2.17.0 instead of git master
Replace git-based kfp dependency with kfp[kubernetes]>=2.17.0 now that
kfp.kubeflow_client has been published to PyPI. Update README version
accordingly.
Signed-off-by: Esa Fazal <efazal@redhat.com>
* docs: update SDK diagram with Pipelines logo
Signed-off-by: Esa Fazal <efazal@redhat.com>
* fix: remove year from copyright boilerplate
Signed-off-by: Esa Fazal <efazal@redhat.com>
* docs: use correct KFP logo in SDK diagram
Signed-off-by: Esa Fazal <efazal@redhat.com>
---------
Signed-off-by: MStokluska <mstoklus@redhat.com>
Signed-off-by: Esa Fazal <efazal@redhat.com>
Co-authored-by: MStokluska <mstoklus@redhat.com>
* fix(trainer): validate polling_interval in wait_for_job_status (#554)
* fix(trainer): validate polling_interval in wait_for_job_status
Adds a ValueError guard in TrainerClient.wait_for_job_status() to reject non-positive polling_interval values before delegating to the backend. Previously this could cause a CPU busy-loop (polling_interval=0) or a cryptic stdlib error (negative values). Fixes #550
Signed-off-by: HarshPopat23 <musichk61@gmail.com>
* Potential fix for pull request finding add proper messages
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: HarshPopat23 <musichk61@gmail.com>
* test(trainer): update polling_interval test assertions to match new error messages
Signed-off-by: HarshPopat23 <musichk61@gmail.com>
* refactor(common): extract polling_interval/timeout validation to shared utils
Move polling_interval and timeout validation into a shared
common_utils.validate_wait_intervals() function, used by both
TrainerC…
What this does
Follow-up to #564. The HuggingFace initializers only rejected a
storage_uriwith an empty path, so URIs missing either component were still accepted and only failed later at download time:hf://user/(no repo name)hf:///model(no user name)HuggingFaceDatasetInitializeralso had no test coverage for its validation.The fix
Consolidate the validation into a shared
_validate_hf_storage_urihelper used by bothHuggingFaceModelInitializerandHuggingFaceDatasetInitializer, requiring a non-empty user and repo. Valid URIs such ashf://user/modelare unchanged.Testing
HuggingFaceDatasetInitializer(previously uncovered) and new rejected-form cases for both initializers.make test-python: 404 passed.make verify: all checks passed.