Skip to content

chore(gpu-health-monitor)!: remove end-of-life DCGM 3.x support - #2102

Draft
deesharma24 wants to merge 1 commit into
NVIDIA:mainfrom
deesharma24:chore/remove-dcgm3-support
Draft

deesharma24 wants to merge 1 commit into
NVIDIA:mainfrom
deesharma24:chore/remove-dcgm3-support

Conversation

@deesharma24

@deesharma24 deesharma24 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Removes DCGM 3.x support from GPU Health Monitor. DCGM 3.x is end-of-life: the 3.3.x branch gets no more patch or security releases (DCGM product lifecycle). The deprecation notice is in #2100.

Important

Merge on 30 November 2026, after that day's release. If you have any concerns, comment on this PR before then.

Changes

  • Helm: delete the gpu-health-monitor-dcgm-3.x DaemonSet and its chart tests. A node labelled nvsentinel.dgxc.nvidia.com/dcgm.version=3.x no longer gets a GPU Health Monitor pod.
  • Images: stop building and publishing the -dcgm-3.x image (Makefile, docker/Makefile, health-monitors/gpu-health-monitor/Makefile, container-build-test.yml, publish.yml, build-image-list.sh, check-image-attestations.sh).
  • Code: drop the DCGM 3.3.7 dcgmEngineRun fallback for the embedded hostengine. _run_dcgm_server now calls dcgmServerRun directly.
  • Dev and validation: remove the 3.x DaemonSet from tilt/Tiltfile and scripts/validate-nvsentinel.sh.
  • Docs: update the GPU Health Monitor configuration page, the labeler page, the DCGM failures runbook, DEVELOPMENT.md and RELEASE.md.

Compatibility

This is the breaking change that #2100 announces.

  • Kept: the nvsentinel.dgxc.nvidia.com/dcgm.version label and its 3.x value. Labeler still writes 3.x, so kubectl get nodes -l nvsentinel.dgxc.nvidia.com/dcgm.version=3.x still finds the affected nodes. No Helm value is renamed or removed.
  • Removed: the gpu-health-monitor-dcgm-3.x DaemonSet, the -dcgm-3.x image tag for new releases, and the DCGM 3 make targets (docker-build-dcgm3, docker-publish-dcgm3, image-dcgm3, publish-dcgm3, docker-gpu-health-monitor-dcgm3, build-gpu-health-monitor-dcgm3, publish-gpu-health-monitor-dcgm3). Images that earlier releases published stay available.

Type of Change

  • 🐛 Bug fix
  • ✨ New feature
  • 💥 Breaking change
  • 📚 Documentation
  • 🔧 Refactoring
  • 🔨 Build/CI

Component(s) Affected

  • Health Monitor
  • Core Service
  • Fault Management
  • Janitor
  • Deployment/Config
  • API/Interface
  • Preflight
  • Plugins
  • Documentation/CI
  • New Component
  • Multiple Components
  • Other

Testing

  • Tests pass locally
  • Manual testing completed
  • No breaking changes (or documented)

Checklist

  • Self-review completed
  • Documentation updated (if needed)
  • Ready for review

Signed-off-by: Deepak Sharma <deesharma@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

@deesharma24 this PR now has merge conflicts with main. Please rebase to resolve them.

@github-actions

Copy link
Copy Markdown
Contributor

@deesharma24 this PR has been inactive for 14 days. Do you need help finishing it, or should we close it for now? Feel free to reopen anytime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant