NVIDIA GPU Accelerated Application Samples in Google Cloud Platform

Table Of Contents

• Description
• Prerequisites
• Samples
• Additional Resources
• Known issues
• License
• Maintainers

Description

This repository maintains sample applications designed for NVIDIA software tools integrated with Google Cloud Platform (GCP), e.g. AI platform, Dataproc, GKE, etc.

For select demonstrations, the sample code will be contained within this repository. For others, we will reference and link to exceptional demonstrations available outside of this repository.

Prerequisites

• Install Google Cloud SDK on your laptop/client workstation, so that gcloud SDK cli interface could be run on the client
• In addition, user could leverage Google Cloud shell

Samples

Samples are organized first by use case, then by Google Cloud service, then by NVIDIA library or stack.

Agentic

• Agent Platform / AIQ: Nemotron AIQ example

Inference

• Agent Platform / Triton: BERT fine tuning, TensorRT optimization, and Triton serving in Agent Platform Prediction
• Cloud Run / NIM: LLM NIM on Google Cloud Run
• Cloud Run / NIM: Cosmos Reason NIM on Google Cloud Run
• Cloud Run / NIM: Llama NIM on Google Cloud Run
• Cloud Run / NIM: Nemotron NIM on Google Cloud Run
• GKE / Dynamo: Dynamo VLLM disaggregated deployment
• GKE / Dynamo: Dynamo G4 Kimi K2.5 reference deployments
• GKE / NIM: LLM NIM on Google Kubernetes Engine
• GKE / NIM: LLM NIM on Google Kubernetes Engine with gcloud
• GKE / TAO and Triton: Building a computer vision service using NVIDIA NGC and Triton in Google Cloud
• GKE / Triton: Triton autoscaling example with TensorRT optimization in Google Kubernetes Engine
• Agent Platform / NIM: LLM NIM Python notebooks on Agent Platform
• Agent Platform / NIM: LLM NIM on Agent Platform Workbench
• Agent Platform / NIM: LLM NIM on Agent Platform Colab Enterprise
• Agent Platform / NIM: NeMo Retriever NIM on Agent Platform Workbench
• Agent Platform / Triton: Triton in Agent Platform Prediction
• Agent Platform / Triton: XGBoost ensemble inference with Triton

Training

• Agent Platform / RAPIDS: XGBoost with LocalCUDACluster Dask single node sample
• Agent Platform / RAPIDS: Multi-node XGBoost training sample
• Agent Platform / NeMo RL: NeMo RL GRPO quickstart

Data Processing

• BigQuery / RAPIDS: BigQuery analytics with Dask on GPU
• Dataflow / TensorFlow: BERT Q&A inference in Dataflow
• Dataflow / TensorFlow: T5 inference in Dataflow
• Dataflow / TensorRT: TensorRT BERT Q&A inference in Dataflow

Physical AI

• Compute Engine / Isaac Lab and Newton: Accelerate robot learning with NVIDIA Isaac Lab and Newton on Google Cloud

Industry Solutions

• GKE / RAG: NVIDIA RAG blueprint on Google Kubernetes Engine
• GKE / BioNeMo: BioNeMo generative virtual screening for drug discovery on Google Kubernetes Engine
• Compute Engine / VSS: Video Search and Summarization blueprint on Google Compute Engine

Additional Resources

See the following resources to learn more about NVIDIA NGC and GPU resources in Google Cloud Platform

Documentation

• GPU in Google Cloud Platform
• Optimize GPU Performance in Google Cloud Platform
• Getting started with NGC on Google Cloud Platform
• DL Frameworks GPU Performance Optimization Recommendations
• Multi-Instance GPU User Guide for A100 GPU

Known Issues

NA

Contributions

Contributions are welcome. Developers can contribute by opening a pull request and agreeing to the terms in CONTRIBUTING.MD.

License

See LICENSE.

Maintainers

• Fortuna Zhang (github: FortunaZhang)
• Dong Meng (github: mengdong)
• Rajan Arora (github: roarjn)
• Ethem Can (github: ethem-kinginthenorth)
• Arun Raman (github: arunraman)
• Juan Pablo Guerra (github: juanpabloguerra16)