The increase in cost of GPUs and AI is hitting an all-time high, so much so that it's hard to even get GPU usage approved within a public cloud environment. Because of that, the ability to splice/share GPUs across workloads that need GPUs (apps, Agents) is a necessity not only for cost savings, but for hardware/resource savings as GPUs aren't an unlimited resource (especially nowadays).
In this blog post, you'll learn how to implement CUDA (NVIDIA's parallel processing) inside an Agent Substrate Actor and see how many intermittent GPU workloads can take turns using a smaller pool of GPU-backed Substrate Workers.
Prerequisites
To follow along with this blog post from a hands-on perspective, you will need:
A GKE or local k8s cluster (Kind, minikube, microk8s, etc.) as the PodCertificate API needs the ability to be enabled on the k8s API server.
