Nvidia GPU on bare metal NixOS Kubernetes cluster explained

This looks fun. The author mentions machine learning workloads. What are typical machine learning use cases for a cluster of lower end GPUs?<p>While on that topic, why must large model inferencing be done on a single large GPU and&#x2F;or bank of memory rather than a cluster of them? Is there promise of being able to eventually run large models on clusters of weaker GPUs?

Ah, this is awesome! I currently run k3s on a decently spec-ed NixOS rig. I tried getting k3s to recognize my Nvidia GPU but was unsuccessful. I even used the small guide for getting GPU in k3s to work in nixpkgs[0], but without success.<p>For now I’m just using Docker’s Nvidia container runtime for containers that need GPU acceleration.<p>Will likely spend more time digging into your findings — hoping it results in me finding a solution to my setup!<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nixpkgs&#x2F;blob&#x2F;master&#x2F;pkgs&#x2F;applications&#x2F;networking&#x2F;cluster&#x2F;k3s&#x2F;README.md">https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nixpkgs&#x2F;blob&#x2F;master&#x2F;pkgs&#x2F;applicatio...</a>

All the links are styled in an unreadable form on that page.