Google employee here, xmanger is one of the main ML experiment tracking/orchestration tool we use internally, I'm pretty excited that it is now available for other to use!<p>In a nutshell, xmanager allows you to:<p>- define an experiment, which is a collection of one or more work units (think combination of hyperparamters)<p>- manage the different jobs/executable required to run this experiment (TPU workers, tensorboard job, etc.)<p>- collect and display measurements from work units (loss, other metrics)<p>- keep a reproducible artifact which allows you to re-run the same experiment at any point in the future<p>See e.g. <a href="https://github.com/deepmind/xmanager/blob/main/examples/" rel="nofollow">https://github.com/deepmind/xmanager/blob/main/examples/</a> for a few concrete examples of a launcher scripts.<p>I wish they had included screenshots of the tool itself in the repo, I'll make that suggestion :).
It's great this is open sourced. This technology was key to enabling ML folks to scale up computation without having to deal with borg and a bunch of other low-level systems.<p>It's one of the few systems in ML that I've used and thought "huh, this was well-designed and properly architected from the start"