Show HN: Hamilton's UI – observability, lineage, and catalog for data pipelines

Hey HN – Stefan and Elijah here from DAGWorks (<a href="http:&#x2F;&#x2F;dagworks.io&#x2F;">http:&#x2F;&#x2F;dagworks.io&#x2F;</a>, YC W23).<p>If you don’t remember us from our previous HN launch (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35056903">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35056903</a>), we’re the authors of Hamilton (<a href="https:&#x2F;&#x2F;github.com&#x2F;dagworks-inc&#x2F;hamilton">https:&#x2F;&#x2F;github.com&#x2F;dagworks-inc&#x2F;hamilton</a>), an open-source library for building self-documenting, modular dataflows in python that works for data, ML, LLM pipelines, &amp; even web-workflows.<p>We’ve been developing this UI for a while and we’re excited to say we open-sourced it! It comes out of the box with the following capabilities, and only requires a single line code change to get:<p>1. Execution + metadata capture, e.g. automatic code profiling<p>2. Data&#x2F;artifact observability, e.g. summary statistics over dataframes, pydantic objects, etc...<p>3. Lineage &amp; provenance of data, e.g. quickly see what is upstream &amp; downstream of code&#x2F;data.<p>4. Asset&#x2F;transform catalog, e.g. search &amp; find if feature transforms&#x2F;metrics&#x2F;datasets&#x2F;models exist and where they’re used.<p>While the UI currently only self-populates for Hamilton dataflows, we’re looking to expand to other frameworks (we’d love your feedback!).<p>Check out the following video for an overview: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=0VIVSeN7Ij8" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=0VIVSeN7Ij8</a>, as well as the documentation: <a href="https:&#x2F;&#x2F;hamilton.dagworks.io&#x2F;en&#x2F;latest&#x2F;concepts&#x2F;ui&#x2F;">https:&#x2F;&#x2F;hamilton.dagworks.io&#x2F;en&#x2F;latest&#x2F;concepts&#x2F;ui&#x2F;</a>.<p>We’re looking for feedback&#x2F;adopters – feel free to reach out if you have any questions!