Netflix has open-sourced its Maestro Workflow Orchestrator

This appears to be the high level architecture: <a href="https:&#x2F;&#x2F;miro.medium.com&#x2F;v2&#x2F;resize:fit:4800&#x2F;format:webp&#x2F;0*SDt718rSvgh2Nclv" rel="nofollow">https:&#x2F;&#x2F;miro.medium.com&#x2F;v2&#x2F;resize:fit:4800&#x2F;format:webp&#x2F;0*SDt...</a>.<p>To me seems to overengineered for most of the companies outside there, how many people do you need to manage it?

1) Scheduling seems to be primitive? (strategy: sequential) 2) That&#x27;s seems to be just a DAG executor? No high-level frontend? 3) No executiun context? 4) No typings? 5) No concept of a stream?<p>It would be way too ambitious to call this thing an orchestrator, it seems to be just a primitive distributed DAG executor.

[dupe]<p>More discussion on official post: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=41037745">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=41037745</a>

I was hoping to try this out, but it doesn&#x27;t build on my mac :&#x2F;<p>Here&#x27;s their tech blog post about it: <a href="https:&#x2F;&#x2F;netflixtechblog.com&#x2F;orchestrating-data-ml-workflows-at-scale-with-netflix-maestro-aaa2b41b800c" rel="nofollow">https:&#x2F;&#x2F;netflixtechblog.com&#x2F;orchestrating-data-ml-workflows-...</a>

So, is this something like prefect, dagster, flyte..?

After almost demise of nomad, market needs another general purpose orchestrator that isn&#x27;t Kubernetes or docker swarm.<p>This is not it however.

I was just thinking didn&#x27;t Netflix already have its own workflow engine: <a href="https:&#x2F;&#x2F;github.com&#x2F;Netflix&#x2F;conductor">https:&#x2F;&#x2F;github.com&#x2F;Netflix&#x2F;conductor</a>, but found out it&#x27;s archived&#x2F;EOL so this makes more sense now.

An abstraction on top of Conductor: <a href="https:&#x2F;&#x2F;github.com&#x2F;Netflix&#x2F;maestro&#x2F;blob&#x2F;main&#x2F;maestro-engine&#x2F;src&#x2F;main&#x2F;java&#x2F;com&#x2F;netflix&#x2F;conductor&#x2F;core&#x2F;execution&#x2F;MaestroWorkflowExecutor.java">https:&#x2F;&#x2F;github.com&#x2F;Netflix&#x2F;maestro&#x2F;blob&#x2F;main&#x2F;maestro-engine&#x2F;...</a>

Interesting that they use CockroachDB.