This looks pretty interesting.<p>Just curious if you looked at Apache Airflow as the integration functionality and pipeline / DAG configuration approaches are similar and it's also done in Python.<p><a href="https://github.com/apache/airflow" rel="nofollow">https://github.com/apache/airflow</a><p>The Nim language is new to me. What was the experience like implementing a framework in it?
I'm not sure I understand the wins of this over just using the standard multiprocessing module.<p>You can trivially define a worker pool or arbitrary size and pipe a stream of values through your function using multiple cores, avoiding GIL issues.<p>If you have something which scales to multiple machines just use celery or redis streams.
Nice readme! It's so common for projects to forget to include an introductory blurb, it's great to see one that does not forget.<p>Small bug in the example code? I think this:<p><pre><code> return even % 2 == 0
</code></pre>
Should be this:<p><pre><code> return number % 2 == 0</code></pre>
I didn't understand one thing: is this implemented in Nim and run Python code? If yes, why don't run Pipelines in Nim too? Wouldn't this be faster?
Should have named it Pypelines. But seriously though I am not sure what the benefits over multiprocessing are. It looks a little cryptic at first sight.