Maybe Everything Is a Coroutine

Not everything, but if by “coroutine” we mean a delimited continuation, then we get: exceptions, async&#x2F;await, generators, and even the IO monad.<p><a href="http:&#x2F;&#x2F;logic.cs.tsukuba.ac.jp&#x2F;~sat&#x2F;pdf&#x2F;tfp2020.pdf" rel="nofollow">http:&#x2F;&#x2F;logic.cs.tsukuba.ac.jp&#x2F;~sat&#x2F;pdf&#x2F;tfp2020.pdf</a><p>In short: algebraic effects.<p>Here’s a whole thesis on the cool things that can do done with this one simple trick: <a href="https:&#x2F;&#x2F;publikationen.uni-tuebingen.de&#x2F;xmlui&#x2F;bitstream&#x2F;handle&#x2F;10900&#x2F;102021&#x2F;thesis-v1.1.pdf?sequence=1&amp;isAllowed=y" rel="nofollow">https:&#x2F;&#x2F;publikationen.uni-tuebingen.de&#x2F;xmlui&#x2F;bitstream&#x2F;handl...</a>

Everything <i>can</i> be a coroutine (goroutine) in Go, very simple. Just add `go` before the function call. No need to do `async` and `await` or try to make sure it&#x27;s turtles all the way down by never calling a synchronous function from within a `async` function. In Go, everything within the function proceeds synchronously unless you use `<i>go</i>`.<p>It&#x27;s literally as simple as it sounds:<p><a href="https:&#x2F;&#x2F;gobyexample.com&#x2F;goroutines" rel="nofollow">https:&#x2F;&#x2F;gobyexample.com&#x2F;goroutines</a><p>Want to wait for a bunch of parallel coroutines to finish (sync) before proceeding? Sure, that&#x27;s easy too:<p><a href="https:&#x2F;&#x2F;gobyexample.com&#x2F;waitgroups" rel="nofollow">https:&#x2F;&#x2F;gobyexample.com&#x2F;waitgroups</a><p>This simple primitive is one of two major go features that made me not really pursue Rust more, simply because Rust&#x27;s concurrency libraries (it doesn&#x27;t seem to be an intrinsic part of the language like Go) seem to be more modeled after Python&#x27;s or Node&#x27;s. Go&#x27;s `go` is even more readable than Erlang and Scala, so in my opinion it&#x27;s one of the great three features of the language<p>(OT, but to me the other two major features are 1: channels being an integral part of the language, and 2: forced immediate error handling -- perhaps you disagree that this second one is a feature and prefer tracebacks, but in my experience large Go projects always seem to have cleaner code than, say, Python, probably because errors are handled very close to their origination. Lastly, there are a minimum of footguns in Go.)

Blasphemy!<p>Everything is a tables and SQL!<p>(hint: tables and queries implement content-addresable memory [1] and, consequently, dataflow architecture [2], which is also Turing complete)<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Content-addressable_memory" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Content-addressable_memory</a><p>[2] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Dataflow_architecture" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Dataflow_architecture</a>

I designed a syntax for this: that everything is a state machine progression, a bit like sequence types in the article.<p><pre><code> state1a state1b state1c | state2a state2b state2c | state3a state3b state3c </code></pre> This means wait for state1a, state1b state1c in any order, then move to the next sequence of things to wait for.<p>In a multithreaded server or multimachine distributed system, there are global states you want to wait for and then trigger behaviour. The communication can be inferred and optimised and scheduled.<p>It&#x27;s BNF syntax - inspired by parsing technology for parsing sequences of tokens but tokens represent events.<p>If you use printf debugging a lot, you know the progression of what you see is what happened and that helps you understand what went wrong. So why not write or generate the log of sequence of actions you want directly not worry about details?<p>But wait! There&#x27;s more. You can define movements between things.<p>So take an async&#x2F;await thread pool, this syntax defines an async&#x2F;await thread pool:<p><pre><code> next_free_thread(thread:2); task(task:A) thread(thread:1) assignment(task:A, thread:1) = running_on(task:A, thread:1) | paused(task:A, thread:1); running_on(task:A, thread:1) thread(thread:1) assignment(task:A, thread:1) thread_free(thread:next_free_thread) = fork(task:A, task:B) | send_task_to_thread(task:B, thread:next_free_thread) | running_on(task:B, thread:2) paused(task:A, thread:1) running_on(task:A, thread:1) assignment(task:B, thread:2) | { yield(task:B, returnvalue) | paused(task:B, thread:2) } { await(task:A, task:B, returnvalue) | paused(task:A, thread:1) } | send_returnvalue(task:B, task:A, returnvalue); </code></pre> Why not just write what you want to happen and then the computer works out how to schedule it and parallelize it?<p>I think iteration&#x2F;looping and state persistence and closures are all related.<p>I have a parser for this syntax and a multithreaded barrier runtime which I&#x27;m working on, I use liburing. I want to get to 500 million requests per second of the and ~50ish nanosecond latency of LMAX Disruptor.<p>The notation could be used for business programming and low level server programming I think.

This seems reminiscent of Session Types to me:<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Session_type" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Session_type</a><p>I think one difference is that session types capture which state a process&#x2F;co-routine is in after receiving a sequence of messages whereas this system does not, it captures how one can respond to each state in isolation (e.g. I don&#x27;t think you can statically capture that once a door is closed and locked you can only receive the locked state from it and respond by unlocking it).

For my game projects in Unity I have been using coroutines quite a bit instead of the normal types of state machines.<p>It&#x27;s a lot easier to read and write, BUT it comes with the downside that its not easy to save or load, which makes it only usable in certain game types and&#x2F;or actions.

This might have promise. I&#x27;d really like to see an example with a heterogeneous &quot;select&quot; construct, which is where a lot of these proposals fall down.<p>Go lets users do this only over channels, so the rest of the ecosystem has to bend everything into the shape of a channel, including timers, cancellation tokens, etc.<p>Rust generalizes this to any kind of future, though the ergonomics get rough in many cases, sometimes requiring pinning and sometimes risking dirty cancellation with no way to account for the resulting state.<p>Under this proposal:<p>How would you allow multiple coroutines to progress in parallel while selecting on the next update from each of them? When you&#x27;re selecting, you don&#x27;t yet have anything to send any of them, and when you get something back from one, you might have to send something to one or more of them, but they&#x27;re still working and are not yet in a state where they can receive.<p>How would you cancel coroutines while still allowing their implementation (not the caller&#x27;s) to decide how to get back to a known state? I can understand if the intention is that you always have to send them a signal to indicate this, but as above they have to be in a state where they can receive it, and it will still be extremely likely that at least one caller forgets to cancel at least one coroutine. Especially since the purity of this approach means it must work recursively as well.

The programming language Beta took this way further many years ago:<p><a href="https:&#x2F;&#x2F;beta.cs.au.dk" rel="nofollow">https:&#x2F;&#x2F;beta.cs.au.dk</a><p>It&#x27;s syntax didn&#x27;t help it&#x27;s spread unfortunately.

Every time I see a post like this and read the comments I&#x27;m more convinced that there&#x27;s a division between programming language enthusiasts and developers and only the former group cares about any of this.

This makes me think of dataflow languages where functions are be suspended until their (logic variable) arguments are sufficiently bound, then they fire, with this being a somewhat more imperative take on it.

<p><pre><code> door() -&gt; | :open(:close) | :closed(:open | :lock) | :locked(:unlock) </code></pre> What feels wrong to me in this example is that the type system doesn’t “know” that :close-ing an :open door will yield :closed. Expressed as a state diagram, it would only have a single node, whose type would be a sum type with three options, and the diagram doesn’t tell you which option you end up with for any given state transition, while at the same time you have to know the current option to know which transition you may take from the current state.<p>So there is an asymmetry regarding the associations to the three sub-states between the two ends of a state transition. Or in other words, the state transition arrows begin in three different states but end in a single state that is the opaque combination of the three source states.

I desperately think GPU programming(or specifically CUDA) needs some language level support like coroutine&#x2F;async&#x2F;await to organize the data flow and the executions among different dispatched device side function calls, and more on that to have some synchronize primitives between different blocks&#x2F;warps etc.

Wait, how do you run two coroutines in parallel? Like, how do I write zip() or something like it?<p><pre><code> a() -&gt; Int() | Void b() -&gt; Int() | Void for(a()) { | na -&gt; for(b()) { | nb -&gt; print(na + nb) &#x2F;&#x2F; now what? From here, I want to continue to a(), not b() </code></pre> I don&#x27;t believe there is much use for coroutines without support for their interleaved execution.

Someone on the internet thinks they have discovered El Dorado, only to discover, it&#x27;s called Erlang, and there are many people already inhabiting that delightful prairie, including many new Elixirian immigrants.<p>Virding&#x27;s First Rule of Programming:<p><pre><code> Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.</code></pre>

Isn&#x27;t this just iteratees? The examples look very similar to conduit code, unless I&#x27;m missing something.

Isn&#x27;t a language described very similar to the (future) OCaml with effects (<a href="https:&#x2F;&#x2F;github.com&#x2F;ocaml-multicore&#x2F;effects-examples">https:&#x2F;&#x2F;github.com&#x2F;ocaml-multicore&#x2F;effects-examples</a>) added?

Very OT:<p>First time that I learn of the .onl TLD. I thought it was a typo but it actually exists for 10 years already: <a href="https:&#x2F;&#x2F;icannwiki.org&#x2F;.onl" rel="nofollow">https:&#x2F;&#x2F;icannwiki.org&#x2F;.onl</a>

&quot;every function is a coroutine&quot; doesn&#x27;t make much sense to me. Notation in this area is a mess though and the author doesn&#x27;t define their terms - maybe they&#x27;re deliberately using coroutine to mean something unusual.<p>A function is some executable code. A closure is that plus some mutable state. A coroutine is usually a closure because something needs to track where it was suspended. Maybe this post is from the context of one of the languages that does the function colouring trick, where the answer is usually that said colouring is motivated by performance concerns.<p>There is a missing abstraction in the pthreads model - one can yield the current thread, but cannot yield to some specific other thread. I vaguely remember a patch set from google to add that to linux but am having trouble finding it now.

...or maybe we should go back to cooperative multitasking with stack switching and just add some familiar language syntax sugar on top ;)<p>IMHO the invisible switch-case code transform only makes sense when stack switching isn&#x27;t available (such as in Javascript or WASM).

This is why we keep having to learn new languages. Stop