Linux was the last to implement "events", they had IO completion ports and kqueue to learn from. They decided to implement something broken and document it as such, it was right in the man page. I find that astonishing. That reminds me, dnotify and inotify are probably still there. I'm bitter because I've had to port a clean kqueue event loop to Linux as a younger programmer and it was a dreadful experience.
My experience of epoll is that it's for single process, single thread network I/O mostly. If you want multithreaded network server use SO_REUSEPORT, and one epoll per thread. Everything else is either broken, overcomplicated or slow.
I have used select and poll before and sort of skipped to io_uring for a recent project, which also doesn't do too well with multiple threads (you have to use multiple rings and do everything else yourself). It's a shame that there is no obvious, relatively easy to use async IO mechanism on Linux that you can use from multiple threads without getting yourself in trouble. Reading more about kqueue it looks a lot like something that would solve it. Why was it not ported to Linux? Would it have been too hard to integrate?
The linked article is great btw. Very informative and concise!
I wrote an epoll echo server that multiplexes multiple network connections over threads (multiple users per thread)<p><a href="https://github.com/samsquire/epoll-server" rel="nofollow">https://github.com/samsquire/epoll-server</a><p>I also have a 1:M:N (1 scheduler thread, M kernel threads and N lightweight green threads) multithreaded userspace scheduler which multiplexes lightweight threads onto kernel threads and can preempt hot loops with minimal overhead. I rely on the fact that you can change the looping variable from another thread if you use a structure. Preemptive interruption is very useful for the illusion of multitasking. That's why I call it a userspace scheduler.<p><a href="https://GitHub.com/samsquire/preemptible-thread" rel="nofollow">https://GitHub.com/samsquire/preemptible-thread</a><p>I think the epoll-server which is kind of similar to what libuv does and the userspace scheduler could be combined into an application server.<p>I also wrote a multithreaded actor implementation in Java. Threads can communicate with each other between 60 million - 100 million messages a second. The epoll-server uses a multiconsumer multiproducer lockless RingBuffer.<p><a href="https://GitHub.com/samsquire/multicersion-concurrency-control" rel="nofollow">https://GitHub.com/samsquire/multicersion-concurrency-contro...</a><p>I think the core fundamentals of building a performant application server should be done once and reused for each application.<p>I want to also split the threading used by recv and send of a socket so that we have a 1:RecvKernelThread SendKernelThread with 1 RK+SK assigned to Socket:N scheduling (1 scheduler thread, 1 assigned Recv thread, 1 assigned send thread per socket). So you can send while you receive and receive while you send. True multiplexing!<p>We can decouple CPU and IO completely with threading.