How we built an auto-scalable Minecraft server for 1000+ players

Hi, I made this! Was gonna wait until it was downloadable and the documentation was more complete before posting it on HN. Thanks for sharing it though.<p>The Minecraft setup will be available and useable by anyone Wednesday of next week. More documentation and a roadmap will follow shortly after.

I&#x27;m always amazed by how so much Minecraft innovation is driven entirely by hobbyists. For years Minecraft was unplayable on many machines without the Optifine plugin. Even with a huge corporation behind it, Microsoft didn&#x27;t build the capacity for large-scale servers, and the very large paid servers seem to use region based sharding with portals. This project is truly exciting.

A couple of weeks ago I made <a href="https:&#x2F;&#x2F;playmcnow.com" rel="nofollow">https:&#x2F;&#x2F;playmcnow.com</a> with the opposite idea - have many super lightweight servers for small groups of players. Instead of saying to friends “register to digital ocean, pay $5 and get a Minecraft box”, I can host thousands of worlds for them.<p>I made a custom minecraft proxy (similar to bungeecord but A LOT less resource intensive) that starts the real server when someone attempts to connect to it.<p>200 lines of go for the proxy and about as much for the http front end in django :).

Impressive work! There&#x27;s a lot of potential here, but probably within a pretty limited market. Outside of 2b2t and maybe Hypixel, is there really anyone who wants 1000 players on the same world? Is the small market the only reason why a solution like this hasn&#x27;t been created before, or are there bigger complications that aren&#x27;t listed or haven&#x27;t come up yet, or maybe the technology wasn&#x27;t available? Surely someone like Hausemaster who stands to gain substantial amounts of money from this has looked into it before, right?

Hmm, I still think spatially allocated servers offer the best scalability. You would just have to approach it a little differently.<p>Server boundaries could move based on where the population is via delaunay triangulation (instead of fixed boundaries), and servers could share high-importance information with their immediate bordering neighbours. (This would be recognized as ghost data on the neighbouring servers.) You could even go further and have neighbours share ghost data with their other neighbours at a lower fidelity&#x2F;frequency.<p>A virtual distributed actor system could potentially be used to address any potential downtime or resource waste created by unpopulated zones.<p>I&#x27;ve been playing around with some of these ideas but haven&#x27;t been able to turn them into an actual implementation, so kudos to you for actually doing it!

Interesting timing.<p>I play with a few friends on a modded 1.7.10 server and we’ve decided to restart with 1.16 due to the lag having become untenable.<p>We run it on an i7 machine at my house dedicated to it, so it’s not a hardware issue.<p>Like clockwork the server would freeze for about 5s about every 30s. Using opus our best guess is that it’s unloading chunks and the GC is happening.<p>We’re going to run 1.16 now which I hope has some performance enhancements and so that we can use more modern Java 16 runtime with its nicer GC systems.<p>I’m also hoping that Minecraft has at least since moved chunk generation off the main thread since there was no good reason for world exploration slowing down things like it did.<p>I also built a JavaScript redstone simulator website and I’m curious how you will handle that and other block updates at server boundaries.

I used to play on a pretty popular MMO-style server called [CivCraft](<a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;Civcraft&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;Civcraft&#x2F;</a>) that unfortunately became a victim of its own success due to the large player count. It would often have 250+ players who would build these massive redstone machines and the server TPS would grind to a halt making it unplayable.<p>After a few years it attempted a re-launch using an approach similar to to the first one mentioned in the article. There were multiple worlds and as you approached the border of one world it would teleport you to another adjacent world. It was cool, but was jarring and suffered from its own complexity issues.<p>Anyway, this project is super cool. I would have loved to see something like it 10 years ago.

Being able to serve 1000+ players in a spatial &quot;MMO&quot; game is like the holy grail of netcode programming...<p>&gt; Here&#x27;s a demonstration showcasing 1000 cross-server players, this simulation is functionally identical to real player load. The server TPS never dips below 20 (perfect) and I&#x27;m running the whole thing on my laptop.<p>If it can run on one laptop, why does it need horizontal server scaling? :P<p>You don&#x27;t really know where the bottle-necks are until you put 1000 actual players on the same &quot;server&quot;.

I had the impression that most of the server&#x27;s cpu time is used for simulating the game&#x27;s content, not the players themselves. Most of the available cpu time is usually spent on monster AI and monster physics.<p>I assume that WorldQL is also used to store monsters besides the blocks, otherwise my understanding is that players cannot interact with monsters from other servers. Is it possible to create redstone circuits on different servers that then interact with each other?

Just to clarify, WorldQL is hosted-only software, right? No on-premise? If I tie my game to WorldQL, do I have any options if WorldQL goes under (other than re-engineering everything)? Not to diminish the impressive work on display here - I&#x27;ve just been wondering this since I saw your r&#x2F;gamedev post.

Interesting but this is kind of a big caveat.<p>&gt;redstone, hostile mob, and weapon support<p>Aren&#x27;t these the slow things? Is it just player position and map data that&#x27;s synced through a central DB? I would assume the bulk of the work isn&#x27;t done yet.

I&#x27;ve been running and doing minor plugin work on minecraft servers for years, getting close to a decade, and I&#x27;ve wanted this for so long. This would be a game changer for some modes I play.

Love the font on this site. Looks like you can find it here: <a href="https:&#x2F;&#x2F;brailleinstitute.org&#x2F;freefont" rel="nofollow">https:&#x2F;&#x2F;brailleinstitute.org&#x2F;freefont</a>

How did MMOs like Asheron&#x27;s Call and Wow handle this?

I’m curious to know how or if this solved any of the issues inherent with multithreading. Moving the threads to new processes can’t fix race conditions.

How many hours have gone into this?<p>Was this all on top of a 9 to 5?

Is there any chance of this supporting modded servers? Performance is universally the limiting factor on modded minecraft; if you&#x27;re not careful you can start having performance problems only thirty hours into a single-player world, and I&#x27;ve never seen a modded server support more than three or four players without severe lag setting in quickly.

This is nice magical software. Well done.

There are very few servers that would really benefit from this. But perhaps the reason is that it&#x27;s incredibly difficult to form large communities in minecraft.<p>I could see this really changing how users interact with each other.

Getting the two players in different servers to sync and see each other seems a headache. Instead can you not dynamicly change the border(s) think Gerrymandering.

FWIW: WorldQL is free for your first $50k in gross revenue.

That&#x27;s really freakin&#x27; neat. Something MC players have been dreaming about forever. I hope you&#x27;re successful and every server starts using it!

Really excellent work. Is this going to be open-sourced at some point?

I wonder how well this would run on a Raspberry Pi 4 cluster.

amazing work. i have been designing my own back end for an mmo in my free time off work. this will help me alot. much thanks for this.