Show HN: SadServers – Test your Linux troubleshooting skills

Based on your architecture diagram it looks like you&#x27;re spinning up an instance per-user? As you&#x27;re probably finding now, you will hit AWS limits quickly.<p>You might instead want to have a smaller pool of (larger) servers that you run co-resident VMs on with <a href="https:&#x2F;&#x2F;firecracker-microvm.github.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;firecracker-microvm.github.io&#x2F;</a>. That will avoid account limits and also keep your AWS costs more predictable.

I&#x27;d suggest integrating <a href="https:&#x2F;&#x2F;bellard.org&#x2F;jslinux&#x2F;" rel="nofollow">https:&#x2F;&#x2F;bellard.org&#x2F;jslinux&#x2F;</a> and running the VM in the browser if you can - then you can scale without running out of resources.

Very cool! This reminds me of the ops challenge @ Slack. I&#x27;m not sure if they still do this, but the SRE&#x2F;platform infra interview used to involve a VM running a malfunctioning LAMP stack.<p>You&#x27;d get SSH access to the VM, then submit a diagnostic report of what was broken (and how you fixed it).<p>Reminded me of how Red Hat used to run their certification test (RHCE). I probably still have the live CDs for my RHCE laying around somewhere.

New challenge: Fix SadServers’ sad servers

Seems like it&#x27;s out of capacity:<p><pre><code> An error occurred (VcpuLimitExceeded) when calling the RunInstances operation: You have requested more vCPU capacity than your current vCPU limit of 64 allows for the instance bucket that the specified instance type belongs to. Please visit http:&#x2F;&#x2F;aws.amazon.com&#x2F;contact-us&#x2F;ec2-request to request an adjustment to this limit. </code></pre> Maybe something like <a href="https:&#x2F;&#x2F;leaningtech.com&#x2F;webvm-server-less-x86-virtual-machines-in-the-browser&#x2F;" rel="nofollow">https:&#x2F;&#x2F;leaningtech.com&#x2F;webvm-server-less-x86-virtual-machin...</a> would be cheaper and more reliable for this kind of thing?

Very cool project!<p>I was the founder of a school training software engineers, we had an infrastructure track that got a lot of our students to land SRE positions. When asking employers for feedback about our grads, one feedback kept coming: they lack experience when it comes to troubleshooting.<p>So I went on a quest to simulate that infra debugging while in an academic context.<p>I came up with the idea of giving students broken servers. I used Docker container and would setup a simple workload and mess it up with classic issues.<p>Needless to say students generally did not like it :) debugging isn’t fun. But it did help a lot.

I&#x27;d love to get the actual VM content offline, packaged as Vagrantfiles or Containerfiles. Love the idea though! Go to Pluralsight and pitch it to them :)

The idea is really cool, but all I see is &quot;Waiting for server...&quot; and nothing happens.

I love this idea, I&#x27;ll definitely try it out when provisioning for scenario machines is up again. Nice work.

Hack The Box -&gt; Fix The Box

Really cool idea.<p>After choosing a problem, the endpoint you poll at <a href="https:&#x2F;&#x2F;sadservers.com&#x2F;celery-progress&#x2F;xxxx" rel="nofollow">https:&#x2F;&#x2F;sadservers.com&#x2F;celery-progress&#x2F;xxxx</a> repeatedly returns {pending: true, current: 0, total: 100, percent: 0} for me.

I&#x27;m assuming you&#x27;re spinning up an EC2 instance for each lab. What do you think about using pre-built docker images for each challenge instead? that way they can spin up in just a couple of seconds. Might also be cheaper?

Completed the first challenge and it was a lot of fun - <i>spoiler</i> I&#x27;ve never had to use the &#x27;lsof&#x27; command before.

Can I download the images so I could run it on my own machine ? I&#x27;d really appreciate, I&#x27;ve got an interview very soon :)

Commenting to give this a try later, I&#x27;ve routinely been the person to get these kinds of gremlins escalated<p>I&#x27;ve long wanted for some sort of mock, &quot;things are broken - I want to see how you think&quot; approach for sysad

Can&#x27;t get to the first problem because of HN hug but anyway there are fake ways to &quot;solve&quot; it like renaming the logfile (what they test for solved is provided).

Are you familiar with Trueability? <a href="https:&#x2F;&#x2F;www.trueability.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.trueability.com&#x2F;</a><p>It seems like this is a similar SaaS.

This looks interesting! But it keeps loading forever saying &quot;Your server is being created&quot; (hit VM limit again?)

&quot;Have you turned it off and on again?&quot;

Well this sucks I wanted to try it lol. It&#x27;s timing out for me or throws an error.

Interesting idea! Looking forward to trying this once some VMs are available. :-)

I only want to say that I love the name SadServers. Strongly memorable.

The tasks loading infinitely, is it a zero challenge?

did you read up on the problems with leetcode?

This is badass, just what I need!

&gt;Practice for your next SRE&#x2F;DevOps interview.<p>Are SREs and DevOps tasked with administration of operating systems?

My only feedback is that this is unrealistic because today developers wouldn’t try to debug something, they’d just destroy the instance, push a commit and hope it fixed something infra related then recreate it.<p>Why would you need to understand how something works? Just use containers. &#x2F;s

&gt; It&#x27;s also my not-so-secret hope that a sophisticated enough version of SadServers could be used by tech companies (or for companies that carry on job interviews on their behalf) to automate or facilitate the Linux troubleshooting interview section.<p>Yup, that&#x27;s what I was afraid of.

Cool!