SSH and User-Mode IP WireGuard

This is such an interesting marketing strategy, I had never thought of selling B2B production infrastructure under the aesthetic of, “Can you believe this shit actually works?”

This sounds very similar to my slirpnetstack, which is using gvisor netstack to do, which I call translating L3 (packets) into L7 (userspace syscalls like connect()):<p><a href="https:&#x2F;&#x2F;github.com&#x2F;majek&#x2F;slirpnetstack&#x2F;" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;majek&#x2F;slirpnetstack&#x2F;</a><p>(btw, gvisor netstack, while not without problems, is likely to be faster than libslirp, see benchmarks <a href="https:&#x2F;&#x2F;github.com&#x2F;rootless-containers&#x2F;rootlesskit&#x2F;pull&#x2F;101#issuecomment-577453708" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;rootless-containers&#x2F;rootlesskit&#x2F;pull&#x2F;101#...</a> )

I added some example code to the post, because, again, I kind of can&#x27;t get over how easy this turns out to be. And if you follow the link into Jason&#x27;s `wireguard-go` code, until you hit gVisor itself, it&#x27;s not much more complicated under the hood.<p>Having complete control of TCP&#x2F;IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique.<p>The whole thing is kind of a vindication for Go&#x27;s standard library network interface, which I have always hated.

I know it&#x27;s not really an HN thing to say, but <i>this is just cool</i>. Reverse ssh tunnels on Wireguard through my VPN are cool enough; the amount of <i>magic</i> here (albeit I think perhaps not totally strictly required magic…) is definitely interesting++.

&gt; Normally, this big balloon thingy would be an elaborate scheme to get you to check out our product, but here it&#x27;s just pointing out some new source code we haven&#x27;t talked about elsewhere.<p>I really enjoy this style of writing from a company.<p>Regarding the article, it <i>seems</i> like Fly has pulled off some insane networking nonsense, but I don’t know enough about networking yet to understand it. Saving this page for later and gonna get back to the TCP&#x2F;IP Guide.

This is fantastic. I maintain a list[0] of tunneling software. One of the few downsides of WireGuard is the inability to run it in unprivileged situations. The complexity and performance overhead here might still be too much to edge out solutions like SSH tunnels, but I love that the space is being explored.<p>I&#x27;m hopeful we&#x27;ll also see some robust QUIC-based tunneling tools over the next couple years.<p>[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;anderspitman&#x2F;awesome-tunneling" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;anderspitman&#x2F;awesome-tunneling</a>

Not having been previously familiar with fly&#x27;s network setup, I gotta say I find it delightful; derived-prefix IPv6 + WG to give you basically static routing + ability to auth on IP is very elegant. I&#x27;ve actually been working on a toy stupid-simple clustering thing that does something similar, and I&#x27;m absolutely going to steal the userspace tcp stack over wireguard thing for API access.

Amazing. The client API is profoundly simple.<p>Also, this post prompted me to look closer at Fly.io, and it&#x27;s leapfrogged to the top of my shortlist for an imminent client &quot;edge proxy&quot; project.

Hacking stuff together using a userspace networking stack is an incredibly fun side project and significantly easier with the gVisor networking libraries written in Go.<p>Last year I implemented TCP&#x2F;IP over AWS Cloudwatch. Tons of &quot;can you believe that actually works?&quot; stuff possible with it:<p><a href="https:&#x2F;&#x2F;medium.com&#x2F;clog&#x2F;tcp-ip-over-amazon-cloudwatch-logs-c1cf08f2296c" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;clog&#x2F;tcp-ip-over-amazon-cloudwatch-logs-c...</a>

Running networking stacks in user mode really opens up a lot of interesting solutions. Wireguard is sort of an enabling technology for this.<p>Just realized this was written by security guru tptacek, nice. What is the contextual meaning of “AFFIANT SAYS NOTHING FURTHER.”?

Man, some people are just next level productive: &quot;How hard could it be to put together a tiny user-mode TCP, just for the purposes of doing pure-userland WireGuard networking, so people could SSH into instances on Fly without installing WireGuard? I made the mistake of musing about this on a Slack channel I share with Jason Donenfeld. I mused about it just before I went to bed. I woke up. Jason had implemented it, using gVisor, and made it part of the WireGuard library.&quot;

Is this super complex infrastructure for fairly simple thing or am I missing something?

Pretty cool write up. It mentions that every host is running a DNS server that instances have access to, which is being utilized to store the public key (neat!)... is there any way for customers to consume this for other purposes, say out of the box service (instance) discovery?

The prefix for ssh command looks good for commandline. However, is there a way to hide with some settings in .ssh&#x2F;config so one can have normal-looking &quot;ssh host&quot; cmdline without special prefixes?

I read this but I didn&#x27;t <i>get it</i> at all. I can&#x27;t see the forest for all the excited talk about particular trees. In simple words, what problem are they trying to solve?

Isn&#x27;t this bad for privacy? Encoding app, org and such information in IP address?

&gt; We take Docker-type containers from users and transmogrify them into Firecracker micro-VMs<p>What is the relationship with micro kernels? Is the feature available separate from the deployment&#x2F;hosting?

<i>&gt; I’ve written a bunch about private networking at Fly. Long story short: it’s like a simpler, IPv6 version of GCP or AWS “Virtual Private Clouds”; we call it “6PN”. When an app instance (a Firecracker micro-VM) is started at Fly, we assign it a special IPv6 prefix; the prefix encodes the app’s ID, the ID of its organization, and an identifier for the Fly hardware it’s running on. We use a tiny bit of eBPF code to statically route those IPv6 packets along our internal WireGuard mesh, and to make sure that customers can’t hop into different organizations.</i><p>My first thought was <i>&quot;Wow, can we make this _more_ complicated please?&quot;</i>, and then I read the rest of the post.<p>I hate technology.

Sounds very cool and all but at the same time it sounds like a terrible thing to maintain in the future.<p>Perhaps it&#x27;s just me, but this is something I would accept as a &quot;hey, I was bored and worked on something on my free time. It&#x27;s probably broken but nobody cares because it&#x27;s a toy thing, but it&#x27;s sooo cool&quot;. I wouldn&#x27;t accept it as &quot; Fly.io OKR 1.3 (2021): SSH and User-mode iP WireGuard&quot;... it&#x27;s sounds pretty much like a hack.

Nice work, I love WireGuard, what it needs is more recognition and definitely more integrations like this.

Off-topic: What&#x27;s the software used for the blog?

Maybe you should put this up on github so everyone can use it rather than just talking about how easy it is?