Infrastructure SaaS – a control plane first architecture

Hey - I don&#x27;t want to be critical, but I think you should rethink your messaging.<p>I&#x27;ll be honest with you: I thought this was a parody. It&#x27;s SO abstract.<p>It&#x27;s like you came from doing this abstract thing inside a big company &amp; decided to do the same abstract thing as a startup. And describe it using the specific terminology used inside that specific team of SuperBigCo.

For recent projects I&#x27;ve been using PostgREST + postgresql-replicant + Kafka (or some durable message queue as appropriate).<p>I don&#x27;t have to work too hard to implement the control plane as PostgREST takes care of generating the API and Postgres already has authorization controls built-in. Authentication is the part that requires a bit of toil to figure out. The rest is managing the schemas of the control plane entities. Basically, design the data and most of the rest is generated for me.<p>The data plane is trickier but I&#x27;m experimenting with using Postgres&#x27; streaming logical replication protocol to convert logical queries on the control plane data into business domain events that are forwarded onto the message queue. This is the part that uses the postgresql-replicant library I wrote but other libraries written in Python exist and can do the same thing.<p>This then enables me to implement business logic&#x2F;data-plane actions asynchronously down-stream as isolated, stateful services that follow the event streams and react accordingly to various policies. They can then update the control plane models as they progress, which then could add more domain events to the stream, and so on. It&#x27;s a bit like a functional-reactive architecture.<p>I don&#x27;t know if it&#x27;s a <i>production ready</i> style architecture. Monitoring replication stream performance can be tricky and integration testing is challenging. And managing changes to the business domain is not a fully-solved problem: still a lot of exploration&#x2F;tooling&#x2F;and do-it-yourself duct-taping to do.<p>But it&#x27;s simple enough that I can do lot of work with very little code so far.

“Data plane” and “control plane” aren’t terms I’ve seen before, and I’m having trouble understanding what they are, even after reading the post.<p>Can you explain them in a more concrete, conversational way?<p>Does this service let me e.g. take any docker image and turn it into a SaaS, handling user accounts and billing etc?

I work at a somewhat well-known unicorn in the data space that has been using this architecture for a while. In fact I&#x27;d wager that any (non-cloud-provider) company that provides any significant amount of compute or storage in their product offering will converge upon a layout that closely resembles this.<p>Overall I&#x27;d imagine there are a lot of parallels to other SaaS-ish architectures, one big divergence is that I&#x27;d consider the data-plane to be a special kind of client, (client in the same way that a user&#x27;s phone or browser is). The big difference is that we (the company) ALSO manage the lifecycle of this &quot;client&quot; (i.e. shutdown, startup, repair, update). Having an untrusted client that you also manage the lifecycle of can lead to some interesting design spaces.

I definitely agree with all the points here and the approach is correct, I&#x27;ve had the same thought before seeing project authors struggling to build cloud versions of their products.<p>But, this raises some important questions. You&#x27;re essentially outsourcing your entire company. Think about it, especially for an open source project, the main way you make money is build a cloud version of you product. If someone else is doing that for you, what are you left doing? It&#x27;s a dangerous place to be in.<p>What&#x27;s to stop the owner of this service to realize he can cut you out from the middle, and just build a service out of your product himself?<p>E.g. that&#x27;s a lot of AWS&#x27;s M.O. recently; they have really robust internal infrastructure to spin up managed services for any project, and they make a lot of money doing that.

Maintainers should be given the tools to monetise their projects using a cloud offering early on so they don&#x27;t burn out on the way to learning what control plane&#x2F;data plane nomenclature actually is!<p>Unfortunately there are limited tools&#x2F;resources out there to answer the question &quot;how can i build a cloud deployment option for my open source project&quot; without 25 layers of abstractions. This is why open-source projects end up raising millions of dollars (e.g strapi, appsmith just off the top of my head). All this money just for all these companies to essentially build the same thing.<p>Ideally, there should be a service&#x2F;tool (maybe thenile will be it) where I answer a few questions.<p>Containerised? Stateful? Long lived? Keep alive? Allow end users to deploy many instances? % premium per instance over cloud costs? License per instance? Pricing (tiered, volume, stairstep?) On end customer&#x27;s own infra? Min resources per instance. Airgapped per user&#x2F;airgapped per instance&#x2F;all instances running on same cloud? Instance management API? Update strategy? Big Green button to update all instances at once?<p>And it should spit out a ready to deploy setup where I can start monetising my open-source project while concentrating on maintaining the project.<p>If the above exists then congrats, you disrupted the main reason for open source projects raising millions of dollars.

I think [1] is a good definition of the architectural pattern (outside of networking applications). It’s the one we use within the crossplane community [2]<p>[1] <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;builders-library&#x2F;avoiding-overload-in-distributed-systems-by-putting-the-smaller-service-in-control&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;builders-library&#x2F;avoiding-overload-in...</a><p>[2] <a href="https:&#x2F;&#x2F;crossplane.io" rel="nofollow">https:&#x2F;&#x2F;crossplane.io</a>

Why is this specific to <i>infrastructure</i> SaaS? Most of what you write applies to any B2B SaaS, doesn&#x27;t it?

I&#x27;m also a bit confused by the terms used here. Is Replicated (<a href="https:&#x2F;&#x2F;www.replicated.com" rel="nofollow">https:&#x2F;&#x2F;www.replicated.com</a>) an example of the kind of &quot;control plane&quot; you&#x27;re referencing?