While there is a point in representing the state of the infrastructure as SQL to enforce type safety and other constraints, I still don’t understand what are the _real_ advantages over the current tooling. All the key advantages listed there, including type safety and cloud providers defined constraints, are already well covered by the actual battle-tested tools and frameworks.<p>Here are some areas where I identified some red flags regarding the present approach. These are key aspects of the everyday life of an infrastructure engineer.<p>Usability. The SQL abstractions simply don't scale to make the human<->machine mapping work. You would need another layer of abstraction to map that infrastructure into a state that is easily readable and understandable for our brain. On the other hand, the more consecrated tools use building blocks that both map well to be translated in infrastructure state _and_ are easily understood by our brain. Add some stored procedures to the party and you’re lost in the weeds.<p>Versioning. What would be the easiest way to extract from the SQL storage the difference between two versions of the infrastructure? With the traditional tools, that is an easy diff which reveals changes in a matter of several keystrokes with no extra layering.<p>Reusability. How easy is to port infrastructure code to other cloud providers? How easy is to define resource templates?
I see a lot of criticisms for not wanting to use SQL to do writes and I think that is misguided. The current state of your infrastructure is absolutely state and SQL is a great language for working with state. While Terraform and all these other "declarative" infrastructure tools are better than what came before them, you're ultimately playing Relation Stitcher by needing to connect the various pieces together. There is nothing declarative about Terraform and others. Infrasturcture is absolutely stateful and relational so why not use SQL and relations to manage it?<p>There are mentions of other tools that address the read side, and that's useful for obvious reasons, but you've punted on the hard problem which is the writes. The key to getting writes right will be constraints and triggers. Constraints can absolutely help operators to not cause outages by creating guard rails around certain state mutations. Triggers are important because unlike data that never sees an update, infrastructure is living and being able to consume those changes is important.<p>I might just have confirmation bias because I have this idea written down and think it should exist. Regardless, good luck!
Steampipe (<a href="https://steampipe.io" rel="nofollow">https://steampipe.io</a>) is an open source CLI to query cloud infrastructure using SQL (e.g. AWS, GitHub, Slack, k8s, etc). It also has a HCL language to define security benchmarks and controls (e.g. AWS CIS, etc).<p>We are a Postgres FDW under the hood with Go-based plugins, so write would be possible, but we've chosen to focus on read only so far. Definitely interested to see how you approach create, update and delete in the SQL model!<p>Notes: Not related to iasql. I'm a lead on the Steampipe project.
Interesting how it implemented under-the-hood. Does it use cloudquery (<a href="https://github.com/cloudquery/cloudquery" rel="nofollow">https://github.com/cloudquery/cloudquery</a>) or steampipe (<a href="https://github.com/turbot/steampipe" rel="nofollow">https://github.com/turbot/steampipe</a>) under-the-hood or does it implement everything from scratch.<p>Disclaimer: Im the founder of CloudQuery.<p>I get why you would want to do select * from infra, but not sure I understand why you would want to do "insert * into infra" and not use something like terraform? interested in hearing the use-case.
What problem does this solve, as opposed to a git repositories?<p>To me, declarative infrastructure management is incredibly important in order to reason about a deployment. Even though under the hood, yes, everything is stateful, it’s something I want abstracted away, not as a primary way of interacting with my infrastructure.<p>I guess what I’m asking is what the canonical use case / target user of this is.
I find the idea quite interesting - and it could be valuable just for the read-only aspect of it alone - especially if it is queryable through an ODBC or JDBC connection.<p>It helps avoid some scripting / token / API overhead - but I'm wondering what the trade off is in terms of initial setup time.<p>One general concern I have is around Data (state) consistency of the data after DML operations (insert / update). If the data represents the state of a resource - how do you know if there's a pending operation on it as a result of an earlier Update operation? How is idempotency handled? How are conflicting concurrent state changes handled (one user forces a restart, another initiates a shutdown)? What happens if a change applied to multiple records/resources only successfully applies on some of the resources?<p>This isn't specific to just this approach though - it's going to apply to any kind of cloud / resource management platform. I'm not sure how they tend to handle it - but a basic single record per resource has limitations around it that mean more work is needed. Maybe you end up with change history records?
I agree with rubiquity, without getting into a philosophical debate about programming paradigms, we have started an open source project InfraQL (<a href="https://infraql.io/" rel="nofollow">https://infraql.io/</a>) which is a framework for all cloud operations (query and provisioning) using SQL, we support SELECT, INSERT and DELETE with UPDATE, UPSERT, REPLACE coming, totally different architectural approach to the other solutions referenced in this thread, supports config/data supplied via Json or Jsonnet. If we look at cloud resources being defined by configuration, and configuration as (simply) data, then a SQL dialect makes absolute sense - as opposed to continuously inventing new and complex frameworks and DSLs.
I can see SELECTs being useful here, but yeesh INSERT/UPDATE/DELETEs would scare the heck out of me.<p>Also not clear on how/why this would be a SaaS product - this seems like it's a library you would download, throw some keys at, and thanks.
Seems to me whether you like this or not probably boils down to whether you like SQL or not.<p>For someone who spends most of their time writing type-safe functional code and loving the expressive power of it, the idea of voluntarily injecting SQL into my life seems horribly retrograde. We write whole frameworks to try and avoid having to manually code SQL(!) and every time I have to do it I sigh in frustration at how frustrating, verbose and repetitive it is.<p>But I'm guessing there's a whole crowd of people out there thinking "finally I can just use SQL and none of these ridiculous programming languages to manage my infra ..."
This is really interesting to me. I absolutely love SQL and the power it has for modeling complexity.<p>I also don't think this whole thing has to be absolutely pure either... I would have no problems seeing user-defined functions that have side-effects in this kind of scope. You can certainly wrap a declarative + gateway pattern around domain registration or other global one-time deals, but perhaps exposing that side-effect to the SQL user would encourage more robust application development. Exception handling is something that is very hard to perfectly abstract away in a declarative sense.
When will an ORM be available?<p>Still not sure whether this is serious or not, but it's not really infrastructure as SQL, it's infrastructure as database records which is stateful and defeats the point.
This has absolutely no relevance to me but that SQL snippet just bugs the hell out of me. I can't fathom why you would do a subquery in the from section and then not utilize a join. You're just bringing 2 datasets and letting them sit side by side, why not just move the subquery to the select section? Execution plan should result the same so I guess this is just a preference thing?
isnt the big advantage to infrastructure as code the fact that you can version control it? and isnt it notoriously difficult to version control SQL? maybe I am missing something
I feel like this has the most value for highly state full services. If your primitives are declarative, then this feels like a step backwards compared to CDK (I don’t have experience with terraform). As it turns out, composition and abstraction are really powerful tools for building complex infrastructures. For reading infrastructure state though, this sounds amazing.
Just imagine the Bobby Tables of iasql ...<p>In all seriousness, one issue I see in the comments here is that we're still not truly in the <i>mindset</i> of "infrastructure as code" mode.<p>If the existing infrastructure is so fragile and precious that DROP DATABASE is a non-starter as opposed to Chaos Engineering 101, then SQL paradigms are not the problem.
Why not just a text file? Is this the main reason?<p>> Unlike IaC, IaSQL makes the relations between pieces of your infrastructure first-class citizens, enforcing type safety on the data and changes to it.<p>It seems easier to solve that problem via a source-control hook to check a text file, than move over to SQL. But maybe this proposal gets you other things too?