AWS Lambda Terraform Cookbook with working examples

Respect to the author.<p>As a guy who&#x27;s done a lot of programming and a lot of technical writing, it&#x27;s clear that this is the result of a TON of work. It is a model of clarity, well-formatted, and explained with just the right level of detail. It is completely pro quality and OP should be super proud of this body of work.<p>This isn&#x27;t just warmed-over Amazon docs. It&#x27;s just what you need when you can&#x27;t figure out what the docs are saying and you want to get something done now.

I do get the reasoning, but it&#x27;s still funny that we&#x27;re using an infrastructure management tool to manage the thing that was supposed to relieve you from the burden of infrastructure management.

The simplest HTTP API example is <a href="https:&#x2F;&#x2F;github.com&#x2F;nsriram&#x2F;lambda-the-terraform-way&#x2F;tree&#x2F;master&#x2F;samples&#x2F;12" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;nsriram&#x2F;lambda-the-terraform-way&#x2F;tree&#x2F;mas...</a> is about 66 LOC of terraform. But wait, it uses lambda-role (12 LOC), lambda (27 LOC), api-gateway (47 LOC) &amp; api-gateway-lambda-integration (40 LOC).<p>So 66+12+27+47+40=192 of terraform in a BUNCH of files.<p>Compare that to 47 LOC of verbose AWS SAM YAML: <a href="https:&#x2F;&#x2F;github.com&#x2F;kaihendry&#x2F;count&#x2F;blob&#x2F;sam&#x2F;template.yml" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;kaihendry&#x2F;count&#x2F;blob&#x2F;sam&#x2F;template.yml</a><p>All in one file.

I’m building data pipelines in AWS (s3&#x2F;sqs&#x2F;dynamo&#x2F;api gw&#x2F;lambda&#x2F;batch) + Snowflake.<p>Earlier this year I tried to use Terraform for everything, using principle “everything is a resource” (everything in my case is AWS, Datadog and Snowflake), so adopted “terraform apply” as universal deployment interface. Like if we need a ECR task and a Docker image, build the image from within Terraform (using null_resource which runs “docker build”). This approach works for everything but Lambda as Terraform requires a pointer to source code bundle at the plan stage. After unsuccessful fights I gave up for Lambda, so I build bundles prior to “terraform apply” (using “make build”, where build target does its magic of zipping either Go binary or Babashka Clojure sources).<p>That approach scales well for already two dozens of Lambdas and counting. Ping me if you want more details.<p>——<p>I disagree with this tutorial about tendency to use Terraform modules per AWS service, hiding well-documented AWS resources behind the facade of module with custom parameters with long names.

One of the striking things about serverless development that is less obvious from the outset is how it blurs the lines between application and infrastructure.<p>Deployment of a service is rarely in practice just deployment of new code to an already provisioned lambda - because that lambda can do nothing in isolation. Instead, it tends to be the lambda alongside an SQS queue and a trigger, and an S3 bucket; or an API Gateway that links an authorizer to the Lambda&#x27;s code. Because of that, evolution and development of the application tends to require evolution and development of those surrounding infrastructure pieces in tandem.<p>As a result, managing the infrastructure of your serverless service is often most naturally done alongside the application code itself - indeed, the distinction becomes somewhat meaningless. That also means the engineers developing the service require the ability to own and operate the infrastructure as well. That may or may not be well served by Terraform. It&#x27;s a tool I absolutely love for mutable, stateful infrastructure, but something like the Serverless Framework or AWS SAM can be a much lower-friction and more natural fit for serverless work.

Alex, OpenFaaS founder here. The author has done a huge amount of work here, I am surprised that it&#x27;s being given away for free, and not being monetized (it should be).<p>I often hear folks complain that Kubernetes is complex, and hard to understand. We&#x27;ve done a lot of work to make the experience of deploying functions simple on K8s, with very little to manage. But it still costs you a K8s cluster - most of our users don&#x27;t mind that, because they have one anyway.<p>But, for everyone else we created a new of openfaas called &quot;faasd&quot; which can run very well somewhere like DO or Hetzner for 3 EUR &#x2F; mo. It doesn&#x27;t include clustering, but is likely to be suitable for a lot of folks, who don&#x27;t want to get deep into IAM territory. <a href="https:&#x2F;&#x2F;github.com&#x2F;openfaas&#x2F;faasd" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;openfaas&#x2F;faasd</a><p>The REST API for OpenFaaS has a terraform provider that works with either version - <a href="https:&#x2F;&#x2F;github.com&#x2F;ewilde&#x2F;terraform-provider-openfaas" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ewilde&#x2F;terraform-provider-openfaas</a><p>And there&#x27;s a guide on setting up faasd with TLS on DigitalOcean, it took about 30 seconds to launch, and makes building functions much simpler than Lambda. &quot;faas-cli up&quot;<p><a href="https:&#x2F;&#x2F;www.openfaas.com&#x2F;blog&#x2F;faasd-tls-terraform&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.openfaas.com&#x2F;blog&#x2F;faasd-tls-terraform&#x2F;</a><p>If I were going to use Lambda, the author&#x27;s examples are probably what I would turn to, but there are other ways and it needn&#x27;t be this complex.

Lambda provides a particular challenge for Terraform. You don&#x27;t normally see Terraform used as a deployment tool for containerized services, even though it could theoretically do that. But because it&#x27;s the only thing close to the lambdas unless you want to introduce another third party tool, deployment ends up falling to it as well, unless you decide to choose another tool for the lambdas, like serverless or CloudFormation, and then you&#x27;ve got a bad build tool or a bad deployment tool for anything but the most trivial lambda builds.<p>And I will continue to be sad that all of the higher order first party tooling is ultimately going to be based on CloudFormation (looking at you, <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;proton&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;proton&#x2F;</a>).<p>Ultimately, after having used Terraform to manage function code bundling&#x2F;deployment and skipping Terraform completely for the lambdas, I think Terraform does best when it manages the infrastructure lifecycle for lambdas and nothing else. You can then rely on more competent tooling for deployment.

Personally I wouldn&#x27;t deploy the lambda code with the terraform. They inherently have different life cycles. In an ideal scenario you deploy some dummy code with terraform (just a hello world). And as a separate pipeline you deploy the actual code. Ideally, if your ci&#x2F;cd supports it, you have two separate pipelines, each one only does it&#x27;s thing if the relevant files have been edited, with the code depending on the terraform.

This is fantastic, I had tried to make terraform and lambda work together before and ended up abandoning that path and leaning on the serverless framework for that part of the project, but I was never happy about it being split out.<p>I look forward to trying this out the next time I want to prototype anything with some lightweight lambdas behind it.

For my team, I decided we would go all in on Terraform for AWS resources. Lambda has turned out to be a particularly tricky one to fit into that mold. It took us some time to sort out where the “build” step lived in our deployment pipeline so that terraform config pointed at the right build artifact.

kudos to OP for undertaking this but 10&#x2F;10 recommend Anto Babenko&#x27;s lambda module if you&#x27;re not already using it: <a href="https:&#x2F;&#x2F;github.com&#x2F;terraform-aws-modules&#x2F;terraform-aws-lambda" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;terraform-aws-modules&#x2F;terraform-aws-lambd...</a>

This is incredibly well written and comprehensive. It&#x27;s a gentle and friendly introduction to both Lambda and Terraform. What a great job.<p>I&#x27;ll just point out that if you&#x27;re using Python, Chalice is excellent and is able to emit Terraform code for all of its resources (<a href="https:&#x2F;&#x2F;aws.github.io&#x2F;chalice&#x2F;topics&#x2F;tf.html" rel="nofollow">https:&#x2F;&#x2F;aws.github.io&#x2F;chalice&#x2F;topics&#x2F;tf.html</a>).

I&#x27;m wondering if people would find it useful to see the cost of using opensource projects that spin-up resources in their AWS accounts before they run `terraform apply`? Or maybe a repo shield&#x2F;badge in the readme? (the idea came from <a href="https:&#x2F;&#x2F;github.com&#x2F;infracost&#x2F;infracost&#x2F;issues&#x2F;43" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;infracost&#x2F;infracost&#x2F;issues&#x2F;43</a>)<p>I&#x27;m not sure how it could work for usage-based resources like Lambda&#x2F;S3, maybe just assuming minimum usage for each resource is good enough to provide a rough monthly estimate? e.g. 1M Lambda requests, 1 GB storage and 1K S3 requests, then let users customize those numbers if they care to find out more?

I&#x27;ve played around with terraform and lambda in a web app context[1] and it works quiet nicely for creating per branch testing environments.<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;netsome&#x2F;djambda" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;netsome&#x2F;djambda</a>

I have personally have nothing against Terraform, and we’re using it for a lot of infrastructure-heavy things on our platform, but I think there are way better frameworks when deploying Lambda function.<p>Particularly the Serverless framework will save you A LOT of boilerplate IAC regarding all the event-driven integrations Lambda currently has to offer, and manages the full build and packaging cycle with the relevant language specific plugins, e.g. for Node or Python.<p>Others are the CDK, SAM or ARC.

IMO, i dont think terraform is the right tool for containerized services. I had experimented with terraform and ansible for deployments earlier but i could see simpler deployments using serverless or apex.<p>Informative article though.

At a quick glance, this looks way more complex than anything I&#x27;ve done with CloudFormation or AWS CDK or AWS SAM.<p>Not saying it&#x27;s bad, it just looks very different to those tools I mentioned.

Cool! Thank you for this.

The only reason I don&#x27;t use AWS is because of their terrible documentation and shady pricing strategy. It&#x27;s a pity.

Great tutorial. Thank for this.

I wouldn&#x27;t wish Terraform on my worst enemy.