InfiniCache: In-memory cache that is built atop ephemeral serverless functions

If distributed, lazy, just-in-time, global cache is what you want, Cloudflare throws it in free (with a few limitations) with Workers [0]. It is so much more straight forward than InfiniCache. I use Workers to cache content from few KBs to high MBs sourced from single-region S3 (could use Scaleway, Backblaze, or Wasabi for even more savings [1]) and DynamoDB (Workers KV can store 25MB [2] and is a viable alternative, but then cache may not be even required).<p>[0] <a href="https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;workers&#x2F;runtime-apis&#x2F;cache" rel="nofollow">https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;workers&#x2F;runtime-apis&#x2F;cache</a><p>[1] <a href="https:&#x2F;&#x2F;www.cloudflare.com&#x2F;bandwidth-alliance&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.cloudflare.com&#x2F;bandwidth-alliance&#x2F;</a><p>[2] <a href="https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;workers&#x2F;platform&#x2F;limits#kv-limits" rel="nofollow">https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;workers&#x2F;platform&#x2F;limits#kv...</a>

Interesting design, but it seems viable only by exploiting &quot;free&quot; cacheing given to AWS lambda functions. There are no guarantees here from Amazon, and I&#x27;m pretty sure whatever free capacity is there right now would evaporate once people started to exploit it more. AWS lambda should be more convenient than manual scaling, but never cheaper than hand-optimized service implementation.

Name reminds me of horrors of JBoss Infinispan cache

Very important caveats from the paper review at <a href="https:&#x2F;&#x2F;mikhail.io&#x2F;2020&#x2F;03&#x2F;infinicache-distributed-cache-on-aws-lambda&#x2F;" rel="nofollow">https:&#x2F;&#x2F;mikhail.io&#x2F;2020&#x2F;03&#x2F;infinicache-distributed-cache-on-...</a>:<p>&gt; Mind that the workload was selected to be a good match for InfiniCache properties (large objects with infrequent access).<p>&gt; The hourly cost increases monotonically with the access rate, and eventually overshoots ElastiCache when the access rate exceeds 312 K requests per hour (86 requests per second).<p>So this is not a replacement for the most common use cases for Elasticache. It is interesting as a cache in front of S3, but if you want a cache in front of S3...just use Cloudfront?<p>Honestly if I was AWS I would be ecstatic if people used this, AWS probably breaks even or maybe loses a little bit on optimal usage of this, and makes an absolute killing if someone using this has a traffic spike and ends up hitting their lambda millions of times.

So it uses serverless functions, but still requires EC2. “ deploy InfiniCache on a EC2 instance with powerful CPU and high bandwidth”<p>I don’t understand.

I like the hacking aspect of their solution. It&#x27;s always interesting to see products and services used creatively.

It&#x27;s always interesting to see terminology evolve. &quot;Serverless&quot; started as a hosting terminology, to mean &quot;somebody else&#x27;s servers&quot;. Here, it&#x27;s still being used to mean &quot;somebody else&#x27;s servers&quot;, but in a domain that could reasonably use &quot;serverless&quot; to mean &quot;without a server&quot; instead. Caching to a local hard drive is, almost by definition, serverless.<p>It&#x27;s a very neat application, but the terminology erks me.

Presentation (slides+video):<p><a href="https:&#x2F;&#x2F;www.usenix.org&#x2F;conference&#x2F;fast20&#x2F;presentation&#x2F;wang-ao" rel="nofollow">https:&#x2F;&#x2F;www.usenix.org&#x2F;conference&#x2F;fast20&#x2F;presentation&#x2F;wang-a...</a>

InfiniCache: Exploiting Ephemeral Serverless Functions to Build a Cost-Effective Memory Cache<p>The paper name fits the title better.<p>The keyword here is &#x27;Cost-Effective&#x27;.

Very creative, extremely complex, what a fun hack. TL;DR exploit the fact that AWS keeps your lambda process running (and in memory) beyond the time you are billed for in order to build an ephemeral cache. Creates one lambda per slice of cache (think redis slots) and provides a proxy to hide the (substantial) complexity from clients, keeping those lambdas warm, etc. I could see this being useful for cost hacking if the request volume was on the order of one request per key every few minutes.