From S3 to R2: An economic opportunity

Cloudflare has been attacking the S3 egress problem by creating Sippy: <a href="https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;r2&#x2F;data-migration&#x2F;sippy&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;r2&#x2F;data-migration&#x2F;sippy&#x2F;</a><p>It allows you to <i>incrementally</i> migrate off of providers like S3 and onto the egress-free Cloudflare R2. Very clever idea.<p>He calls R2 an undiscovered gem and IMO this is the gem&#x27;s undiscovered gem. (Understandable since Sippy is very new and still in beta)

Here’s a tweet from Corey Quinn describing how bonkers R2 pricing is:<p>&gt; let’s remember that the internet is 1-to-many. If 1 million people download that 1GB this month, my cost with @cloudflare R2 this way rounds up to 13¢. With @awscloud S3 it’s $59,247.52.<p><a href="https:&#x2F;&#x2F;x.com&#x2F;quinnypig&#x2F;status&#x2F;1443076111651401731?s=46" rel="nofollow noreferrer">https:&#x2F;&#x2F;x.com&#x2F;quinnypig&#x2F;status&#x2F;1443076111651401731?s=46</a>

As an indie dev, I recommend R2 highly. No egress is the killer feature. I started using R2 earlier this year for my AI transcription service TurboScribe (<a href="https:&#x2F;&#x2F;turboscribe.ai&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;turboscribe.ai&#x2F;</a>). Users upload audio&#x2F;video files directly to R2 buckets (sometimes many large, multi-GB files), which are then transferred to a compute provider for transcription. No vendor lock-in for my compute (ingress is free&#x2F;cheap pretty much everywhere) and I can easily move workloads across multiple providers. Users can even re-download their (again, potentially large) files with a simple signed R2 URL (again, no egress fees).<p>I&#x27;m also a Backblaze B2 customer, which I also highly recommend and has slightly different trade-offs (R2 is slightly faster in my experience, but B2 is 2-3x cheaper storage, so I use it mostly for backups other files that I&#x27;m likely to store a long time).

Is there any reason to <i>not</i> use R2 over a competing storage service? I already use Cloudflare for lots of other things, and don&#x27;t personally care all that much about the &quot;Cloudflare&#x27;s near-monopoly as a web intermediary is dangerous&quot; arguments or anything like that.

There are some dealbreakers to me.<p>1. No Object history and locking. So there is absolutely no way to recover files when you do any kinds of mistakes.<p>2. No object tiering and storage is not that cheap. Although R2 egress is free, R2 is only 35% cheaper than S3 in terms of storage, but it is not cheaper than other alternatives. Furthermore, R2 is a lot more expensive than S3 infrequent&#x2F;cold tier.<p>For example, Backblaze B2 is 4 times cheaper than S3, and B2 offers history&#x2F;locking. When B2 egress is free up to 3x monthly storage, B2 is much better option than R2 for most cases if a considerably high egress is not needed.

OP is missing that a correct implementation of Databricks or Snowflake will have those instances are running inside the same AWS region as the data. That&#x27;s not to say R2 isn&#x27;t an amazing product, but the egregious costs aren&#x27;t as high since egress is $0 on both sides.

Cloudflare has been building a micro-AWS&#x2F;Vercel competitor and I love it; i.e., serverless functions, queues, sqlite, kv store, object store (R2), etc.

We(Databend Labs) benchmarked #TPCH-SF100 on S3, Wasabi, Backblaze B2, and Cloudflare R2: #S3 leads with its direct connect feature. #Wasabi offers good performance. #B2 and #R2 may not be suitable for big data needs<p>Details: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;DatabendLabs&#x2F;status&#x2F;1719580350677237987" rel="nofollow noreferrer">https:&#x2F;&#x2F;twitter.com&#x2F;DatabendLabs&#x2F;status&#x2F;1719580350677237987</a>

The other hidden cost when you are working with data hosted on S3 is the LIST requests. Some of the data tools seem very chatty with S3, and you end up with thousands of them when you have small filed buried in folders with a not insignifcant cost. I need to dig into it more, but they are always up there towards the top of my AWS bills.

I wish the R2 access control was similar to S3 - able to issue keys with specific accesses to particular prefixes, and ability to delegate ability to create keys.<p>It currently feels a little limited and… bolted on to the Cloudflare UI.

I did some like-for-like comparisons across S3 vendors. S3 perf is way better than the challengers, R2 is the worst performer. Also it doesn&#x27;t support concurrency on list operations, or object versions. So it&#x27;s a bit more complex than &quot;R2 is best coz it&#x27;s cheapest&quot; it&#x27;s not super optimized yet<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;tomlarkworthy&#x2F;status&#x2F;1711846776905293967?t=rqaQXsuNWMsvcSTSG1m0Nw&amp;s=19" rel="nofollow noreferrer">https:&#x2F;&#x2F;twitter.com&#x2F;tomlarkworthy&#x2F;status&#x2F;1711846776905293967...</a>

We moved entire infrastructure to AWS last year, to speed up&#x2F;simplify&#x2F;rethink it. We lasted 3 months on S3&#x2F;CloudFront. We are still heavily invested in AWS, but moved our production storage&#x2F;distribution to R2&#x2F;Cloudflare and couldn&#x27;t be happier.<p>Next up: moving our cloud edge (NAT Gateways, WAF, etc) to Fortinet appliances, which licenses we purchased bundled with our on-prem infra.<p>I know Corey Quinn always harps on AWS&#x27; egress pricing but you really can&#x27;t emphasize it enough: it&#x27;s literally extortionary!

S3 and R2 aside, OVHs object storage offering is really robust and great. It performs better than S3 and is way cheaper, in storage and egress cost.

&gt;&gt; you’re paying anywhere from $0.05&#x2F;GB to $0.09&#x2F;GB for data transfer in us-east-1. At big data scale this adds up.<p>At small data scale this adds up.<p>And..... it&#x27;s 11 cents a GB from Australia and 15 cents a GB from Brazil.<p>If you have S3 facing the Internet a hacker can bankrupt your company in minutes with simple load testing application. Not even a hacker. A bug in a web page could do the same thing.

If you are storing large amount of data: E2 is the cheapest (20$&#x2F;TB&#x2F;year, 3x egress for free)<p>If you are having lots of egress: R2 is the cheapest (15$&#x2F;TB&#x2F;month, free egress)<p>R2 can get somewhat expensive if you have lots of mutations, which is not a typical use case for most.

&gt; In fact, there’s an opportunity to build entire companies that take advantage of this price differential and I expect we’ll see more and more of that happening.<p>Interesting. What sort of companies can take advantage of this?

I just love minio. It is a drop-in replacement for S3. I have never done a price comparison for TOC to S3 or R2, but I have a good backup story and run it all inside docker&#x2F;dokku so it is easy to recover.

The simple reason cloudflare hasn&#x27;t emerged as a real competitor is that they don&#x27;t offer traditional compute therefore you can&#x27;t just do what you normally would do in the hyperscalers in the clouflare regions. If they really are trying to be a fourth hyperscaler and&#x2F;or compete on price it feels like generql compute is what they need. What am I missing

R2 and Sippy solve a specific pipeline issue: Storage -&gt; CDN -&gt; Eyeball<p>The real issue is how that data get&#x27;s into S3 in the first place and what else you need to do with it.<p>S3 and DynamoDB are the real moats for AWS.

It blows my mind that anyone would consider S3 cheap.<p>You always had available plenty of space on dedicated servers for way cheaper before the cloud.<p>You could make an argument about the API being nicer than dealing with a linux server - but is AWS nice? I think it&#x27;s pretty awful and requires tons of (different, specific, non transferable) knowledge.<p>Hype, scalability buzzwords thrown around by startups with 1000 users and 1M contract with AWS.<p>Sure R2 is cheaper but it&#x27;s still not a low cost option. You are paying for a nice shiny service.

Interesting side note that while S3 the service continues to get more competition, S3 the protocol has definitively won. It&#x27;s a good protocol, but man I wish it were more consumer-friendly. Imagine if S3 specified an OAuth2 profile for granting access. Every web app could delegate storage to a bunch of competing storage providers.<p>This would be very useful in genomics, where pretty much everything is stored on S3 but always a pain to connect to apps.

My fear is that as R2 becomes more &#x27;discovered&#x27; and adopted, Cloudflare will hike prices since they&#x27;ll have a captive audience.

But what are you going to do with your data in R2? They don&#x27;t have all the other cloud services to use the data. Unless your only use of the cloud is literally for raw storage, it&#x27;s not that practical.<p>Compare with say Oracle cloud which tries to compete by having 1&#x2F;10th the egress charge. But nobody uses it anyway and they DO offer all the other services.

It looks like Backblaze B2 combined with Cloudflare gives the cheapest storage and free egress. Is there any reason to use R2 over B2 + Cloudflare?<p>My use case is image storage + serving for a service that users will upload a lot of images to. Currently using Cloudflare + storing all files on disk but space will soon become a concern.

The problem here is as long as cloud services are sticky, moving your data doesn’t really solve the vendor locking, egress is just one way to leverage that characteristic, I can easily come up with another ten way to charge you as long as you can’t not easily migrate your stacks off a cloud platform.

If I understand correctly when storing data to vanilla S3 (not their edge offering) the data live in a single zone&#x2F;datacenter right? While on R2 they could potentially be replicated in tens of locations. If that is true how can Cloudflare afford the storage cost with basically the same pricing?

For the Cloudflare fans out there (i am one of them), it seems that the sales&#x2F;finance guys entered the company and start to apply the usual upsell tricks. (See the advanced firewall and bots stuff)<p>Perhaps i’m too hasty with my judgement, hope so….

R2 is a nice and cheap service, I just want to caution people, it does have a reduced feature set than something more mature than S3 or GCS, for most people who just want to server an image etc, it&#x27;s fantastic though.

What would be great is a tiered storage service or library where oft-accessed data is in R2 and infrequently accessed has metadata in R2 but blobs in the cheaper S3 storage tiers or Glacier.

We absolutely love R2, especially when paired with Workers.

Did Cloudflare share any information on how R2 is built? Like what kind of open source systems they use as the foundation or they built it from scratch?

Should we simply ignore the tremendous amount of phishing hosted using r2.dev? Or is this also part of &quot;an economic opportunity&quot;?<p>Cloudflare may well be on their way to becoming a monopoly, but they certainly show they don&#x27;t care about abuse. Even if it weren&#x27;t a simple matter of principle, in case they aren&#x27;t successful in forcing themselves down everyone&#x27;s throats, I wouldn&#x27;t want to host anything on any service that hosts phishers and scammers without even a modicum of concern.

I read this and cannot believe that I can optimize our $5-6K GCP egress bill to zero. Just wow.

Backblaze B2 ftw!

Since I know there will be Cloudflare people reading this (hi!), I&#x27;m begging you: please wrestle control of the blob storage API standard from AWS.<p>AWS has zero interest in S3’s API being a universal standard for blob storage and you can tell from its design. What happens in practice is that everybody (including R2) implements some subset of the S3 API, so everyone ends up with a jagged API surface where developers can use a standard API library but then have to refer to docs of each S3-compatible vendor to see figure out whether the subset of the S3 API you need will be compatible with different vendors.<p>This makes it harder than it needs to be to make vendor-agnostic open source projects that are backed by blob storage, which would otherwise be an excellent lowest-common-denominator storage option.<p>Blob storage is the most underused cloud tech IMHO largely because of the lack of a standard blob storage API. Cloudflare is in the rare position where you have a fantastic S3 alternative that people love, and you would be doing the industry a huge service by standardizing the API.