TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Is there something like s3 but with append?

1 pointsby dhbradshawover 6 years ago
We have a small stream of data -- perhaps 100 or so messages per second. It would be awesome to be able to divide that data up between a few s3 objects and append to them bit by bit. But of course s3 doesn&#x27;t allow appending.<p>Does anyone have suggestions for a service we can use in its place?

2 comments

dhbradshawover 6 years ago
Wow, I just realized that I asked this same question 76 days ago.<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=17903036" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=17903036</a><p>Suggestions include batching first with AWS Firehose and using multipart upload.
shshhdhsover 6 years ago
Maybe use SQS and then batch every X messages?