TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

S3cmd S3 sync memory usage solution?

1 pointsby jaddisonover 10 years ago
I&#x27;m sure I&#x27;m not the only one to encounter OOM (MemoryError) issues using `s3cmd` to sync files to AWS S3. I believe that the culprit is that the local&#x2F;remote file lists are stored in memory.<p>Has anyone got any novel solutions to this? As for fixing s3cmd itself, I&#x27;ve suggested the use of sqlite3 as a replacement for in-memory lists: https:&#x2F;&#x2F;github.com&#x2F;s3tools&#x2F;s3cmd&#x2F;issues&#x2F;408<p>A bad path? Any better options?

1 comment

jaddisonover 10 years ago
clicky-link: <a href="https://github.com/s3tools/s3cmd/issues/408" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;s3tools&#x2F;s3cmd&#x2F;issues&#x2F;408</a>