TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Syncer: fast stateful file/disk data syncer written on Go

56 pointsby stargraveover 9 years ago

6 comments

josteinkover 9 years ago
I assume the title has a typo and that it&#x27;s supposed to say &quot;written <i>in</i> Go&quot;.<p>With that said, as a user of a tool&#x2F;library&#x2F;service, why should I care what language it&#x27;s written in?<p>I want to know if a data-syncing tool is reliable or not. I know rsync is, but OK that&#x27;s not always super-duper fast.<p>I know volume-streaming with btrfs and ZFS works wonders, without the need to do manual book-keeping of a secondary state and hoping you ALWAYS keep that secondary state up to date. I doubt this out-performs the FS just doing the job itself, and it sounds much more prone to user error.<p>So what does this offer over those? It sounds like it has less features and is not as reliable. Being written in Go offers me nothing I don&#x27;t already have.
评论 #10247260 未加载
评论 #10246910 未加载
评论 #10248602 未加载
评论 #10248113 未加载
评论 #10247016 未加载
评论 #10246947 未加载
kevanover 9 years ago
It&#x27;s late and maybe I missed something looking through the code, but what happens if you don&#x27;t have write permissions on the directory that the statefile is in? The copy operation is performed but then the state is lost when the statefile rename fails.
评论 #10246903 未加载
teteover 9 years ago
Suggestion: Don&#x27;t set a default for dst. That looks really dangerous.
评论 #10248587 未加载
mercoraover 9 years ago
A former colleague has written something quite similar[0] few years ago, which we used to backup encrypted disks (actually snapshots of encrypted disks to be precise).<p>[0] <a href="http:&#x2F;&#x2F;chunksync.florz.de&#x2F;" rel="nofollow">http:&#x2F;&#x2F;chunksync.florz.de&#x2F;</a>
gionnover 9 years ago
Nice, but in which use cases it may be useful?
评论 #10246893 未加载
perlpimpover 9 years ago
this is great. I wonder how it would perform when I need to sync 10mil. files. Rsync and anything else breaks down at this point, well it does work but it takes forever and rsync bloats to 10-20GB memory foot print and after a few hours starts streaming updates.
评论 #10248715 未加载