TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Distributed Data Management Solutions?

4 pointsby traucoalmost 2 years ago
I run a small earth sciences research group. I have a need for a good distributed management solution so group members can access data, contribute changes, etc. The aim is to be able to track changes to both data processing scripts, civ-type data text files, and processed binary files (think MatLab .mat files or NetCDF).<p>I&#x27;ve been experimenting with DataLad (https:&#x2F;&#x2F;www.datalad.org) which is a front-end for git+git-annex. It&#x27;s ok although painfully slow as we have many small files to put into git-annex. It also requires quite granular understanding of how git-annex and git work, and it doesn&#x27;t have any nice front-ends.<p>Are there any other open source solutions for this out there?

1 comment

PaulHoulealmost 2 years ago
<a href="https:&#x2F;&#x2F;rucio.cern.ch&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;rucio.cern.ch&#x2F;</a>