I'm curious how others store their photos. Right now I use Arq to back up to an AWS Glacier instance. I also back up to Shutterfly. And I use Time Machine to an external HD.<p>Problems:<p>0. I'm using a ton of local HD space to store them. I want to free up a bunch of space.<p>1. But if I delete photos from my computer, Arq will eventually delete the backups that contained them. Could be months or years but if they're not local, they'll be gone when the cost to store them exceeds the monthly budget I've set with Arq.<p>2. Additionally, Shutterfly apparently will let you upload videos up to 2GB. But I've randomly found a couple of videos well under that size that just don't get backed up.<p>I think this is what a good storage situation looks like:<p>1. Photos and videos are automatically backed up to the cloud from my devices (nice to have)<p>2. Photos are retained forever in the cloud even if they're deleted from my devices. I have to go to cloud storage to delete something.<p>3. Very nice to have: photos are encrypted locally before getting sent to the cloud like with Arq.<p>4. Bonus points: the UI for browsing cloud photos is nice (Shutterfly bought the company I was using, which had the best UI I'd found).<p>What do you use? Have you solved any of the problems I've laid out with my approach?<p>edit: thanks for all the great replies. I'm in the middle of a lengthy process of moving off of Google services, so Google Photos sounds great but isn't something I'm into. Will look at iCloud and other solutions mentioned below.
Google Photos has been my goto for a while now. The search feature is great and I use it frequently. For example, I was recently looking for a blue screen error code I'd taken a picture of months ago, and was able to find it within seconds by searching "monitor". It's really incredibly good, I'd say whatever model is on the backend is probably close to state-of-the-art. I recently hit the free limit, but for ~$3 a month I now have 100gb which I don't expect to reach for years to come.<p>I don't typically take photos of really private things, so the convenience of quick automatic backups from my phone and the availability of the search feature outweigh any privacy concerns I might otherwise have. I do also take local backups occasionally to my external hard drives in case Google somehow suffers major data loss.
I use iCloud Photo Library which solves most of your problems except client side encryption - which I agree is a big negative in its corner.<p>It works very well if you're all in in the Apple ecosystem.<p><a href="https://support.apple.com/en-us/HT204264" rel="nofollow">https://support.apple.com/en-us/HT204264</a>
I don't trust any service with plaintext of my photos so I self host everything. I also have a relatively huge amount of data so most cloud storage is out even if I wanted it.<p>My setup is a Linux-based NAS with LUKS on everything and each HDD in ZFS with RAIDZ2. I store my photos in their own ZFS filesystem on the zpool and use a tool that automatically creates snapshots and keeps varying resolutions for various timeframes (e.g. 24 hours of hourly snapshots, a week of daily snapshots, a month of weekly snapshots and I think 6 months of monthly snapshots).<p>As far as organizing them goes, I put my photos in date-based folders (e.g. 2018-12-09) and for larger ones go through and give them more useful names like "2018-12-09 - Brother's birthday".<p>I tend to edit them with Darktable the most these days, which also stores XMP sidecars alongside the photos.<p>I haven't yet set up a backup, which I know is horrible. When I do I'll likely go with either Wasabi or a NUC or something in a nearby datacenter.<p>For remote access I have Owncloud, though I _really_ want something nicer. Unfortunately there isn't much that works well with RAW files.
I use iCloud to store, backup, and sync my phone photos. I use a couple of 2/4TB drives to back up my photos taken on my DSLR/Film cameras. I manually copy over directories to redundant drives. I keep one 4TB drive that's the final backup, and it's unplugged and kept in a safe in my house. Every couple months, I pull it out, backup the newest photos to it and return it to it's safe. I know it's a little extreme, but I have tons of photos and want to keep multiple backups.
Google photos is the best I've found. Your privacy feature seems to be a "nice to have", so I would use that. The UX and cross-device syncing is far better than anything else I've used.<p>To your bonus points: Google photos web UI is truly impressive even ignoring the awesome search capabilities. They wrote an article[0] on how they built it and since then I've been keeping an eye out to use similar techniques in different applications.<p>[0]: <a href="https://medium.com/google-design/google-photos-45b714dfbed1" rel="nofollow">https://medium.com/google-design/google-photos-45b714dfbed1</a>
I have my photos and videos on a ZFS personal NAS at home, backed up independently to two external hard drives(secondary backup). The plan was to use recline to backup the ZFS NAS to Backblaze B2, but that bit is not setup yet.<p>I have also been eyeing a project called photoprism to serve as a front end for my media, and perhaps Filebrowser as well as a secondary interface to view them as "files".<p><a href="https://github.com/photoprism/photoprism" rel="nofollow">https://github.com/photoprism/photoprism</a><p><a href="https://filebrowser.github.io" rel="nofollow">https://filebrowser.github.io</a><p>Another project in the same "Google photos alternative" space that I have bookmarked is Ownphotos.<p><a href="https://github.com/hooram/ownphotos" rel="nofollow">https://github.com/hooram/ownphotos</a><p>Some day, I will find time to set it all up ;-)
I have the primary store on a RAID-Z2 zfs volume on a FreeNAS server in the basement. That backs up all photos to Amazon Photos ("free" with Amazon Prime, which I have anyway) and important things (family photos, family videos, and legal docs) to S3 with lifecycle policies defined.<p>Family photos are additionally rsynced nightly [one-way] to a Synology server (just because I had it before I built the FreeNAS storage and it has a reasonable picture browser app while on the home network).<p>zfs gives me copy-on-write (protection against crypto-locker, though I'd lose effective backup while crypto-locked) and availability even in the face of loss of 2 drives (RAID-Z2 ~= RAID-6).<p>I'm posting not because my solution is perfect, but because I'd like a better answer to client-side encryption for backup of legal docs, a better solution for video backup, and a better solution for browsing/tagging/auto-albuming accessible from mobile devices. I don't object to the current costs, though probably wished I'd skipped the Synology and went straight to FreeNAS.
I use a Synology NAS.<p>They've got that decent photo library app called Moments, which is available on mobile devices to upload your pictures from your phone directly.
They also have another app called Drive, on mobile and desktop to upload photos and files. It's basically a closed source Google drive.<p>The NAS itself can very easily be plugged to Amazon S3 for encrypted incremental backup (I do a weekly backups).
A while back, I've tried building a solution that does encrypted storage with locally managed keys, with a cloud backup.<p>Unfortunately, it did not go to production: the majority of people are ok with sharing their digital artifacts with google, dropbox, advertising and insurance companies, and any other entities who manage to have an agreement with of hack into the cloud provider. (See, for example, how many people talk about google photos in this thread). The market for a truly secure digital archive is, in my opinion, too small.<p>A prototype can be still seen at [0].<p>0: <a href="https://goryachev.com/products/secure-archive/index.html" rel="nofollow">https://goryachev.com/products/secure-archive/index.html</a>
Random iPhone snapshots stay in iCloud and end up on Google (when I remember to open the app). I don’t really care about them after sharing on social media.<p>I own a small photography business. Anything I really care about—almost everything captured on my DSLR or drone—starts life in Lightroom. After images are edited, and uploaded to appropriate online sites (e.g. Flickr), they are moved to an external 2TB drive.<p>The external drive is cloned to a separate drive to ensure I have a physical backup. My laptop and the primary external drive are backed up to CrashPlan.<p>It cost me $2700 to learn that one physical backup is not enough. When I bought my two drives a few years ago, they cost $99 each. That’s nothing relative to the value of my time/work.
In theory, iCloud is a really good solution. It is glitchy and sometimes slow to sync and the Photos app is very poorly integrated for use with other apps on the Mac (works much better in iOS). I have my entire 35,000+ library in iCloud with the original full-sized files uploaded and with optimized -for-size complete libraries on my Mac and iPhone. The 150GB library and my extensive hierarchical folder and album structure take up only 7GB on my phone. It's a neat system and will be great if/when they work out the many little annoyances and errors.
Unfortunately, Google Photos. It's very convenient, but there's this background fear that if my account gets unexpectedly closed, as it happens with Google, there would be no way to recover my photos.<p>I try to download the Photos data regularly via their takeout tool, but the process is cumbersome. I wish there was an automated way of doing that. I'd even pay for that.<p>I moved from Gmail to Fastmail, but I still haven't found a good replacement for Google Photos.
I just started using Lightroom CC with 1TB. It is a bit expensive when compared with 2TB of storage with Google and Apple.<p>But I like that it works across multiple ecosystems, Windows, MacOS, iOS, and Android (with bugs). And I wanted to use Adobe Portfolio for sharing some photos.<p>It meets your requirements for 1, 2, and 4.<p>Personally, I keep full resolution photos on external drive, and rsync those photos to NAS. Just in case there is bug in Lightroom and it deletes wrong photos or videos.
What long term editable “master” format do you use for videos? Most of mine have become unviewable over the years as video formats changed and support faded away, except some that were edited and the edits exported into crappy mpegs. I can export now into the hopefully longer-lived h.264 but is that reasonable for source files?<p>This is both a container and codec problem which just makes life even more complicated.
My data flow for phone photos is as follows:<p>1. capture photo with iPhone<p>2a. iPhone syncs with iCloud<p>2b. Now and then, open Dropbox App to upload newest photos to Dropbox<p>3. Synology NAS pulls photos from Dropbox<p>4. Synology backups photos to several backup locations using Hyper Backup.<p>For DSLR photos, I just upload the RAW files to the NAS and it also backups automatically.
Many people forget that their Amazon prime subscription includes unlimited* free photo storage (terms of use apply) I love it and it keeps getting better with each new app release. Worth checking out as an option.
Looking through other user's replies, all the above for me. Everywhere I can back up photos I do.<p>iCloud primarily but all the old family photos I have scanned, retouched also are uploaded to Ancestry.com, Facebook....<p>These are, as I say, family photos (a couple photos are scans from tintypes — so some quite old). There are no privacy issue for me (quite the opposite, I want these shared with far flung relatives). So I try to copy them as many places as I can think to. Hoping of course that future generations of the family will still be able to find them somewhere.
I store my photos in 4 places.<p>1. Laptop: Full quality. Storing and viewing purposes.<p>2. External HDD: Full quality. Storing and viewing purposes.<p>3. Amazon Glacier: Full quality. Storing purposes.<p>4. Google Photos: Reduced quality. Viewing and sharing purposes.<p>I name my photos as per the instructions in <a href="https://blog.budhajeewa.com/how-to-freely-and-properly-store-your-photos-in-google-drive/" rel="nofollow">https://blog.budhajeewa.com/how-to-freely-and-properly-store...</a> 's "1. Organize photos in your PC by albums" section.
I'm in a similar situation re: moving off Google services.<p>I use Mylio (<a href="https://mylio.com/" rel="nofollow">https://mylio.com/</a>), I think it hits most of your points.<p>Works great for managing my ~200k images. Syncs between my laptop/desktop/phones and also to Google/Amazon drive with optional encryption.<p>You can define which size of photos are synced to various devices, so my phones have thumbnails, my laptop has preview size photos, and my server/workstations have full copies.
Thirteen years ago, I ripped all my CDs to mp3 onto a new HD. Allowing me to eliminate a cubic yard of jewel-boxes. The HD was later copied to another, then stowed in a carboard box (kept indoors).<p>Twelve years later, I put the drive into a USB enclosure and copied the contents. The startup and copy went without a hitch, and so far no (audible) decay.<p>That might work for a large pictures collection. Certainly more trustworthy than existing physical media. (Although CDs I cut in that era are -mostly- still readable.
I use sftp+lftp [1] (faster and more secure than rsync) to transfer them to one of my backup servers. That server uses rsnapshot to keep diffs and also backs up to multiple external USB drives. The same method of sftp+lftp can be used to back files up off-site.<p>Sftp can't touch the snapshots which protects against malicious access (cryptolockers and such)<p>[1] - <a href="https://tinyvpn.org/sftp/#lftp" rel="nofollow">https://tinyvpn.org/sftp/#lftp</a>
Gsuite with unlimited data and google photos with original quality enabled. Photos are backed up once a month to aws in lower but still acceptable quality. (Might be changing that as storage costs are starting to get really brutal). Also generate quite a bit of video for a side project of mine and that also goes right to google drive in a team share. Don’t have the space or funds to store those elsewhere. Talking ~7tb of video and premier pro project files.
I prefer to keep my data on my own machines. I have a freenas server and use syncthing to synchronise photos between my phone and the server, with a cron script to organise the photos into folders.<p>I wrote a blog post about the setup a couple of years ago: <a href="https://www.jonblack.me/using-freenas-and-syncthing-to-automate-mobile-photo-organisation/" rel="nofollow">https://www.jonblack.me/using-freenas-and-syncthing-to-autom...</a>
I built a media pc with a 2tb drive for continuously uploading to backblaze, google photos, and an attached 2tb usb drive (it’s also a Plex server, which comes in handy for viewing the videos). I have google photos on my phones but mainly I periodically pull them off the phones and move them to the media pc, along with photos from my dslr.<p>Since my wife and I both have our own desktops it made sense to build the media pc to have a central place for photos and videos.
Crashplan $10/mo. Encrypted, off-site backup with unlimited storage, versioning and it keeps deleted files as well.<p>My challenge is that I shoot uncompressed RAW with my Sony A7R3 which makes it 80MB/image. And I shoot high res close to gigapixel res. panoramas with bracketing - which triples the storage (3 shots per frame for HDR blending).<p>So the challenge is it takes very long time to upload this stuff across my bandwidth.
Why not an external drive for your photos? I have a 1TB external drive dedicated for photos, and a 4TB external drive for Time Machine. My internal disk and the 1TB external disk both get backed up to the Time Machine disk and to Backblaze.<p>Backblaze supports encryption and I'm pretty sure you can access your backed up files in the cloud too, but I haven't tried that.
I think git annex assistant might work well for you.<p>I have a terabyte of photos and videos. I use git annex (without the assistant) to store and manage them. I push a copy to my desktop computer at work and to the NAS at home.<p>I'm not much of a believer in using the cloud for archival and backups. Hard drive space is much cheaper than the cloud, even with lots of redundancy.<p>I just purchased a cheap external drive which I'm going to use for offline backups in the fire safe.<p>I also have a friend who is willing to set up a backup-swap with me, but I need to sneaker-net the first backup to him.<p>Git annex will push to the cloud as well, I don't use that because I don't want to pay for the storage space.<p>For the photos on my phone I'm using resilio sync plus a cron job to copy them off and then free up space on the phone.<p>For organizing them I use a react/rails web app that I've been developing on and off for a few years. It's similar to google photos (without the AI). It's at <a href="https://github.com/jewel/hypercheese" rel="nofollow">https://github.com/jewel/hypercheese</a>.
Dropbox has treated me well. I think it hits all the boxes you're asking for. The app automatically backs up my photos from my phone, but because it's not photo specific, I'm able to backup all my other files (music, documents). I've also not hit a video limit.
All my pictures from my phone automatically go to Dropbox. I manually move pictures off my camera and put them in a folder that also gets sync on Dropbox.<p>Then everything on Dropbox is also on a physical drive.<p>As an extra redundancy, all of that is on backblaze.
I chose Flickr in addition to my main storage NAS at home. Flickr keeps things synced and I can share it with my folks, and some relatives and friends easily.<p>Google was too expensive (for no compression photos) for the bulk I have.
Also not the answers you're looking for;<p>i have my family use Google photos. I have old family videos on there, and most of have Google Android phones that have data storage boasts.<p>No privacy guarantees, few guarantees of never having a sunset.
>I'm using a ton of local HD space to store them. I want to free up a bunch of space.<p>How much space?<p>Consider:<p>1. That you're becoming a hoarder. Don't aim to keep all photos forever. Spend time to go over them and delete pointless ones. If you have 3-5 photos of the same object from the same angle "just in case", delete all but one.<p>2. HD space is cheap. Just upgrade. Don't use SSD for photos - get a regular 7200rpm HD. It's the simplest solution compared to trying to find a cloud provider that won't delete your photos. In the Internet age, you should <i>never</i> assume your online service will exist for long (paid or otherwise).<p>I have 170 GB of photos. Probably a third or so are simply because I have not yet had the time to go and prune them. I have a backup HD, and a cloud service. It should be all you need.
I’ve used Dropbox for several years for sharing files and such and found it also works great for photos. Pictures and videos from my devices (almost all from the iPhone, an excellent camera) are automatically uploaded to the Camera Uploads folder. Every couple of months I drag them over to a folder for the year, which I store inside a master Photos folder.<p>Only the current and maybe one more yearly folders are synced to my laptops, saves a ton of space. (Investigate selective sync on Dropbox)<p>I do have a gigantic hard drive on my desktop at home that has everything so I have one local copy of everything.<p>I also use Arq to periodically upload an encrypted version of the Photos folder from this desktop up to One Drive, since I get 1 TB free with my Office 365 subscription. Just for an emergency backup.<p>Hope this helps.
Dropbox and OneDrive only for the most important photos.<p>Storing all my photos online would require for me to get a very expensive plan. I do have offline storage of multiple external HDD where I regularly sync together.
I guess you probably don't want to hear "Take fewer photos, keep them on a HDD with a USB backup, and always be too busy doing awesome shit to look back at them?" That's far too ridiculous to be good advice. My entire life so far fits in 30GB of photos & videos. I rarely look at them. Life's too short. Everybody holding up a phone for the entire duration of the concert I'm trying to actually watch would probably beg to differ. Although then again I bet they do agree with me on one thing: they're never going to watch those videos either.
NFS(Wdmycloud) mounted to MacBook and raspberry pi. Currently the MacBook syncs the nfs storage with owncloud running on a docker host on linode.<p>When I have some free time I'm going to set up the pi to run the sync via cron so that when I copy files via my MacBook over to the NAS, I can close it and let the pi do the syncing when I am out of the house.
I use Dropbox to backup my photos. The Dropbox app on Android backups all photos in your gallery when given permission. I also dont have as many photos so the 1 TB limit on the pro account is more than sufficient for me.
Hum,
let me answer per points:<p>0. I may empathize BUT if you value data on them... Well you ask something like how I can have my wife drunk and my barrel full (ancient Italian proverb). Sometimes you can "optimize" your storage for instance lowering a bit quality of images that you may do not need at super-high resolution, stripping metatada, change formats, ... Depending on the case you may save big size of storage.<p>1. Why not having a more flexible backup then? RClone is a nice tool if you need to use someone else computer and you may use various vendors, also rsync+something you can mount as a local filesystem may also enlarge market offers list...<p>2. No proprietary tool can be consider trustworthy, no service will take care of your stuff better then yourself...<p>To answer more directly: I do not feel the need of buy someone else resources for storage, however consider a thing: local backups may not be super reliable, may be stolen, lost etc. however if you properly store and use these kind of event are not much more likely to happen than a "disaster in the cloud" especially from cheap services that in turn themselves re-sell someone else resources. IOW my own personal suggestion is assembly/buy a decent PC, with enough PCIe ports to support many sata port, fill it with reasonably big plate disks at the best price/Gb ratio you are able to find, install a GNU/Linux distro you prefer or pay someone to do so for you, I suggest using NixOS because being declarative if something goes wrong recovery will be quicker. Create a proper raid structure and use that as a personal cloud.<p>File transfer from various device may vary depending on the source device, from Syncthing to rsync+webdav you may have many options. It's automation again highly depend on "sources"...<p>To serve your content GNU MediaGoblin may be interesting, simply locally mount your server storage via nfs and use any local tool you want may also be an option.