> It’s hard to believe that as of 2016, the best method for offline storage in a web app was localStorage<p>IndexedDB has been around and usable for long enough that I used it on projects >3 years ago, with a fallback to localStorage just in case (looking at you, IE9). Apache Cordova even gave a nice little abstraction over sqlite so that using IndexedDB + phonegap was seamless and gave even more storage (IIRC). It was a pretty solid success for the data-heavy project I was on at the time. Load times the second time around were slashed to almost nothing. The app logic just needed to retrieve anything new since <latest timestamp> from the back end and update the UI.<p>Our experience was pretty great, though. I definitely highly recommend implementing some sort of domain-specific data caching layer with IndexedDB if you have the chance and if you're moving enough data to justify it. Just make sure you think through the update logic (e.g. use timestamps or log-structured data), and handle merge conflicts appropriately if necessary (i.e. if your app can be used offline).<p>There are plenty of great wrappers out there that simplify most use cases, too. We ended up going with localForage [1] to simplify refactoring away from localStorage.<p>[1] <a href="https://github.com/localForage/localForage" rel="nofollow">https://github.com/localForage/localForage</a>
Maybe it is a poor example, but I don't understand how IndexedDB and WebCrypto have anything to do with a distributed chat system like Slack.<p>For a system where I can join a channel at any time and see what's been said in the past, the information has to live on their servers and be readable. Even more so if you want to be able to search for something someone said across your entire Slack account. You aren't going to have all of that data local to your machine and browser.
For everybody starting to use indexedDB I recommend to take a look at Dexie.js (<a href="http://dexie.org/" rel="nofollow">http://dexie.org/</a>) for a minimalistic wrapper which might save you from a lot of pain as working directly with indexedDB is no fun imho.
Even worse, the pathetic 5mb limit is trivially bypassed, so it's really just an annoying but ineffective restriction:<p><a href="http://feross.org/fill-disk/" rel="nofollow">http://feross.org/fill-disk/</a>
This still doesn't explain how large scale search indexing could take place effectively... the data still needs to float upstream un-encrypted, processed and indexed.<p>Somehow offloading full-text indexing to the client, uploading encrypted indexes to the server and then elegantly combining those indexes across users to display a unified search would be arduous and I don't see anyone doing it.<p>Also, IndexedDB has been around for <i>years</i> and still suffers from a variety of cross-browser inconsistencies that make it a little painful to work with.
How on earth is SlackHQ's response reasonable? Let me count the ways it's not.<p>There are plenty of database systems (MySQL enterprise, Postgres, Oracle, SQL Server) that support encrypted data-at-rest and there are even more ways to run a DBMS on top of encrypted disks and volumes (dm-crypt, Bitlocker).<p>Running in the cloud? Amazon AWS provides EBS encryption (and likely more) and Microsoft Azure provides data at rest encryption for all storage types (tables, queues, page and block (disk) blobs) and transparent encryption for Azure SQL Database.<p>Want more control over the encryption in the cloud? Amazon and Azure both provide HSMs, called CloudHSM and Azure Key Vault, respectively. Azure supports additional encryption at the VM level integrating with Bitlocker and/or DM-Crypt.<p>There's really no excuse for not having figured out data encryption for customer data and secrets at this point.<p>Edit: It appears Slack has since updated their security docs and they now use encryption at rest. Good! It's inexplicable to me that anyone would believe that a database index would be incompatible with encryption.
Haven't read through the WebCrypto standard, but it's hard to create a real barrier between a server process and a client process on the web. In essence, any good crypto UI would have to happen outside of the client process, so that the server cannot intercept your input. (This is my objection to well-meaning companies that say "Hey, we can't even read your stuff, it's encrypted on the client" because, if you have auto-update, or if you ccan inject code into the client, that privacy guarantee can be revoked at any time.)
Having a local storage of this type is half the problem. The other half is syncing the data between different browsers and keeping it backed up. Imagine if all your mail, years and years of it, lived in IndexDB inside your browser on your one device.<p>I think browsers should implement a standard way to sync local storage, identity information (think private keys instead of passwords), cookies, etc. across all the browsers you use. Oh and obviously it should all be encrypted at rest and you get to choose which service you use to sync the data, so you can avoid the less trusted ones. What kind of utopia would this be?
The sad thing is, there was this idea for WebSQL ages ago and it would have solved this problem AND made database-backed apps easy to write, plus firefox and chrome already use sqlite internally so you could reuse that as the engine. Apparently it wasn't hip enough for the NoSQL folks though.
Assuming you are storing only encrypted data on the server and cannot read it without the user's password, what happens if the user loses their password? Are they just left with a bunch of unreadable data that came from the server?<p>Doesn't this approach also make it difficult to share information between other users?<p>Having offline-first, client side encrypted apps seem to have a lot more problems than just "you know who" spying on you.
More details about the 50% disk space limit: <a href="https://www.raymondcamden.com/2015/04/17/indexeddb-and-limits" rel="nofollow">https://www.raymondcamden.com/2015/04/17/indexeddb-and-limit...</a>
re: "But if the server could read your data, then practically any engineer of the company could read your data [...]"<p>A caveat: this tends to be true of startups. If you don't know who you're dealing with, it's probably a safe assumption to make. But larger companies often have internal permissions in place to prevent any engineer from reading user data.
Hey, I'm kind of a novice to web dev (especially when it comes to security) so could somebody please explain why you can't implement end-to-end encryption and search capability ?
How do others handle cross subdomain support of local data? IndexedDB and localStorage do not support subdomains best I can tell.<p>eg. mysite.com and app.mysite.com sharing local db.
A question - does it matter for webcrypto API's trustability to have window.crypto be redefinable to anything a piece of JavaScript code desires? (Ex: extensions)<p>I find it hard to make any claims about safe local encryption using webcrypto API if, post encryption, a new extension can be installed that will sniff the keys as you decrypt to access your data.<p>Thoughts?
This article is a very long write about nothing. Throws up topics and questions, and provides no explaination or answer.<p>An example of medium click bait that leads to medium's eventual demise. Being a platform for badly written PR bullshit will not work out for medium imho.
So after a bunch of work is done, I can have search on Slack like I have now. I understand it's better to have my chats encrypted at rest, but I wonder how many users are interested enough in that to make this change happen.
WebCrypto is security theater. The server could just swap the JS for a version that decrypts your secrets client-side and uploads the plaintext (or just uploads the key) to the server. Next time you hit the page your secrets are compromised.<p>Also, if you use "50% of the user’s disk space" you can go fuck yourself.