Show HN: Simperium, a realtime data layer

<i>"With Simperium, you own your data - that's important!"</i><p>Not to take away from some of the great projects featured on HN recently, but statements by Simperium like this make me much happier as a developer and business owner: <i>"We believe the best apps have both a great user experience and unique backend services."</i>.<p>From <a href="https://simperium.com/overview/" rel="nofollow">https://simperium.com/overview/</a>: <i>It is not a backend-as-a-service. We believe the best apps have both a great user experience and unique backend services. Our focus is to provide a great data layer between your frontend and backend while integrating with other providers of tools, hosting, and services.</i>

This one is the best so far:<p>1. Works with any JavaScript library<p>2. You can <i>write server-side logic</i><p>3. Operational Transformation baked in<p>Seriously I cannot find anything bad about this project. (I really wanted to.)<p>My questions:<p>1. Is the server side written in Python?<p>2. Is there offline support for JavaScript apps too?<p>(I tried this with my own library and I think it was a wrong direction -- it only made it more complicated than it should be)<p>3. How about relationships in JavaScript objects? How does that work? (I suppose it's all automatic in Core Data objects, but how about JavaScript?)<p>Thanks!

We desperately need a high quality open-source Firebase/Simperium/Spire. A simple, robust, realtime data synchronization layer with libraries for major platforms. These projects all look great, but outsourcing this infrastructure is simply not feasible for lots of projects.

<i>Pricing is free during the beta period.</i><p>Simperium team: can you give any guidance on when you'll announce pricing, or roughly what pricing is going to be?

Thought it would be relevant here, I quickly hacked up a Ruby client: <a href="http://pheuter.github.com/Simperuby/" rel="nofollow">http://pheuter.github.com/Simperuby/</a>

This looks incredible. My one burning question: For integrations on mobile devices, how does this affect the battery life? I'm concerned that there is some open-ended connection from the client to server listening for data changes which will drain the battery while the app is open.

Absolutely priceless! This is an amazing product, and I can not start to imagine what developers are going to be able to do with this. Very exciting!

This almost seems unreal. What kind of latency would you be dealing with when your application(s) deal with many concurrent connections? For example, say a cross-platform multiplayer game with a persistent, shared world?

The website mentions that you are using google-diff-match-patch in the JavaScript client to merge changes, it seems as the API does not require the developer to actually specify things like "the user added an A at this position" as opposed to just "commit the changes to this entire object". Is there a reason other than simplicity for this API (I guess maybe because CoreData doesn't have that abstraction, and I presume your timeline was to start with figuring out how to sync CoreData), and on iOS are you also using google-diff-match-patch (there was not the same explicit mention of it in the documentation there)? (edit: Actually, I guess the comment from zbowling about DiffMatchPatch answers the second half of that. ;P)<p>BTW, this is generally really awesome: I am (right now, as in I'm sitting there right now ;P) helping teach a class on cloud computing at UCSB that happens to currently be discussing database synchronization and replication; after spending a bunch of time today discussing "how PostgreSQL is implemented and the basis of different isolation levels in the SQL standard and in MVCC" I took the time to tell everyone about Simperium (which probably makes more sense if I mention that I've looked into building something similar before for my projects; I'm glad someone else finally seems to be coming at it from the correct mindset). Everyone here seems to agree: this is going in a great direction.

Simperium dev here. We're launching our beta to gather feedback. Let us know what you think.

Guys congrats on the launch! I was wondering what was taking you so long ;) Very exciting stuff!<p>I'm amazed that we're on the same wavelength - we've had to build very similar infrastructure for ourselves for Unipost (www.post.fm). Can't believe we didn't collaborate on this, we'd happily be your first customer :(<p>A few interesting differences:<p>- Our approach is more like Meteor - web only, no iOS support<p>- The backend is a python tornado app that handles validation and conflict resolution before saving stuff to dynamodb<p>- We have a JS datastore backed by websql/indexeddb/memory that syncs with our backend datastore<p>- We have "live" Backbone collections that update themselves when datastore queries return different results<p>- We have a Backbone sync adapter that uses the datastore to persist data locally and kick off synchronization<p>- We sync a subset of the data (eg 3 months of mail) - thats a core requirement for us<p>- We sync all of the tables at once, not per bucket, cause queries are joining tables so the datastore has to be consistent at all times<p>- No operational transforms cause it doesn't seem to apply to us - pretty "notepad" specific I think<p>- No versioning as we didn't see benefits for us<p>- We'll probably open source this stuff when we're done<p>What do you use for storage?

This is interesting. We built something very much like this internally at SeatMe. On the iOS side, it's almost identical. It's how we keep our iPads up to date. Might not of built something internally ourselves a year ago if there was a platform like this.<p>The biggest difference is that since we are not a general platform, we can make assumptions about the model and how version each release and we can built in some constrains and unique security models.<p>We took a lot of cues from the OData spec and Microsoft's reference design.<p>The rest of this comment is mostly targeted at the creators of Simperium.<p>You made very similar design decisions to us in a lot of ways. A lot problems though that you will face I see with your path here so I have a few tips for you.<p>* It sucks the iOS client isn't open source. I get scared of linking in third party libs into iOS projects because I have to account for anything you do when I go to Apple to submit my app.<p>* You really got to brush up on the objective-c naming conventions. Not to be harsh but `-(id)getCustomObjectForKey:(NSString * )key;` makes me cringe.<p>* Don't require me to have to know about your categories.<p>* Namespace your categories so you don't smash mine ("something like "SP_encodeBase64WithString" instead of "encodeBase64WithString")<p>* If you include third party libs, you MUST rename them and prefix with your prefix. I see you use ASIHTTPRequest, DDLogger, SocketIoClient, AsyncSocket, Reachability, and a few others. You will smash everyone else's implementations if they already had them included.<p>* Don't use ASIHTTPRequest internally (it's old and unmaintained and doesn't play nicely with ARC)<p>* PREFIX ALL YOUR CODE. We don't have any real name-spacing objective-c. As a framework developer, you have to be aware of that more than anyone else. I shouldn't be seeing DiffMatchPatch and SocketIoClient show up in my symbol list after linking your lib<p>* Your addDelegate/removeDelegate is funny. After you exhausted the need for one delegate, switch to NSNotificationCenter.<p>* DON'T USE XIB/NIBs. Interface builder for iOS was an after thought and it's only a 90% solution (unlike with Cocoa where interface builder was a first class product built side by side with AppKit). Especially don't make me have to include your XIBs in my bundle. At the very least give me a bundle with it in it.<p>* Separate your GIT repo. If I want to include your library as a submodule I have to take all the client libraries as well.<p>Now when it comes to the actual sync layer and how you generate JSON dictionaries and apply "patches" this is fine code.<p>Here are some feature requests:<p>* Instead of having to give you a single NSManagedObjectContext let us register them. We have a few (some use different concurrency types).<p>* Let us override what gets generated or if we want to ignore a field with userInfo keys in the core data model.<p>* Let us get an idea of your backend sync processes to be able to suspend and start them when we need and know if anything is pending. Give us a callback that you still have things to queued and when we are done so we can at our leisure set up UIBackgroundTasks on iOS 4+.<p>On an unrelated note, why not create this as a NSIncrementalStore and just put your code behind the persistent store coordinator instead of monitoring it? We are doing the same as you at the high level because we wrote our code pre iOS 5.0 but iOS 5 gives you an awesome new toy there.

One thing that came up in the discussion about some other related offerings (such as Firebase) is the question: "what is your durability story"? When I'm spending the time to store things for my company, I make certain to ask such questions as "is this data important enough to keep on multiple servers, or is having a backup sufficient".<p>When storing data in services provided and managed by other companies, it becomes more difficult to be certain that the risk tradeoffs are reasonable, as I'm in essence trusting that you aren't just storing the data on a single server in RAM on memcache or something ;P. Some more details would thereby be much appreciated.<p>(I understand, btw, that I can also have a server keeping a mirror of all of my data, which is definitely awesome, but I also would then want to get a better understanding of whether I would be a fool not to be doing that, or whether I can feel comfortable with having data stored on your servers ;P.)

I've been a SimpleNote user for a while, great to see the platform layer finally coming out.<p>I have a few questions:<p>- As a potential customer, I'm curious as to how sustainable is the company. I understand there was a YC seed investment in '10, but is the company well funded for the next couple of years at least?<p>- The platform seems to be ahead of Meteor, Firebase, etc, in that it already seems to have implemented a basic login and security model based on expiring tokens, but from the docs it seems like "finer grained control of these permissions are under development". Does this mean that presently any user can erase/modify data from any bucket, such as global data, data from other users, etc? If so, that's a big deterred for me.<p>- Are there any plans to allow querying the data? Key-value stores are fine for simple games, to-do lists, etc, but for any non-trivial app querying arbitrary fields is a basic requirement.<p>Thanks again, those are some great strides in the right direction.

Is every object on a separate timeline for purposes of operational transformation and synchronization? As an example where this would be noticeable, if I make a change to one object, and then a different one, am I guaranteed that everyone will see the first change before the second? Put from the different side, is it possible for me to not have seen changes on one object yet, but end up downloading a fresh copy of a different object that is from a later point in time? Finally, related to that latter stating, is it possible to use the data before it has fully finished downloading, in that I might only initially need a subset of the objects (and is it even Simperium's goal to keep the entire data store synchronized, or just the active subset)?

Excellent product and superb demo.<p>Question that may be a bit off-topic but one I'd really like to know: what editor are you using there when you're editing the python service? Thanks and congratulations on your launch.

Something I don't understand: when your fundamental value proposition is moving data between platforms ("everywhere it's needed"), why didn't you consider your MVP to include an Android library?

That is awesome and looks 300 times easier to implement than iCloud! I wish inClass had Simperium to sync across any device and the web.

Looks awesome, quick question, is it possible to mutate data from a javascript client? In the API reference, all I see for javascript are event listeners. Also, how would a javascript client receive an access token? <a href="https://auth.simperium.com/" rel="nofollow">https://auth.simperium.com/</a> does not allow cross origin requests.

Are there any recommendations for the maximum size of a data set used with Simperium? It looks like the perfect solution for a back office app for a small business, but I'd like to know if I will be running into a brick wall at some point.<p>Clients will be iOS and Mac OS X only, using Core Data.<p>Thanks.<p>Brad

How many concurrent users will this support? I have a "school" database which could potentially benefit, but might have upwards of 500+ users adding and manipulating data at the same time.

What is the brown text editor used in the video? Is it only on Mac?

Mildly depressing that this is all proprietary.<p>I'd much much rather use something that's a hybrid of CouchDB and Unhosted...<p>It's definitely the future to build syncing as the core operation though.

I have no idea what this does from reading the homepage. You need use cases for where your product shines.

The github links on the samples page are broken (they have 'examples' instead of 'samples' in the path).

So, message queue as a service. With conflict handling. Interesting.

Please demo this to Evernote. They need to leverage this asap!