We are incredibly open to any ideas, comments or concerns on how we're doing this. This is a big step up from what we had previously, but there’s always room for improvement. Happy to hear thoughts in the comments.
Why all the trouble with hashes - can't you just do it on the client and not having to store any data at all?<p>"For tracking unique page views"<p><pre><code> if(!sessionStorage[location.href]) {
sessionStorage[location.href]=1;
navigator.sendBeacon("/unique-pagehit?" + encodeURIComponent(location.href));
}
</code></pre>
"For tracking unique site views"<p><pre><code> if(!sessionStorage["Hi!"]) {
sessionStorage["Hi!"]=1;
navigator.sendBeacon("/unique-sitehit");
}
</code></pre>
"For tracking previous requests"<p>I'm not sure I fully understand what is being measured (is it session-only?). For the duration someone watched a page, you can use sendBeacon in onBeforeUnload. To detect a bounce, set a Math.random() in a session variable, send it at the start of the page, and have every page load send the previously stored random variable. Then count the unique random keys you received on the server - those are the bounces.<p>I know, in practice you'll need to trim sessionStorage, sanitize URLs, use something less-colliding than Math.random, dealing with new tabs, some polyfills and other robustness, etc... but I don't yet see why the tracking mentioned needs any user ids or hashing at all.
As a user of the open source version of Fathom, I'm a little concerned by the lag in publishing this update to the community edition. I assumed development work was happening in the open on Github, but I guess that's not the case?
This is a common solution to the problem of PII, but without any information on returning users I would argue that it's value as an analytics platform is limited. Few are the tools where you can grow the business without knowing the difference between a first-time and return user which is the reason cookies were invented in the first place.<p>However, since such businesses already need to collect personal info as part of your account creation it shouldn't be hard to build analytics on top of that existing PII. If they are already collecting PII it doesn't seem to save much to have their analytics tool avoid it?
Most schemes of this kind are just more complicated cookies that people hope will avoid the GDPR provisions by dint of being obfuscated.<p>What the article is discussing looks (at first brush) to be a sensible way of aggregating users up-front before it hits the database, rather than later. So no personal data is stored.<p>Does this meet the requirements for a site to avoid notifying users under the GDPR? I have no idea.<p>Even with the best of intentions, if you use a service like this then you are relying on them a) doing what they claim, and b) not screwing up (by leaving logs around, etc).<p>If I use this service and data from my users gets leaked by Fathom, who gets blamed? The users were on my site, so I guess it is I that gets fined. Maybe the risk is worth it, maybe it isn't.
If visits expire after 30 min, why not rotate the salt every 30 min? Keep current and previous salt, update as needed.<p>I would have more faith in privacy, if you didn't store the salt in the DB or permanent storage. If you manage to statically load-balance the users (e.g. hash site, ip, user-agent, don't forget site), the hash could be in-memory only. Sessions would break on server restart, but that's more of a feature.<p>To move thing further, you might not even need to store the hashes in the DB. Keep them in server memory only and (real-time) update aggregate data in DB.
Hi Jack and Paul - love what you're doing! This solution is so needed.<p>I wondered whether you could explain what makes your hashing different from the hashing used by Facebook for their custom audiences tool which was deemed unsuitable for anonymisation as per <a href="https://www.spiritlegal.com/en/news/details/e-commerce-retail-facebook-custom-audience-not-allowed-without-consent.html" rel="nofollow">https://www.spiritlegal.com/en/news/details/e-commerce-retai...</a>
Looking at their live demo (<a href="https://stats.usefathom.com/#!p=1w&g=hour" rel="nofollow">https://stats.usefathom.com/#!p=1w&g=hour</a>), I can see a lot of traffic is coming from ycombinator. So...<p>(I mean, I don't have a point here but I find it pretty interesting. xD)
The data is not anonymous. Anonymity is actually very hard to claim (read k-anonymity, differential privacy, etc).<p>Nevertheless, the chances of identifying someone are probably pretty low, and it´s a good effort to make analytics more privacy friendly.
Maybe I'm missing something but (1) I don't think this is GDPR compliant, and (2) why so complicated?<p>Regarding (1),<p><i>> Brute forcing a 256 bit hash would cost 10^44 times the Gross World Product (GWP). [...]<p>> We have rendered the data anonymous to the point where we could not identify a natural person from the hash.<p>> It's possible that GDPR does not apply to Fathom since data is made completely anonymous. Even if GDPR did still apply, we reiterate the stance that there is legitimate business interest to understand how your website is performing.</i><p>This seems to imply a profound confusion between the difference of hashing vs. anonymity. Just because it's hashed doesn't mean it's anonymous! You don't need to "brute-force" the hash, you just need to find a user that matches your hash... which is 1 in 7 billion (or so), much more tractable. This is also the principle e.g. MD5 rainbow tables are based on...<p>They claim to change the hash every 24 hours, so it's equivalent to having a session cookie with 24-hour expiration (session cookies are "anonymous" by their definition, they don't have any user information and they're impossible to "brute force", they "just" <i>enable tracking</i>). I've no idea if 24-hour session cookies are GDPR-compliant...<p>Regarding (2), given that this seems (again, I might be misunderstanding) equivalent to a 24-hour session cookie, why not just do that? However, then you're ... drumroll ... giving control to the user. Why not just <i>give control to the user, period?!</i> For example, by storing the list of pages visited in Local Storage, and only pinging the server once for each page(view) every 24 hours?
> Tracking page views alone, without visits, is completely useless and means that you won’t have insight into how many people visit your site / pages each day.<p>What's the difference between a page view and a visit?
At first glance this appears to be a well thought out solution. It will never be able to give you some of the stuff that GA can give you but that is by design. The problem that I see is that as long as GA is able to claim they are GDPR compliant there will be very few websites that will see this as a necessity and so adoption will be relatively low. But, and this is just an idea, one of the things that company could do is to proudly present a 'zero retention' button or logo assuming they do not have other trackers on their pages. That way it might become a distinguishing factor for the adopters and that might drive further adoption.<p>Thanks for building this, I will promote it.
I think the GDPR was enacted into law not to prevent cookies, but to prevent collecting data on regular people. This seems to circumvent the technicalities of the law but not the spirit. The risk is that they enact a new law that puts even further restrictions on website operators.<p>I'm not sure this is a good idea.
I am not sure what exactly they did here. How do they persist the hash between requests?<p>My guess is they use localStorage and sending the hash to their servers with each request.<p>So we are talking about a mechanism that’s just like a cookie.<p>As long as they don’t have any PII and can’t figure out who the user was, then I think the GDPR gives them an exception.<p>But “without cookies” claim is dubious!
This is still logging everything the GDPR says you can't without asking for consent, but you made your search convoluted (but not less efficient if you have all the pieces) to (suggest|lie?) that you need to break the hash and that's why you don't need consent.<p>None of the information you are using on the hash wouldn't be in the search query itself! ip, user agent, path, date, etc. So there is no way to reverse the hash. You just hash your search query and compare in O(1) time.<p>The <i>only</i> piece of information that realistically makes the hash slightly difficult to get is the random number refreshed every day. But either you store it (and i have no reason to believe you do not) or it make the brute force effort trivial as I only need to generate the hash with that variable now.
This is very weak reasoning, because you cannot identify an individual by IP either. This project looks like trying to exploit loopholes. The idea behind GDPR is to make sure companies log only data they need. This project looks into logging the data but without expressing why this is even necessary. Therefore I don't think this is compliant with GDPR.