Regarding the histogram issue, I worked on a project that had a few hundred histograms based on data from over 3 billion data points. It turns out that after a few thousand data points many histograms will stop changing significantly.<p>So, unless you really need to show exactly how many data points each bucket contains, it's much easier to run the analysis once offline, then serve just the histogram percentage data. From that you can make an SVG and overlay additional user-specific data on top. The point is that this histogram data is small and easy to cache.<p>You can then rerun the histogram analysis later if you'd like. However, for this project I never saw anything change with more data. It was overkill even to run it as a cron job.
You were sending the entire database (all times from all users) to each individual end user to compute histograms client-side after they finished? Ah, yeah, that could get expensive on Firebase.
I have to brag here. I silently visualized a wall clock ticking off the seconds and got 60.02 seconds.<p>2 hundredths of a second off from reality!<p>I could try it again to validate, but I can't be bothered. As far as I'm concerned, I'm super accurate judging the passage of time. No need to find out if it was a lucky random pick in the interval [58, 62] :)
Just some feed back. I started the challenge, but the constant text changing like "I'm bored" etc was enough to throw off my internal clock to guess when to stop it.
> I pushed a quick fix to the issue by freezing the data being sent to the client, thereby halting the rapid growth in data consumption.<p>What do you mean by "freezing the data"?<p>Regarding the excessive download problem, my first instinct is to periodically (for example, every hour) compute summary statistics for the bar chart and store that in Firebase. This, of course, would require an additional script/service to perform these periodic jobs.<p>I'm not sure if that's what you ended up doing and I'm curious what your solution is.