I do wonder if the power of a distributed database is really needed here;
it gets ~1 update a day, so there's no need to have clever consistency stuff.
Most of the queries relate to either today's data (you open the map and zoom in to see how doomed your area is today), or the graphs showing a standard set of history (e.g. cases over the last year).
You'd think you could extract that data to be static and not require database queries, and only fire up the database for the tiny proportion that go digging in history.
First, when you see public officials doing a blog post on "Microsoft.com" website instead of on a public website, you know that something fishy is going on...<p>On the other side, I have the feeling that this thing that clearly over-engineered. Just look at data their diagram... If I'm not wrong there is one writer and multiple reader for the data, or at least multiple writers on one side and multiple readers on another side, without a need for "real time" consistency.<p>So, this thing could probably have been better splitted to not have the use for "scaled" databases
I like the UI design in figure 1. There’s no crap in the way of the data but i don’t feel overwhelmed either. My eyes can scan across sections and it feels natural, theres no firehose effect. I like the thought that’s gone into showing the % vaccinated in the top right. I like the dashed underlines telling me that some explainatory text is available.<p>I think the page looks inoffensive but is clearly focussed on being informative. I wish more data repositories took care and attention towards how data is represented.
It's funny to read about a dashboard with TBs of memory and distributed DBs when on HN, people pride themselves on getting Web servers to run on floppy disk based systems.<p>Joking aside, I liked the description of the dashboard, and generally speaking the UK's government Web sites are better quality, support open data more, are easier to read and navigate than other European countries from what I have seen. This includes this dashboard, which looks clean, simple and functional.<p>I was waiting for the big SQL Server advertising language and positively suprised that the article is very tech agnostic. I did all seem to be rather over-engineered, but Microsoft needs to make some money and government agencies don't generally have wizards from HN working for them, so I can live with an occasionally over-engineered system as long as important systems are working and remain up.<p>The most mysterious part for me was why one would put JSON inside relational tables?
Elsewhere[0] Microsoft have redefined "Open Source" to not include the right to redistribute, or to host on a cloud service.<p>So while there's nothing wrong here with calling an MIT project open souce, it's not inconsistent with their own definition, and useable as propaganda.<p>[0] <a href="https://azure.microsoft.com/en-gb/services/developer-tools/data-studio/#documentation" rel="nofollow">https://azure.microsoft.com/en-gb/services/developer-tools/d...</a><p>>Is Azure Data Studio open source?<p>>Yes, the source code for Azure Data Studio and its data providers is open source and available on GitHub. The source code for the front-end Azure Data Studio, which is based on Microsoft Visual Studio Code, is available under an end-user license agreement that provides rights to modify and use the software, but not to redistribute it or host it in a cloud service. The source code for the data providers is available under the MIT license.
> From the beginning of the COVID-19 pandemic, the United Kingdom (UK) government has made it a top priority to track key health metrics and to share those metrics with the public.<p>According to Dominic Cummings (ex-adviser to the PM), this isn't true at all - one of their biggest failings early on was to not have the data and not see the priority in getting it.[1]<p>[1] <a href="https://news.sky.com/story/dominic-cummings-hearing-the-inside-story-of-the-timeline-of-the-weeks-before-covid-lockdown-12317517" rel="nofollow">https://news.sky.com/story/dominic-cummings-hearing-the-insi...</a> : He added later that there was no data system at that point, and he needed to use his iphone as a calculator to make predictions about the extent to which infections would spread, which he then wrote down on a white board.
I skimmed the article and it seems interesting.<p>On the data side, they have ~7.5 billion total records and they add in 55 million new a day. On the web side, they have ~1 million daily unique users and 100k concurrent users at peak ("concurrent" means "in one minute" is seems).<p>I'm no expert on the web part, but I'm kind of curious why they went with the design they did for the data part. The design, and the chosen technologies make me think they treated it more like a normal web app, not like a dashboard. I would expect OLAP database, not a sharded Postgres, and the data model feels very OLTP to me as well. Or maybe is that because it's mostly time series and not traditional data model?<p>I'll have to go through the article in more detail.
The UK covid dashboard[2] mentioned in the article also has a "simple" version.[1]<p>I love it when websites have a simple text version.<p>[1] <a href="https://coronavirus.data.gov.uk/easy_read" rel="nofollow">https://coronavirus.data.gov.uk/easy_read</a>
[2] <a href="https://coronavirus.data.gov.uk/" rel="nofollow">https://coronavirus.data.gov.uk/</a>
<a href="https://github.com/publichealthengland" rel="nofollow">https://github.com/publichealthengland</a> powered by a lot of Python and Ruby, nice.
This feels like a marketing-led attempt to shoehorn Citus into something topical and shareable, having realised that they've barely talked about it since acquiring the company a couple of years ago.<p>I'm all for Citus, but cmon. Overkill.
The covid dashboard I've found the nicest to use, is actually not one of the official ones but instead this <a href="https://www.travellingtabby.com/scotland-coronavirus-tracker/" rel="nofollow">https://www.travellingtabby.com/scotland-coronavirus-tracker...</a> .<p>The information is presented clearly and it's easy to see what's going on, although in my case the main reason is the breakdown for Argyll & Bute, which isn't a focus area for the national ones!
It is miles better than my state's local Covid Dashboard (<a href="https://coronavirus.iowa.gov/" rel="nofollow">https://coronavirus.iowa.gov/</a>) that is updated fully once a week on Wednesdays. On Monday and Friday they simply post a screenshot of a pixelated version of a summary page only.
Given the original meetings were all about producing a dynamic simulation dashboard which would allow people and politicians to understand the impact of various measures on lives saved...<p>Typical this turned into a pro cloud puff piece that frankly shows a serious amount of over design for what should be a data filtering/processing step to any reasonable "data scientist". And if I'm having to say a data scientist could do it better you know you got it wrong...
> Government project<p>> awarded to Microsoft<p>Hey Europe, want to stop being several decades behind in IT compared to US/China?<p>One simple trick:<p>Ban FAANG from public procurement in Europe!<p>It‘s a no-brainer really.<p>Buy locally, ideally giving small companies and startups a chance.<p>You will have to do it anyway very soon if you want your privacy laws to be taken seriously.<p>There might be a couple of months of friction while buerocrats have to find new procurement partners, but that's it.<p>And then the European tech scene will rise.