Hey, HN. We’re thrilled to release Observable Framework today — a new open-source tool for developing data apps. I highly recommend viewing this example report adapted from our internal dashboard analyzing web logs:<p><a href="https://observablehq.com/framework/examples/api/" rel="nofollow">https://observablehq.com/framework/examples/api/</a><p>This technique of “just plot everything” (7.6M requests as a scatterplot) has revealed surprising insights we’ve used to optimize our servers and better control traffic. We’re also sharing a more traditional dashboard that visualizes the adoption of our open-source visualization library (and in some ways the successor to D3), Observable Plot:<p><a href="https://observablehq.com/framework/examples/plot/" rel="nofollow">https://observablehq.com/framework/examples/plot/</a><p>In addition to releasing Observable Framework, we’ve also made Observable free again for individuals (including private notebooks and databases connectors). Let me know if you have any questions!
Another tidbit buried in this announcement is that Observable Framework is 100% vanilla JavaScript syntax — so you get Observable’s reactive runtime without the quirky Observable JavaScript syntax (as in Observable notebooks). And you can use static ES imports from npm or local modules, declare multiple top-level variables in a code block (not just a single named variable per cell), call the built-in display(…) function to put things on the page, etc. It’s a huge relief to have vanilla syntax and greatly improves interoperability. And we’re figuring out how to port these improvements back to Observable notebooks in the near future.
I appreciate the nod to "File over app"[1] in the announcement. It's so cool that a Markdown file with code blocks can be the source for complex data visualizations and dashboards. Interoperability of this kind makes me giddy. I played around with editing an Observable site from Obsidian and it works great[2].<p>[1]: <a href="https://stephango.com/file-over-app" rel="nofollow">https://stephango.com/file-over-app</a><p>[2]: <a href="https://twitter.com/kepano/status/1758202572446581025" rel="nofollow">https://twitter.com/kepano/status/1758202572446581025</a>
I love Observable. And, this is a phenomenal approach, untethering Observable from observablehq.com. I'm so excited.<p>It probably goes without saying that EVERYONE should have a blog, and this approach from Observable means journalists everywhere can now easily create a dynamic and information-driven blog. It isn't a coincidence that Observable came from a guy that did amazing data visualizations at the NYTimes. We are on the precipice of a major power shift back to journalists and away from dubious corporations, and tools like this enable that transition.<p>(Shameless plug: Svekyll is going towards the same goal. Svekyll is a static site generator inspired by Jekyll. If you want to use Svelte in your blog, check it out. Svekyll bundles the incredible Apache ECharts so you can use echarts in your blog with a few lines of code (and no complicated build process). <a href="https://extrastatic.dev/svekyll/svekyll-cli" rel="nofollow">https://extrastatic.dev/svekyll/svekyll-cli</a>. These are ideas I've been thinking about too.)
There's an almost bewildering amount of interesting ideas buried in this.<p>Things like data loaders which are ANY script that can output data (as JSON or something else) to standard output. Markdown files with ```js blocks in that get executed. The reinvention of the core Observable notebook to avoid custom syntax.<p>This is really big.
I'm super excited to try this out! Couple of questions since I see @mbostock active in the comments.<p>1. Is the flexibility of languages used in data loaders/backend going to eventually come to the front end/ui? Or will the paradigm always be bring-your-own-language for the data loading but build your dashboard with observablejs/observable plot?<p>2. Considering ObservableJS is supported by Quarto, can we look forward to Observable Framework integrated with Quarto too? Or is the fact that the latest Quarto version also featured Dashboards more of a competitor to Framework?<p>3. Saw some comparison to Evidence.dev in the comments. I saw some shades of similarity with the markdown focused dev experience too but I recall Evidence chose Apache Echarts for their main charting library. Any thoughts of the pros/cons of Echarts vs ObservableJS/Plot?
A couple of questions:<p>1. Let's say I got a Sqlite/Duckdb database file on my server. It's got multiple tables and some of them 100M to 150M records. I want to create a plot/table that would have a slider/filter to only bring and show a slice of data at a time. Since it's statically generated data, how is this interactivity achieved? All the possible facets of data filtered by which ever way will be generated? Won't it be huge and how long will it take to generates this static data or is there an actual call back to the server to the duckdb file (I assume it works with .duckdb file too?)<p>2. If Observable Framework provides the front-end, does it mean I can use any auth library if I want to create a web site with a log in and subscription options?<p>3. If it's a static web page, does it mean that at any time a user views a chart, they will also be able to go to the Dev Tools and download the file with data that's behind the viz?<p>4. When (if you can share of course) is the planned release of Plot's interactions: zoom, pan, interactive legend, brush?<p>5. Deployment - with big parquet, sqlite, csv files, it's impossible to do CI/CO through github or vercel and such. Will your hosting services offer an option to host those files and runtimes to generate them?<p>Thanks
I am totally psyched up to try this. Observable (1.0) has been a very effective outlet for my research. There is no other platform that could have hosted it and offered me the quick and easy tools to make persuasive visualizations. I was a little concerned when I heard some people got laid off from the company, but it seems like it is still going.
Observable is such an incredible, powerful and enjoyable tool. I use it heavily, including to power my blog. I love it, but I've always had a slight concern about needing to rely on the Observable notebook website. So this has really made my day.<p>To give a sense of the kind of performant, statically hosted interactive content that has only really been within my reach since using Observable, here are some examples:<p>Highly interactive vis:
<a href="https://www.robinlinacre.com/visualising_fellegi_sunter/" rel="nofollow">https://www.robinlinacre.com/visualising_fellegi_sunter/</a><p>Editable computations that flow through the document
<a href="https://www.robinlinacre.com/computing_fellegi_sunter/" rel="nofollow">https://www.robinlinacre.com/computing_fellegi_sunter/</a><p>Multiple synced representations of data:
<a href="https://www.robinlinacre.com/prob_bf_mw/" rel="nofollow">https://www.robinlinacre.com/prob_bf_mw/</a>
<a href="https://www.robinlinacre.com/partial_match_weights/" rel="nofollow">https://www.robinlinacre.com/partial_match_weights/</a> (half way down, under 'Understanding the partial match weight chart and waterfall chart)<p>Of course, there are a huge number of additional examples:
<a href="https://observablehq.com/trending" rel="nofollow">https://observablehq.com/trending</a>
but i think the whole thing makes much more sense to the end-user when embedded (sadly at which point they don't even know it's observable!)
This looks amazing! I really like the clear seperation of loading/prepping data, and presenting it.<p>Some requests:<p>Add simple examples and more clarity to the publish docs.<p>I assume most ppl would prefer to deploy via github actions[1], which the docs currently just link to a complex deploy file - can you please add some more documentation on this, or add an example of the simplest possible deploy file?<p>Suggestion: is it possible to use a interface (like in vercel) to connect to a github repo and build/publish on changes?<p>[1]: <a href="https://observablehq.com/framework/getting-started#deploying-via-github-actions" rel="nofollow">https://observablehq.com/framework/getting-started#deploying...</a>
If you play the history of Observable backwards you start with a company creating a static site generator for dashboards, which then struggles to find a market fit as it tries to bring datascience to middle management, to finally reach a focused, simple and elegant tool for exploratory programming, data visualisation, and interactive documentation in javascript.
No mention of math formatting but from the docs it looks like there is TeX support!<p><a href="https://observablehq.com/framework/lib/tex" rel="nofollow">https://observablehq.com/framework/lib/tex</a>
The new direction seems very similar to what evidence has been doing for a while<p><a href="https://evidence.dev">https://evidence.dev</a>
Sounds a bit like the “baked data” pattern. Which I think is a really good idea. I have long been toying with how to use Datasette to make a deliverable dashboard, so this is interesting.
Getting started guide: <a href="https://observablehq.com/framework/getting-started" rel="nofollow">https://observablehq.com/framework/getting-started</a>.<p>Looks very nice!
Interesting pivot from the Observable team.<p>I loved observable and wrote a couple of notebooks, it worked great !
I'm gonna try Framework asap !
This is really a game changer for creating data apps. I loved observable, but convincing scientists with no js knowledge to try the platform was almost impossible. Markdown + language agnostic loaders seems like the perfect way to collaborate.<p>Are there any plans to allow existing observable notebooks to be deployed? For example, a one-click “deploy” button?
This is very cool to see! We've been building something very similar (in some regards, very different in others) in <a href="https://rysana.com/bundown" rel="nofollow">https://rysana.com/bundown</a><p>Somehow ideas like this emerge in waves seemingly with no coordination, kind of like what happened when Calculus was first invented<p>(Of course there's a lot of prior art here but it's interesting to see a specific jump towards polyglot single-file Markdown 'apps' and so on happen around the same time)
It would be really cool if you guys supported Malloy. Maybe you already do? <a href="https://www.malloydata.dev/" rel="nofollow">https://www.malloydata.dev/</a>
Congrats on the launch! Excited to hear that you go fully open-source with this! There is a certain need for great visualization tools that enable building apps.
I love how much is possible with the Observable framework and support for libraries like d3.js. However many data apps cannot precompute their outputs. For example, a pipeline that extracts text from documents based on what a user queries, cannot precompute the results and any visualizations must be updated every time. The best hack to accomplish this seems to be rebuilding the app on each update. Or is there another solution?
I've setup a codespace starter template that pulls in the required dependencies to start building dashboards with Observable Framework, right inside your browser. You can access the starter template here:<p>[0]: <a href="https://github.com/dleeftink/observable-codespace">https://github.com/dleeftink/observable-codespace</a>
I miss source loading D3 into a simple HTML page, and having D3 tutorials that were up to spec with the latest D3 release.<p>Yeah Observable data pages look cool, but it really feels like excessive JS bloat for the features.<p>I think I miss throwing a D3 viz together without having to load an entire framework library.
ObservableHQ is great but for me this crashed on install. Unfortunately I'm not aware of any npm mechanism to report errors back to the developers.<p><pre><code> $ npm init @observablehq
node:internal/errors:477
ErrorCaptureStackTrace(err);</code></pre>
Are you planning to add new UI components like Data Table or other in the future or it’s purely Plotting and Data ingestion and the UI through Tailwind or CSS or would it be possible to add UI libraries like shadcn or DaisyUI to make it a full fledged web site? Thanks
What would be a good design pattern to put these dashboards behind auth? I suppose since they're static files you could just serve them with something like FastAPI or Spring Boot and have your CI/CD refresh the static files throughout the day on shared storage?
Are there any similar projects that allow you to plot thousands or tens of thousands of datapoints and also allows the viewer zoom into time series plots? I'd love to have an in-browser matplotlib replacement. So far I haven't found one. Observable plots look static.
This is super cool! I’ve been looking for ways to integrate Observable with my blog posts and make them more interactive and engaging. This might just be it.<p>Thank you and congrats on the release!
I love this and hope that a django-observable package comes along that makes it very easy to integrate and serve these static apps through a larger Django site.
And if you want a simpler tool for creating interactive docs, maybe try Codapi:<p><a href="https://codapi.org/" rel="nofollow">https://codapi.org/</a>
Lots of good stuff here, but are you really naming your framework "Framework" (as in "With Framework, you can build...")?<p>You do realize that naming only makes sense inside your own company, right? To everyone who uses it, it's "a" framework or "Observable's framework". No consumer of the framework is going to refer to it as "Framework" without ridiculous amounts of confusion resulting.