We’ve been developing Observable Plot for a while now, and I’m thrilled to finally have it out in the public. Feel free to ask me any questions! Or share your plots on Observable and I can make suggestions. ;)
As soon as I saw this, I thought: 'how is this different from vega lite'.
An answer is here: https://observablehq.com/@observablehq/plot-vega-lite
It'll be very interesting to see this develop. My initial reaction, as a vega-lite user is I'm not sure it's worth learning the new API immediately.
Having said that, I do find the idea of writing transforms in javascript compelling, and I can see there may be some situations in which this is a very elegant solution.
I think the idea of a plotting library allowing transforms is initially a little jarring - feels potentially like poor separation of concerns. But the more I think about it, the more it seems really logical: a lot of transforms (e.g binning) are done only for plotting purposes, in which case keeping them in the visualisation logic makes complete sense.
Looking further ahead (not sure this is possible in Plot yet, but giving its integration with Observable I'm sure it will be), the idea of a reactive model where charts can both react to user interactions, but also be the source of inputs (select a range, drag a bar, draw a distribution freehand) is very exciting.
Transforms and signals are both possible in vega/vega lite, but I can definitely see the benefit of them being cleaner and being able to accept arbitrary javascript.
Yep, transforms in Plot are totally optional — you can do them outside of plotting if it’s more convenient for you, say using Arquero, and it tends to be efficient since you can supply Plot with columnar data. But having transforms integrated with plotting is convenient for fast exploration, especially since it plays nicely with faceting.
We allude to this in overview, but yes it is our intention to leverage Observable’s dataflow for interactivity, both for any cell to drive what is shown in a plot, and for the plot to drive computation in downstream cells. E.g., you can brush a scatterplot and then show the selected data in a table. Observable Plot is designed to work with Observable Inputs (or anything else interactive):
https://observablehq.com/@observablehq/inputs
Plot is designed to leverage Observable’s language-level interactivity (dataflow) rather than design a new interaction system that is limited to within the visualization.
Really interesting, thanks!
Great that it accepts colunar data, that was a question that had immediately sprung to mind. There's something very satisfying about processing data using Arquero's grammar of data transformation and then immediately passing it over to a grammar of data visualisation (Vega Lite, or now Plot).
Alongside arrow, This kind of thing makes me very excited about javascript becoming a serious option for processing even moderately large amounts of data.
Thanks so much for Observable - it's is by far my favourite programming tool for working with data (my day job is Spark, Python and R, pretty strong competition!)
The first-order difference seems to be: Plot looks designed to be quicker to learn and take less syntax when you want to throw something generic up quickly, with an API carefully designed to get out of the way.
One of the things I have previously found Javascript to lack compared to Matlab, R, Python, or Julia (or Excel for that matter) for doing exploratory data analysis is a standard dead-simple plot tool when you want to draw a bar chart or scatter plot without thinking about it or making any up-front decisions.
Vega-lite (and especially D3) make it possible to make pretty and sophisticated charts with lots of bells and whistles, but I still find vega-lite heavyweight when I want a plot right now. This would surely get easier with experience, but for a newcomer it still adds noticeable friction.
That was my reaction. It seems very similar (which makes sense, they're all using grammar of graphics style).
Vega Lite is very close to R's ggplot2, so we use it for our online science visualizations. The built in downloading svg/pngs built in is pretty awesome.
Vega-Lite it gets us 90% of the way there easily, the remaining 10% is always a struggle. Its well documented but sometimes lacks examples (In a side by side bar graph, how do I move the column heading to the bottom. (column->header->labelOrient: Bottom..)
We plot single cell experiment data (lots of points) and we find vegalite, very performant.
But We'll give this a look. The Javascript transforms seem nice.
> We plot single cell experiment data (lots of points) and we find vegalite, very performant.
I'd be interested in hearing more about your experience here. While I love the grammar of vega-lite (largely via altair), I've definitely run into trouble with many points. Is your work in this area available somewhere? Also, is this for Vitessce?
I think a lot of the performance issues I've come across is based on from copying data between python and the browser. For example, with a million cells you've got at least 5 floats per cell + json structure shipped to the browser. Plus this data tends to stick around in the browser. I'm bullish on approaches like moving data more quickly (possibly even sharing memory, if it isn't a real browser) via Arrow and doing aggregations on the server side with tools like altair-transform. But it's still early days for these.
How does this compare to Apache Echarts (https://echarts.apache.org/en/index.html) ?
p.s. My favorite data exploration toolkit is Clickhouse + Tabix (it uses apache echarts). e.g. https://tabix.io/doc/draw/Draw_Chart/
I'm very excited about ECharts, Superset is moving all of their charts to ECharts (I wrote about this in fact: https://preset.io/blog/2021-4-1-why-echarts/)
Wow those are some great looking charts! The one showing the animated busses driving around Beijing is very impressive.
I'm so excited about this release. I use and like Vega-Lite, and this is openly inspired by that.
The ideas embedded in Vega-Lite are such good ideas that having an alternative implementation of them - especially with the pedigree of D3 and Observable behind them - is a big win. I can't wait to see what happens as Vega-Lite and Observable Plot continue to be inspired by each other in the future.
Thanks for the Vega-Lite feedback. We are huge fans of Observable and D3 and will see what we can learn from Plot.
Exciting work; great to see the new wave of data science oriented JavaScript modules like this and Arquero focused on general principles such as the grammar of graphics and relational algebra.
If I could make a suggestion to the author: it would be a great selling point to include some interactivity in the example demo. This is really hard in R and Python, but easy in JavaScript, and would be a good/easy selling point to a lot of data science people.
Thanks for the suggestion. We will be adding more examples soon (both interactive inputs driving plots, and interactive plots driving outputs). Here’s one with a search box:
This looks great! I spent a few months earlier this year learning and implementing D3 and while a little bit of a learning curve to pick up, it made sense once you got into the flow of things. Looking at this and amazed with how simple it could have been to use as a starting point if there was no need for customization. Great addition to the growing JS solutions for data needs. Another step and tool towards a JS suite of tools for data analysis!
As a D3 user for years, this looks great, the API looks really clean, congrats on the release!
Thank you!
Mike I just wanted to say I really enjoyed using D3 and I was also very impressed when reading the source code. It’s an excellent model for good programming in JavaScript that served as a great example for a younger me.
Thank you to you and your team for D3 and for this library. I’m sure I’ll get lots of use (and freelance gigs) out of it.
Has been a pleasure working with your work.
Ditto. I'm learning D3 and the examples and the d3 source itself has been a boon to a new learner like me.
If you're interested in learning more about Plot, we'll be streaming a free workshop about it on May 18th! Details here: https://www.meetup.com/observablehq/events/277855082/
When I think of easy to use charting for almost any scientific domain I work with, I usually think of Plotly. How does this compare with Plotly?
If you're familiar with Plotly Express in Python, this is like a Javascript API with similar feel. It's a lot less verbose than the equivalent raw Plotly.js code.
Got it, I had assumed there was an equivalent Plotly express API in JS as there is in Python. Thanks!
Hehe, nope! We'd discussed it a few times, but never got around to it :)
I've been reading about D3 and am thinking about integrating it into my React project. But I've come across a number of posts that discuss how React & D3 don't play nicely together because they both manipulate the DOM. [1] is a good starting point for this deep dive.
Does Observable Plot resolve those issues or do we need still to be careful integrating it into a React environment?
[1] https://medium.com/capital-one-tech/a-comparison-of-data-vis...
This is really neat! 'Been using observable for about a year now, and I really appreciate how easy it is to share notebooks with it. Circular dependencies can be fun, but that's not really unexpected considering the model y'all use.
How well does this framework do with something relatively complex like an animated choropleth? Vega-lite and D3 can do them both, but they can both get complicated pretty quickly.
We (and by we I really mean my inimitable collaborator Philippe Rivière) have started prototyping plugins for rendering geographic shapes using D3 projections, so I expect Plot would be able to support this fairly soon. Then you can use something like this Scrubber component to drive the animation:
Nice, thanks! Scrubber is what I use now, and it's great. Can't wait to see the geometric plugins.
Are there Typescript definitions? Bit of a JS/TS noob, maybe it’s obvious, but I couldn’t find any.
Not currently, but we’ve talked about it. We’ve been using a lot more TS internally lately so there’s a growing case and it’s good to hear your vote.
Id also love that
I published a thin wrapper for React https://github.com/iddan/plot-react
Nice....Appreciate the drive towards easier use and non-programmers in general. D3 and Observeable in general are doing excellent work
Awesome. Does it require access to the document/window or it can be used in React?
Ok, I see it does use document since there is d3.create() is under the hood.
Yup. There’s some discussion of this in this issue https://github.com/observablehq/plot/issues/359, which notes that the ~100 test plots in https://github.com/observablehq/plot/tree/main/test/plots are rendered by Node with JSDOM by https://github.com/observablehq/plot/blob/main/test/plot.js.
This seems very well crafted. You have my attention!
I went through the tutorials/demos and this seems very intuitive. I don't do a lot of visualisation, but when I do, I've found manually coaxing data and configuring visual parameters to please a library to be tedious as it's time spent doing busy work I'd rather spend analysing the results. So for me the major drawcard is being able to drop a simple table of results in and let the library handle the necessary inferences for the type of vis I'm after. If it works for the majority of my use-cases, that's a big timesaver for me.
One type of visualisation I sometimes (honestly rarely) use but is extremely useful for finding unintuitive relationships in higher dimensions of data are parallel coordinates charts (actually it's your d3 code I've been borrowing in the past - so thanks for that!) Is this something that can be achieved with Plot at this stage? I'm not sure if it would be a facet, a transform, or something else, like a cross between marks and links (if that's doable).
Anyway, definitely bookmarked - I'll be sure to spend more time using this next time I need to visualise some data, looks like it will handle a lot of the standard vis without the typical boilerplate, thanks for releasing this.
Thanks for the feedback. There’s a test in the repo that does parallel coordinates:
https://github.com/observablehq/plot/blob/main/test/plots/ca...
It’s not a perfect fit for Plot because Plot wants at most one instance of each scale in a plot. (I.e., Plot isn’t designed to support dual-axis charts where you have two y scales. Dual-axis charts tend not to be a good idea anyway.) So to implement parcoords we use a normalize transform to map values to [0, 1] and then render axes manually. We might revisit this topic in the future as Plot grows. I agree that parcoords are very useful for initial exploration, especially with interactive filtering.
It would also be reasonable to have a more specialized component for parcoords — not everything has to live in Plot. For example we found ourselves making a lot of dashboards of time-series data, so we built this reusable component on top of D3 instead of Plot:
https://observablehq.com/@observablehq/timechart
Thanks for the reply - even if it's not within the design scope your test code reads much cleaner than the code I've used in the past (where perhaps naively I've manually transformed and presented each dimension individually where you've done a single normalise operation). For sure every tool fits a particular paradigm and parcoords are incredibly niche, but plotting aside from the demo code you provided I can see some general purpose uses for the transformations as well and definitely a good addition to the tool-belt. Good stuff!
I didn't see any mention of this, but does Observable Plot support interactive charts? E.g. detailed summary on data point hover?
Yes, in a variety of ways. All marks currently support native tooltips (the title channel), and in the near future we’ll be releasing more interactions that you can compose into the chart, e.g. for brushing or pointing. As I mentioned in another comment here, Plot is designed to leverage Observable’s language-level interactivity (dataflow) — so all plots are reactive by default, and plots can expose an interactive selection to the rest of the notebook.
I'd love to use something like Plot or Vega but this is exactly why I am still using D3. Zoom, Delaunay tooltips, and extensible interactivity that I can get to work well in a React based application are very important. Unfortunately Vega's solution to anything interactive seemed incredibly awkward to me at the time.
The only time it seemed like Vega made sense was publication style graphics which was unfortunate.
This can help build interactivity: https://vimeo.com/453645616 http://vega.github.io/lyra/
I've been using D3 for charts for a while. I'm not a power-user by any means, but I understand it enough to make basic plots.
When would I use this instead of straight D3?
Most of the time. Unless you want or need a bespoke visualization, Plot will let you construct a meaningful plot with much less time and effort. (That said, Plot is brand new and still under development, so it might not yet have the specific features you need.)
When you only care about which visual encodings best represent your data, and not how those encodings are implemented in SVG/CSS.
Hey Mike, thanks for all the work you've been doing - I first used D3 back in 2013/2014 and been using them for side projects since.
I haven't dug into the whole v3/v4 modularisation, so that might answer it, but is there a way to minimise the dependencies that Plot brings? You say it needs D3, but what of it does it need specifically, the whole thing? It's just that it's 250K or so, so I was wondering what the minimal setup here is.
Cheers and congrats on the release!
Always a joy to read your code and loved the 'In the spirit of show don’t tell' notebook. Quickly grasped what it's about.
Edit: deleted my question, as it is answered here https://observablehq.com/@observablehq/plot-vega-lite
It looks awesome, but from the forums it doesn't appear that there is a built-in way to output direct to a pdf. I would value that feature, as options to convert html to pdf often have page boundary and formatting difficulties.
Am I missing anything, and is there a chance this feature might be added in the near term?
Plot generates SVG. There’s a variety of tools you can use to convert SVG to PDF. Plot is intended to be used on the web so I don’t think we’ll add direct to PDF rendering. (Though it’s possible that Observable might support it — Observable already supports download options that render SVG to PNG for example.)