Apache Superset

"semi-abandoned disasterware"

Hmm. I suppose all open source looks that way if it doesn't get regular funding/attention.

Apache does house a lot of abandonware. They had some relevance as recently as 6-7 years ago but they've been largely replaced by nginx I think. That being said, I view them like the local soup-kitchen - important to have and maintain, but not where I want to go for a 5-star meal.

almost everything I've ever worked with that was run by the Apache foundation turned out to be semi-abandoned disasterware over time

Can you name a few examples?

topping HN for no apparent reason

I think the HN algo is pretty easily manipulated. I worked at a startup that had an effective process to get things to the front page

Because we (the FBI Surveillance Van) saw that you picked Metabase, called our shady French-accented overlord, and he told us to dump it.

I used Metabase at my last gig (CTO @ e-commerce, 30+ users) and it was well-received and dare I say even a bit adored. It was the only self-hosted tool I'd receive after-hours text messages about going down that someone urgently needed back up for some task due tomorrow.

Business users loved the self-serve query builder, and it wasn't uncommon to walk around the office and see Metabase up on someones screen. My CEO absolutely loved it, and used it daily including to put together data for board decks.

None of my users cared about visualizations, and lived in tabular data. This included finance, marketing, merchandising, operations, and executives (CEO/COO/CFO). The only people that lamented the limited visualization were analysts. Power users did all their day-to-day work in Excel or other tools anyway, such as managing marketing spend or inventory allocations.

Metabase was great for dashboards and self-service (ad-hoc). 10/10 would deploy again.

Yeah, I'm in a similar thought process. I've been burned multiple times by Apache, will not touch ever again.

Apache Airflow, Kafka, Spark, ECharts, and many others are still going strong! It really depends on the project to be honest.

I think that there is an active company behind Superset called Preset.

https://preset.io/

I don't think it's semi-abandoned. I had a brief interaction with the project in my previous job, and I found the community and the company to be reasonably engaged and responsive.

Apache Software Foundation is just an umbrella organization to keep things on life support till someone can apply sufficient motive force to resurrect. I think that's really valuable. Lots of projects there have had that effort applied to them and kept going.

I have the opposite experience. Lots of good stuff is hosted by Apache Foundation, such as Kafka, Maven, Cassandra, Camel, the Tika project, Superset, Solr, but I will admit they had more relevance 10 years ago. And I dont think there are many organizations that keep open source projects alive longer than the Apache projects.

Everything I've ever worked with that was run by the Apache foundation turned out to be semi-abandoned disasterware over time.

Amen brother.

I also found Superset unintuitive to use and setup as well. I settled on standing up Metabase because it was so simple to get started with trying it since it can be launched as a single jar. The business users loved it and so did I and administration with a Postgres backend instead of the internal h2 database was a breeze.

Let’s be honest, intuitive is the last word we’d use to describe most Apache projects.

It’s more intuitive than the open source alternatives but is not as intuitive as tableau and others.

Had a similar experience with Superset. A few others have mentioned Metabase and I agree it's better, but if you're looking for a different approach to data, check out Definite (https://www.definite.app/). It's a "data stack in a box". A few things we're doing differently:

1. Built-in data warehouse - We spin up a duckdb database for you to load data to

2. 500+ connectors - You don't need to buy a separate ETL and you can pull in all your data (e.g. Postgres, Stripe, HubSpot, Zendesk, etc.) automatically

3. Semantic layer - Define dimensions, measures, and joins in one place. We have pre-built models for all the sources we support (e.g. the Stripe model already has measures for MRR, churn, etc.)

4. Simple BI - Build a table with the data you want and generate visuals off that table

I'm mike@definite.app if you have any questions.

I've just been playing with superset. I'd have to agree. Things which are easy in SQL are... disturbingly hard or nonobvious in superset.

And the documentation is sparse at best.

Are there better alternatives?

It wasn’t fast either when I used it.

What it was though, was riddled with dozens of Python runtime errors and innumerable glitches.

Metabase is where it’s at.

We use ECharts in our open source BI tool (Evidence) and it's a great library. Has helped us build a declarative syntax for viz which can be version controlled (https://evidence.dev)

Previous HN discussion: https://news.ycombinator.com/item?id=35645464 (97 comments)

How do you deal with data visibility and permissions? I mean, most tables have data that should only be seen by a specific user or group ID, and that layer is usually handled by the application. It would be awesome to expose the power of Superset for users, but I imagine creating the security layer would be a pain.

I have no experience with Superset. Can you elaborate on a few points where you see it excel beyond Tableau?

I’d like to see these types of apps start offering SVG embedding of things like graphs. Frames are such a pain.

eCharts is awesome. We moved from plotly after using it for several months to echarts at https://github.com/openobserve/openobserve and are super happy.

Had good results with echarts. With Superset not so much: complicated to install, lost all dashboards after an update, cryptic error messages, custom queries meh: we decided to use views in Postgres. The project with Superset was finished successfully, but the time spend is a multiple compared to using something like Power BI.

All in all, not very innovative, but highly needed open source version of a traditional BI tool. Definitely something to follow and to use in temporary, not too demanding use cases. And hopefully a future replacement of Tableau or Power BI.

I love Grafana but Grafana doesn't really support non-time-series visualization that well.

They're both washboarding apps, and while I'm sure they each have panel types the other doesn't yet support, I don't think that's intrinsic. The differentiation as I see it, is that Superset is designed to craft SQL queries and visualize the results. The query builder is probably where this shows the most.

To make it more concrete -- coworkers tell me Grafana doesn't work so well with Apache Druid, while Superset supports it quite well.

You can’t trivially plug grafana in front of any SQL database, and grafana is more about graphing/plotting (usually time series).

The fundamental difference is that Grafana isn't great at cross referencing data in different data sources. (I love Grafana and I pay for the Cloud version.)

Much more focused on interactive slicing and dicing of data, rather than mostly following a few pre-defined time-series, as is the focus of Grafana.

As such, closer to an open source replacement for PowerBI.

grafana is built more for operational and timeseries data, but not so optimal for complex analytical queries. Ex: up-to-second data on cpu load on a host.

superset is the flip side of grafana; not good for up-to-second updates, but good for complex queries. Also, non-time series stuff. Ex: Which customer groups bought which products for all time? <— that type of BI stuff.

Yes, I am at a company using Metabase, but I have a decent amount of experience with Superset (albeit from many years ago).

The reason we chose Metabase was that it had table joins, while Superset doesn't (unless it has added them since I used it). It also looks a bit sleeker. But I strongly prefer Superset; I found that with Metabase I had to turn a lot of things off to make it usable (Let me see "the_table" not "The Table"!), I was constantly annoyed at the opacity around models vs "questions", etc. and every time I wanted to change a question Metabase insisted on creating a new one instead. The real issue here was when we wanted to swap out the data source for a lot of questions but there was no clean way to do so without MB just creating new questions.

Also, Metabase doesn't have serialization unless you pay them AND you self-host, (if I'm self hosting then what exactly am I paying for?) and that's pretty annoying. https://www.metabase.com/docs/latest/installation-and-operat....

But it does let you join tables. Sometimes that's enough to make MB worth dealing with.

Metabase is great, I use it with a Oracle Database.

Metabase is a bit more user-friendly to be honest than Superset. Superset has a WAY more liberal license, so it's ideal for people who want to customize Superset and build data apps.

Reposting from a comment of mine about 60 days ago:

I recently ran a little shootout between Superset, Metabase, and Lightdash — all open source with hosted options. All have nontrivial weaknesses but I ended up picking Lightdash. Superset is the best of them at data visualization but I honestly found it almost useless for self-serve BI by business users if you have existing star schema. This issue on how to do joins in Superset (with stalebot making a mess XD) is everything difficult about Superset for BI in a nutshell. https://github.com/apache/superset/issues/8645

Metabase is pretty great and it's definitely the right choice for a startup looking to get low cost BI set up. It still has a very table centric view, but feels built for _BI_ rather than visualization alone.

Lightdash has significant warts (YAML, pivoting being done in the frontend, no symmetric aggregates) but the Looker inspiration is obvious and it makes it easy to present _groups of tables_ to business users ready to rock. I liked Looker before Google acquired it. My business users are comfortable with star and snowflake schemas (not that they know those words) and it was easy to drop Lightdash on top of our existing data warehouse.

Tableau is the best, most powerful, most mature of the three, most feature complete and easiest of the three. I think they give you a 30 day trial.

This is a single user application, unless you make it part of your built application.

If you are doing data analysis I don't think any of the 3 pieces of software you mentioned are going to be that helpful.

I see these products as tools for data visualization and reporting i.e. presenting prepared datasets to users in a visually appealing way. They aren't as well suited for serious analytics.

I can't comment on Superset or Tableau but I am familiar with Power BI (it has been rolled out across my org), the type of statistics you can do with it are fairly rudimentary. If you need to do any thing beyond summarizing (counts, averages, min, max etc). It is not particularly easy.

For data analysis I use SAS or R. This software allows you do things like multivariate regression, timeseries forecasting, PCA, Cluster analysis etc. There is also plotting capability.

Both these products are kind of old school, I've been using them since early 2000's, the "new school" seems to be Python. Pretty much all the recent data science people in my organization use Python. Particularly Pandas and libraries like Seaborn (https://seaborn.pydata.org/).

The "power" users of Power BI in my organization tend to be finance/HR people for use cases like drill down into cost figures or Interactively presenting KPI's and other headline figures to management things like that.

No built-in thick semantic layer, compared to Looker.

I wrote about Superset's semantic layer here: https://preset.io/blog/understanding-superset-semantic-layer...

One popular option is to use dbt or Cube for the semantic layer and pair with Superset: https://preset.io/blog/announcing-presets-ui-integration-wit... and https://preset.io/blog/open-source-looker-cube-superset/

I posted a comparison to Looker in a different thread: https://news.ycombinator.com/item?id=39523454

Same for Airflow? I’m not sure I understand what you mean.

Should it? If you really need that, join the different sources with TrinoDB (or any related managed service like AWS Athena) and connect it to Superset.

Superset would be on my shortlist if I had to use something else, but the join limitations were part of why I passed.

We use https://cube.dev/ as intermediate layer between data warehouse database and Superset (and other "terminal" apps for BI like report generators). You define your schema (metrics, dimensions, joins, calculated metrics etc) in cube and then access them by any tool that can connect to SQL db

This might be what you're looking for: https://www.youtube.com/watch?v=kGfUIOK87V8

You can check this out. This is a Preset Demo, but shows quite a bit of Superset within Preset (which offers multiple instances of Superset as "Workspaces") https://www.youtube.com/watch?v=V0HwGnC1rU8

Well Redash got acquired so development stopped, biggest difference between Superset & Redash. Preset.io supports Superset still

There is a major release on the horizon (4.0) and there were just a couple of patch releases for the 3.x variants. I'm surprised to see it trending too, but I'm happy about it. More people need to know that Open Source BI is here, and here to win.

Is there more than this single HN submission?

And sometimes projects are handed to them to die. The way they (mis)handle OpenOffice is unforgivable.

Mhmm this gives me an idea.. what if I could "group" metabase sql queries by "similarity" (either of results or of the query itself)

Another option could be to use LLM to summarize, tag and group queries for better discoverability.

100% agree.

One thing that helps is hooking metabase up to its own database and building queries on your queries, e.g.:

    select *
    from report_card 
    where dataset_query ilike '%' || {{query}} || '%'

We also try really hard to keep the Collection directory structure clean and consistent. But it's still really hard.

We try! We also accept PRs and Issues if there are things bugging people, of course. It's always a balancing act between building some new feature that people are clamoring for, or fixing those cosmetic issues that always crop up.

I wanted the same info, sadly lacking.

AFAICT needs a db (MySQL/Postgres) and a cache (Redis/Memcached) and one (or more?) web workers.

Then optionally also Celery workers (for "async queries" i.e. slow running)... not sure how optional that is though.

In my experience, people with a business related background have an easier time learning how to use BI tools (this is true even if Superset may be less user-friendly than other commercial product like Tableau); Jupyter is an interactive computing platform that is based on notebooks and cells, that's more useful for data scientists/engineers whose needs might exceed the capabilities of a SQL interface.

Yep... there's Drill By, which is more flexible than drill-down. Rather than having to specify a strict hierarchy of drilling "levels" you can pick columns, hierarchical or otherwise, to drill into.

Maybe you can't say who, but I'm sure curious. Add yourself to this page if you can: https://github.com/apache/superset/blob/master/RESOURCES/INT...

It should be possible (have not tried myself):

https://preset.io/blog/accessing-apis-with-superset/

"Shillelagh (ʃɪˈleɪlɪ) is a Python library and CLI that allows you to query many resources (APIs, files, in memory objects) using SQL. It's both user and developer friendly, making it trivial to access resources and easy to add support for new ones"

https://github.com/betodealmeida/shillelagh

It's definitely come a long way since 2017! It's improved markedly in terms of functionality and performance. It looks much prettier now as well.

Maybe try the Docker installation to keep the dependencies off your system:

https://superset.apache.org/docs/installation/installing-sup...

A big thing here is that Superset and most of the other BI tools can connect directly to databases which is commonly the source of truth or data warehouse in some businesses. Secondly, Elastic have focused on other operational areas such as security, observability, and indexing / search. Kibana can do some dashboarding on those areas and its UI is nice, but Superset and similar tooling are more suited for BI purposes.

You can style dashboards with CSS as much as you'd like, though there are some limitations (canvas/webGL elements). I wrote a whole blog post on it: https://preset.io/blog/customizing-superset-dashboards-with-...

If you want to style the whole application, you can fork the repo and go bananas. If you're looking for theming, there's more to be done yet on that front, and I wrote an article on that too: https://preset.io/blog/theming-superset-progress-update/

Took me a while figure out how to embed it into my app using Superset Embedded SDK.

Superset Embedded SDK - "Embedded SDK allows you to embed dashboards from Superset into your own app, using your app's authentication. Embedding is done by inserting an iframe, containing a Superset page, into the host application."

Well, it's free! Or significantly cheaper even if you opt to use Preset to run a hosted/managed/compliant version of it, and not have to deal with config/security/upgrades/migrations. This article is a year old, but it might help a bit: https://preset.io/blog/apache-superset-vs-tableau/

Thanks for sharing, it's so exciting to see so many OSS BI frameworks