Just wanted to also give a shout out to Apache DataFusion[0] that IOx relies on a lot (and contributes to as well!).
It's a framework for writing query engines in Rust that takes care of a lot of heavy lifting around parsing SQL, type casting, constructing and transforming query plans and optimizing them. It's pluggable, making it easy to write custom data sources, optimizer rules, query nodes etc.
It's has very good single-node performance (there's even a way to compile it with SIMD support) and Ballista [1] extends that to build it into a distributed query engine.
Plenty of other projects use it besides IOx, including VegaFusion, ROAPI, Cube.js's preaggregation store. We're heavily using it to build Seafowl [2], an analytical database that's optimized for running SQL queries directly from the user's browser (caching, CDNs, low latency, some WASM support, all that fun stuff).
[0] https://github.com/apache/arrow-datafusion
Just want to say congratulations to the team!
2 years and 9,500+ commits is a hell of a feat.
Happy longtime Influxdb user here. I wanted to congratulate Paul and the team on reaching this milestone. Followed IOx development a bit - can't wait to finally test it out!
I evaluated InfluxDB with the Prometheus Kube Stack chart.
It didn’t really work.
I’m not stupid and I can read docs.
My feeling was it’s like Elastic. Default configuration is so flawed and inscrutable, on purpose, you can forget about using it yourself.
I use Thanos now. At least it fucking works.
I suppose if I need fast queries, I’ll use Postgres.
You guys need to focus on making stuff that works. It’s competitive out there and you don’t have the insights into people who try and wind up hating your guts for being annoying.
Congrats! Was always a pleasure to hear about IOx when I interned there last summer! They’re an awesome company to work for.
Congrats to the team at InfluxDB - great to see this released.
Will InfluxDB IOx eventually replace InfluxDB v2?
Will be very interesting to see this compared to Clickhouse
Isn't InfluxDB rewriting their storage engine for the nth time? It makes me have a little less faith in their project to be honest.
>Unbounded cardinality
This has been the largest criticism of InfluxDB in the past. Kudos to the team for acknowledging and solving it!
> IOx supports SQL natively and our cloud customers can connect using Postgres-compatible clients like psql, Grafana’s Postgres data source, and BI tools like PowerBI and Tableau.
Initially InfluxDB had InfluxQL, a SQL like language for querying data. Then they transitioned to Flux, indicating it was superior to writing complex SQL queries over time series data. Now they are highlighting native SQL support. Since this was only announced today, hopefully there will be clear messaging on which query languages will be supported going forward.
It’s also worth noting that queries can also be executed over an HTTP API that platforms like PowerBI can consume today.
>First introduced in 2020 as the open source project InfluxDB IOx, the new storage engine is the product of sustained development by InfluxData and considerable contribution from the InfluxDB open source developer community. Today, the new engine based on IOx arrives first in InfluxData’s multi-tenant InfluxDB Cloud service, available to developers worldwide.
Will this later be available in an OSS package for self-hosting?