Show HN: bpfquery – experimenting with compiling SQL to bpf(trace)

by knuckleheadson 8/27/2024, 2:55 PMwith 9 comments

Hello! The last few weeks I've been experimenting with compiling sql queries to bpftrace programs and then working with the results. bpfquery.com is the result of that, source available at https://github.com/zmaril/bpfquery. It's a very minimal sql to bpftrace compiler that lets you explore what's going on with your systems. It implements queries, expressions, and filters/wheres/predicates, and has a streaming pivot table interface built on https://perspective.finos.org. I am still figuring out how to do windows, aggregations and joins though, but the pivot table interface actually lets you get surprisingly far. I hope you enjoy it!

by 3r7j6qzi9jvnveon 8/30/2024, 7:17 AM

> --I ran ctags across the linux kernel source code, pulled out all the signatures and then reference the signature when compiling the query, so you don't have to do a bunch of casts in the query.

note you don't need the casts if you use kfuncs instead, which also let you reference arguments by name (from https://github.com/bpftrace/bpftrace/blob/master/man/adoc/bp... ):

    kfunc:tcp_connect {
        if (args->sk->__sk_common.skc_daddr == (uint32)pton("127.0.0.1"))
            ...
    }
With that said, kfuncs don't work (yet?) on aarch64, so this is great for me -- I'll definitely give it a try next time I need it.

(EDIT: formatting)

by password4321on 8/28/2024, 1:21 AM

This seems like it will be awesome, kind of like WMI but ridiculously more capable. Don't be discouraged that it didn't catch on it's first time on HN!

I'm on Firefox Mobile on Android and the first time I visited the page the 3rd/data section just kept refreshing continuously rather than showing results (maybe showed results the first load, but then refreshes of just rows with a dash?). On the second visit to the page the 2nd/C translation section failed to load.

by westurneron 8/30/2024, 3:33 PM

RIL about how the ebpf verifier attempts to prevent infinite loops given rule ordering and rewriting transformations.

There are many open query planners; maybe most are hardly reusable.

There's a wasm-bpf; and also duckdb-wasm, sqlite in WASM with replication and synchronization, datasette-lite, JupyterLite

wasm-bpf: https://github.com/eunomia-bpf/wasm-bpf#how-it-works

Does this make databases faster or more efficient? Is there process or query isolation?