SQLFlow
Single-node stream processing engine that lets you write SQL instead of standing up a Kafka Streams or Flink cluster to aggregate real-time data.
🔗 Visit SQLFlow
Description
Real-time data pipelines have a reputation for being one of the more painful corners of software engineering — usually because the standard tools (Kafka Streams, Apache Flink, Spark Streaming) assume you're willing to run and tune a whole cluster just to compute a rolling average. SQLFlow is built for the much more common case: you have one stream of events and you want to aggregate it, and you'd rather write a SQL query than provision a JVM cluster to do it.
SQLFlow runs as a single-node, single-process engine built on DuckDB and Apache Arrow, consuming from Kafka or WebSockets, transforming with plain SQL (aggregations, windowing), and writing results to PostgreSQL, Kafka, or cloud storage in Parquet/Iceberg format. In benchmarks the Go-rewritten version claims 927,000 messages/second Kafka-to-sink throughput while staying within a 240 MiB memory footprint — numbers aimed squarely at people who don't want the operational overhead of a distributed streaming cluster. It's MIT-licensed, Docker-deployable, and built by Turbolytics, which also offers optional paid consulting for architecture reviews and pipeline migrations on top of the free open-source core.
💬 Our review
The short version: SQLFlow bets that most streaming SQL use cases don't actually need a cluster, and backs that bet with real throughput numbers on a single node — a genuinely useful middle ground between "just use Kafka Streams" and "write custom code."
The obvious comparisons are Apache Flink SQL, ksqlDB, and Materialize — all mature, capable streaming SQL engines. Flink and ksqlDB both assume a cluster (JVM-based, more moving parts to operate) and Materialize is a hosted product with its own pricing. SQLFlow's differentiation is being genuinely single-node and self-hosted with a Go runtime, which trades some scalability ceiling for a dramatically simpler operational story — no coordinator, no cluster to provision, one Docker container to run. That's a real advantage for edge computing, sidecar deployments, or teams whose streaming needs are real but modest. The honest caveat: single-node means there's a ceiling on how much data you can process before you genuinely need a distributed system, and it's a smaller, younger open-source project than Flink or ksqlDB with correspondingly less battle-testing at extreme scale. Worth using for a straightforward streaming aggregation pipeline that doesn't need a cluster; go with Flink or a hosted option like Materialize once your volume outgrows a single node.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Moteur open source MIT gratuit ; Turbolytics propose du conseil payant en option (revue d'architecture, migration de pipelines)
Pros
Débit élevé sur un seul nœud (927k msg/s en benchmark)
Empreinte mémoire réduite (~240 MiB)
Interface 100% SQL, pas d'API d'opérateurs à apprendre
Open source MIT, déployable en un conteneur Docker
Cons
Plafond de scalabilité inhérent à l'architecture mono-nœud
Projet plus jeune et moins éprouvé que Flink/ksqlDB à très grande échelle
Écosystème d'intégrations plus restreint