Trino

TLDR Iceberg tracks every Flink checkpoint as an immutable snapshot. That gives you time travel queries on a streaming pipeline for free, with about 30-second granularity. You can add columns to a live Iceberg table while Flink is writing to it. Old rows get NULLs for the new columns, and the pipeline keeps running without rewriting data or taking downtime. DuckDB reads the same Iceberg tables without Trino running. Three different query engines can hit the same data without copying anything. ...

TLDR I built an open-source streaming lakehouse: Kafka ingests events, Flink processes them, Iceberg stores them as tables, Trino queries them, and Superset visualizes them. One make demo command runs it all locally. Even when your data lands in Iceberg automatically (as it does with Confluent Tableflow), you still need a query engine and a visualization layer. This stack builds that full picture with open-source components. MinIO went closed-source, so I switched to SeaweedFS (thanks to Robin Moffatt’s research). And Flink’s dependency management is still a jar-shaped nightmare. ...