Time Travel and Schema Evolution in the Streaming Lakehouse
TLDR Iceberg tracks every Flink checkpoint as an immutable snapshot. That gives you time travel queries on a streaming pipeline for free, with about 30-second granularity. You can add columns to a live Iceberg table while Flink is writing to it. Old rows get NULLs for the new columns, and the pipeline keeps running without rewriting data or taking downtime. DuckDB reads the same Iceberg tables without Trino running. Three different query engines can hit the same data without copying anything. ...