Lakewright solutions / brief
Migrate a Parquet data swamp to governed Apache Iceberg tables
A bucket of accumulated Parquet files becomes a set of governed Iceberg tables — schemas read from footers, duplicates skipped by content digest, existing Delta/Iceberg/Hudi tables detected and left alone.
How does it handle a mixed bucket?
Parquet footers give exact schemas; files clustering to the same schema become one feed. Directories that are already table formats (_delta_log, Iceberg metadata, .hoodie) are recognized as existing tables and reported — never re-ingested file-by-file.
The steps
lakewright scan s3://data-lake-old # inventory: feeds + existing tables lakewright approve plans/events_archive.plan.yaml lakewright load plans/events_archive.plan.yaml --rest $CATALOG
Avro lands the same way. Snowflake, Databricks, Trino, and DuckDB query the result directly.
FAQ
The question that matters
"Which of these tables is anyone actually using, and which are carrying dead weight?"
The answer separates the estate into three piles rather than two.
There are tables queried regularly, which stay. There are tables nobody has touched this year, which are archival candidates. And there is a third pile that surprises people: tables in active use whose columns are largely empty — in the sample estate several carry columns that are entirely null across every file, which means a producer stopped populating a field and nobody downstream noticed.
The first cut comes from your warehouse's own query history; the second and third come from the null rates Lakewright profiled at scan time and wrote into the semantic layer. Together they answer what to keep, what to archive, and what was never real data in the first place — the third being the one that usually starts a conversation with an upstream team.
Want this run against your data?
We start with a read-only scan and an inventory report — no installation on your systems, nothing leaves your environment. Most engagements produce findings the team didn't know about in the first afternoon.