Delta Lake Table tools

Creating Delta Lake Tables

Using the deltalake Python library

You can use the deltalake Python library to create Delta Tables and write to the bucket

Using the lakehouse-loader utility

You can also use the lakehouse-loader utility that EDB has created for this task, to export data from an arbitrary Postgres instance to Lakehouse Tables in a storage bucket.

For example, with the lakehouse-loader utility:

export PGPASSWORD="..."
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
# export other AWS envvars

./lakehouse-loader postgres-to-delta postgres://test-user@localhost:5432/test-db -q "SELECT * FROM some_table" s3://my-bucket/my_schema/my_table

This will export the data from the some_table table in the test-db database to a Delta Table in the my_schema/my_table path in the my-bucket bucket.

You can now query this table in the Lakehouse node by creating an external table that references the Delta Table in the my_schema/my_table path. See External Tables for the details on how to do that.