This repository has sample iPython notebook files which show you how to use open data dake formats; Apache Hudi, Delta Lake, and Apache Iceberg on AWS Glue Interactive Sessions and AWS Glue Studio Notebook.

Following examples are based on native connectors.

• Hudi DataFrame example
• Hudi Spark SQL example
• Hudi to Redshift incremental load

• Delta Lake DataFrame example
• Delta Lake Spark SQL example
• Delta Lake Spark SQL example using S3 path
• Delta to Snowflake incremental load

• Iceberg DataFrame example
• Iceberg Spark SQL example

Following examples are based on custom/marketplace connectors. See details in Process Apache Hudi, Delta Lake, Apache Iceberg datasets at scale, part 1: AWS Glue Studio Notebook.

• Hudi DataFrame example
• Hudi Spark SQL example

• Delta Lake DataFrame example
• Delta Lake Spark SQL example
• Delta Lake Spark SQL example using S3 path

• Iceberg DataFrame example
• Iceberg Spark SQL example

• Building RAG using AWS Glue for Apache Spark