Data Science Wire

Event-driven pipeline orchestration with Amazon MWAA and Airflow 3.0

AWS Big Data Blog1mo4 min read

Data engineering teams running Apache Airflow across multiple AWS accounts have no built-in way to coordinate workflows between separate Amazon MWAA environments. With Airflow 3.0 on Amazon MWAA, you can use asset-based scheduling and Asset Watchers with Amazon SQS to build event-driven, cross-account orchestration that replaces polling with near real-time triggers.

Read the full story at AWS Big Data Blog

More in Data Engineering