3 items tagged with "airflow"
Write an Airflow DAG with scheduled tasks and dependencies, run it locally, and monitor execution in the UI.
Classic big-data batch pattern: Apache Airflow orchestrates scheduled DAGs that run distributed Apache Spark jobs over data-lake storage.
The established task-based workflow orchestrator versus Dagster's asset-oriented, developer-friendly data orchestrator.