How to manage Airflow Dags in Production | Dags versioning & Deployment
#apacheairflow #airflowatjagex #jagex #apacheairflowforbeginners #airflowinproduction
In this video, I'll discussing on how we (at Jagex) manage and operate more than 100+ ETL pipelines in Airflow efficiently.
00:00 Intro
00:27 Issues we faced
02:08 Airflow & DAGs versioning
05:47 Airflow Deployment Process
06:51 Automated CICD deployment
08:53 Airflow Dag versioning proposal AIP-36
09:11 Outro
Learn Something New Below:
https://maxcotec.com/blog/chip-chick-...
https://maxcotec.com/blog/what-is-tra...
👍 If this video was helpful to you, please don't forget to thumbs-up,
⌨️ leave a comment and
🔗 share with your friends.
⭐ Support our work by hitting that subscribe button.
🔔 hit bell icon to stay alert on more upcoming interesting stuff.
DAG versioning proposal
https://cwiki.apache.org/confluence/d...
Watch video How to manage Airflow Dags in Production | Dags versioning & Deployment online without registration, duration hours minute second in high quality. This video was added by user MaxcoTec Learning 14 October 2021, don't forget to share it with your friends and acquaintances, it has been viewed on our site 10,631 once and liked it 165 people.