Airflow with Kubernetes Executors | Run on multi-node k8s cluster | step-by-step guide
This is a step-by-step guide on how to run Apache airflow with Kubernetes Executors on a multinode kubernetes cluster. In this video we will;
1. run a multi-node kubernets cluster on local machine using KinD
2. prepare Dockerfile for your airflow dags
3. minimal helm chart required to run airflow on Kubernetes using Kubernetes Executor
Get the code:
https://github.com/maxcotec/airflow-o...
Chapters:
00:00 intro & agenda
01:17 Airflow kubernetes executors
02:45 How Kubernetes runs airflow on multinode cluster
06:33 Setup kubernetes cluster with KinD
10:08 prepare airflow docker image
12:27 airflow kubernetes helm chart
20:40 deploy airflow on kubernetes
24:45 test DAG via airflow webserver
26:14 sum-up
27:22 outro
Learn Something New Below:
https://maxcotec.com/blog/pv-center-f...
https://maxcotec.com/blog/5-cool-tech...
Watch video Airflow with Kubernetes Executors | Run on multi-node k8s cluster | step-by-step guide online without registration, duration hours minute second in high quality. This video was added by user MaxcoTec Learning 20 September 2022, don't forget to share it with your friends and acquaintances, it has been viewed on our site 15,219 once and liked it 256 people.