How to Install Apache Airflow on Windows using Docker Container |
How to Install Apache Airflow on Windows using Docker Container
1. create a directory named airflow and inside that create three sub directories - logs, dags, plugins and place docker-compose.yaml file inside airflow directory
2. install docker desktop for windows
3. download docker-compose.yaml file of airflow latest release
https://airflow.apache.org/docs/apach...
Airflow releases: https://airflow.apache.org/docs/apach...
4. docker-compose up airflow-init
5. docker-compose up
Cleanup - below command will delete all the conainer and free up the volumes and memory
`Run docker-compose down --volumes --remove-orphans`
command in the directory you downloaded the docker-compose.yaml file
airflow-scheduler - The scheduler monitors all tasks and DAGs, then triggers the task instances once their dependencies are complete.
airflow-webserver - The webserver is available at http://localhost:8080.
airflow-worker - The worker that executes the tasks given by the scheduler.
airflow-triggerer - The triggerer runs an event loop for deferrable tasks.
airflow-init - The initialization service.
postgres - The database.
redis - The redis - broker that forwards messages from scheduler to worker.
Reference: https://airflow.apache.org/docs/apach...
For any queries, you can connect with me using below links:
LinkedIn: / ashutoshtripathiai
Instagram: / ashutoshtripathi_ai
Twitter: / ashutosh_ai
Website: https://ashutoshtripathi.com
If you want to message me directly, then connect with me on LinkedIn and send a direct message.
If you want to discuss any topic around ML or have queries related to job switch, please feel free to schedule a 30 min call with me, I am available on topmate: https://topmate.io/ashutosh_ai
#airflow #mlops #ashutosh_ai
Watch video How to Install Apache Airflow on Windows using Docker Container | online without registration, duration hours minute second in high quality. This video was added by user Ashutosh Tripathi 29 January 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 14,27 once and liked it 19 people.