Quick ML Pipeline using MLflow, Dagster, and Github Actions
This is a quick ML pipeline using:
MLflow for experiment tracking
Dagster for orchestration
Github for version control
Github Actions for CI
As I've been learning more about MLflow, I've been trying to figure out how to use it as part of a more standard software workflow (which often means tools like VCS, CI/CD, and orchestration).
From my naive perspective this all still boils down to: if I change something, can we minimize the chances that my changes either break our project or make it worse. And saying "hey, works on my machine and I can see the model is better" doesn't seem to cut it. As I layer in tools like MLflow, GH Actions, and Dagster, I'm getting more clarity.
Next up I want to build an actual dagster ML pipeline - my current setup is pitiful. Also, I want to see if I can surface some of the model metrics in Dagster (and maybe even the CI tool, GH Actions).
Watch video Quick ML Pipeline using MLflow, Dagster, and Github Actions online without registration, duration hours minute second in high quality. This video was added by user Gus Cavanaugh 08 December 2022, don't forget to share it with your friends and acquaintances, it has been viewed on our site 2,388 once and liked it 39 people.