Imperfections Unveiled: The Intriguing Reality Behind Our MLOps Course Creation
THE FULL STACK 7-STEPS MLOPS FRAMEWORK
This article represents a last bonus lesson out of a 7-lesson course that walked you step-by-step through how to design, implement, and deploy an ML system using MLOps good practices. During the course, you built a production-ready model to forecast energy consumption levels for the next 24 hours across multiple consumer types from Denmark.
During the course, you learned all the fundamentals of designing, coding and deploying an ML system using a batch-serving architecture.
This course targets mid/advanced ML or software engineers who want to level up their skills by building their own ML end-to-end projects.
Nowadays, certificates are everywhere. Building advanced end-to-end projects that you can later show off is the best way to get recognition as a professional engineer.
Table of Contents:
- Course Introduction
- Course Lessons
- Data Source
- Bonus Lesson: Behind the Scenes of an ‘Imperfect' ML Project – Lessons and Insights
- Conclusion
- References
Course Introduction
During the 7 lessons course, you learned how to:
- design a batch-serving architecture
- use Hopsworks as a feature store
- design a feature engineering pipeline that reads data from an API
- build a training pipeline with hyper-parameter tunning
- use W&B as an ML Platform to track your experiments, models, and metadata
- implement a batch prediction pipeline
- use Poetry to build your own Python packages
- deploy your own private PyPi server
- orchestrate everything with Airflow
- use the predictions to code a web app using FastAPI and Streamlit
- use Docker to containerize your code
- use Great Expectations to ensure data validation and integrity
- monitor the performance of the predictions over time
- deploy everything to GCP
- build a CI/CD pipeline using GitHub Actions
If you haven't followed the series and it sounds like something you are interested in, I want to let you know that after completing the course, you will understand everything I said before. Most importantly, you will see WHY I used all these tools and how they work together as a system.
If you want to get the most out of this course, I suggest you access the GitHub repository containing all the lessons' code. This course is designed to quickly read and replicate the code along the articles.
During the course, you learned how to implement the diagram below. After explaining it step-by-step, it doesn't sound so scary anymore, right?
In this final bonus lesson, we want to talk about potential improvements that can be made to the current architecture and design choices made during the course. We also want to highlight the trade-offs we had to make and give you some ideas for future projects.
Think of it as the behind of scenes section
