Welcome to the MLOps Repository! This repository is dedicated to sharing reading contents, labs and exercises for the MLOps (Machine Learning Operations) course at Northeastern University. The primary goal of this repository is to provide a centralized platform for students, instructors, and anyone interested in MLOps to access and collaborate on course-related materials. You can learn more on Machine learning topics by watching my videos on Youtube or visit my Website.
MLOps is an emerging discipline that focuses on the collaboration and communication of both data scientists and IT professionals while automating and streamlining the machine learning lifecycle. It bridges the gap between machine learning development and production deployment, ensuring that machine learning models are scalable, reproducible, and maintainable. This repository serves as a resource hub for students and instructors of Northeastern University's MLOps course.
The MLOps course at Northeastern University is designed to provide students with a comprehensive understanding of the MLOps field. Throughout the course, students will learn how to:
- Build end-to-end machine learning pipelines
- Deploy machine learning models to production
- Monitor and maintain ML systems
- Implement CI/CD/CM/CT (Continuous Integration/Continuous Deployment/Continuous Monitoring/Continuous Training) for ML
- Containerize and orchestrate ML workloads
- Handle data drift and model retraining
This repository hosts the labs, code samples, and documentation related to these topics.
This repository offers a series of hands-on labs designed to enhance your understanding of MLOps concepts. Each lab focuses on a specific aspect of the machine learning lifecycle, providing practical experience with tools and methodologies essential for deploying and managing machine learning models in production environments.
-
- Objective: Learn to develop and deploy APIs for ML models.
- Sub-Labs:
- FLASK_GCP_LAB: Flask lab data.
- FastAPI Labs: FastAPI lab details.
- Streamlit Labs: Streamlit README - updated.
-
- Objective: Learn how to deploy lightweight functions using cloud-based services.
- Sub-Labs:
- Lab1-CloudFunction Setup: Setting up Google Cloud Functions.
- Lab2-CloudFunction Intermediate: Intermediate Cloud Function concepts and use cases.
-
- Objective: Understand data engineering and preprocessing steps.
- Sub-Labs:
- Apache: Apache setup for data handling.
- DVC Labs/Lab 1: DVC setup and basic commands.
- Data Labeling Labs: Lab focused on data labeling processes.
-
- Objective: Learn containerization techniques for ML applications.
- Sub-Labs:
- Week7_Docker_Container: Introduction to Docker containers.
- Week8_Docker_Container: Advanced Docker techniques and orchestration.
-
- Objective: Set up logging and monitoring using the ELK stack.
- Sub-Labs:
- Lab1_Setup_Windows_WSL_Ubuntu: ELK setup on Windows with WSL.
- Lab2_ELK_Setup_Mac: ELK setup on macOS.
- Lab3_Example: Example of ELK in practice.
-
- Objective: Track and manage ML experiments.
- Sub-Labs:
- Logging Labs: Tracking logs for model training.
- Mlflow Labs: Using MLflow for experiment tracking.
-
GCP Labs
- Cloud Composer Labs: Set up and manage workflows with Cloud Composer.
- Compute Engine Labs: Hands-on with Google Compute Engine.
- KServe Labs: Serving ML models with KServe on Kubernetes.
- Kubernetes Labs: Running and managing containers on GKE.
- Vertex AI Labs: End-to-end ML workflows with Vertex AI.
-
- Objective: Implement GitHub Actions for CI/CD.
- Sub-Labs:
- GitHub_Actions_GCP_Lab_beginner: Beginner-level CI/CD with GitHub Actions.
- Lab1: Basics of GitHub Actions.
- Lab2: Intermediate CI/CD practices with GitHub.
- github-actions-gcp-intermediate-lab: Intermediate GCP integration with GitHub Actions.
-
- Objective: Orchestrate ML workflows with Kubeflow.
- Sub-Labs:
- Lab1-Kubeflow Setup: Setting up Kubeflow environment.
- Lab2-Kubeflow Katib: Hyperparameter tuning with Katib in Kubeflow.
-
- Objective: Gain hands-on experience with TensorFlow for ML model development.
- Sub-Labs:
- TFDV Labs: TensorFlow Data Validation labs.
- TFDV TFX Installation: Setting up TFX and TFDV.
- TFT Labs: TensorFlow Transform labs.
- TFX Labs: TensorFlow Extended for production pipelines.
Each lab is accompanied by detailed instructions and code examples to facilitate hands-on learning. It's recommended to follow the labs sequentially, as concepts build upon each other. For additional resources and support, refer to the Reading Materials section of this repository.
To get started with the labs and exercises in this repository, please follow these steps:
- Clone this repository to your local machine.
- Navigate to the specific lab you are interested in.
- Read the lab instructions and review any accompanying documentation.
- Follow the provided code samples and examples to complete the lab exercises.
- Feel free to explore, modify, and experiment with the code to deepen your understanding.
For more detailed information on each lab and prerequisites, please refer to the lab's README or documentation.
Contributions to this repository are welcome! If you are a student or instructor and would like to contribute your own labs, improvements, or corrections, please follow these guidelines:
- Fork this repository.
- Create a branch for your changes.
- Make your changes and commit them with clear, concise messages.
- Test your changes to ensure they work as expected.
- Submit a pull request to the main repository.
Your contributions will help improve the overall quality of the labs and benefit the entire MLOps community.
The reading materials of this repo was collected from Coursera under the Creative Commons License.
This repository is open-source and is distributed under the Creative Commons License. Please review the license for more details on how you can use and share the content within this repository.
