Skip to content
Frank David Lee DE

All projects

Completed

Movie Recommender

End-to-end MLOps pipeline for movie recommendations.

Period
Sep – Oct 2025
Role
Final project for the MLOps certificate, team of three
Status
Completed
Stack
  • Python
  • scikit-learn (TruncatedSVD)
  • FastAPI
  • Streamlit
  • MLflow
  • Airflow
  • MySQL
  • Docker Compose
Personal recommendations: enter a user ID, and FastAPI returns the top movies with predicted ratings.
ratings (MovieLens 20M)
20M
RMSE on the test sample (MAE ≈ 0.68)
0.90
training per run
≈ 30 s

Product tour video

No sound, with captions. All data in the video is made up.

Overview

Over seven weeks of the MLOps programme with Liora, our team of three built a production-like recommender – from raw data to serving – and wrapped it in a Streamlit app with presentation and demo modes. The data is MovieLens 20M (20 million ratings, 138,493 users), developed on a sample. The model is collaborative filtering with TruncatedSVD; new users with fewer than five ratings get the most popular movies, unknown movies fall back to the global mean.

Architecture

  • Data: download CSV, split into batches, append to MySQL – each retraining adds about 70,000 new ratings.
  • Training: the Airflow DAG retrain_on_new_batch (daily) picks a batch, ingests it, trains via FastAPI – Airflow doesn't run training itself – compares RMSE and MAE with the production model, promotes the better one and archives.
  • Registry: MLflow stores runs, metrics and versions; the current model carries the @production alias.
  • Serving: FastAPI (with Basic Auth) loads the production model; the Streamlit app shows recommendations, single predictions, movie info, popular titles and system status.
  • Operations: Docker Compose with five services – API, Streamlit, MLflow, Airflow, MySQL – and an API health check.

My contributions

  • Designed the Streamlit app: presentation mode and live demo.
  • Implemented FastAPI endpoints with Basic Auth and MLflow model loading.
  • Built the CSV-to-MySQL ingest with chunking and idempotent upserts.
  • Logged metrics and artifacts to MLflow and wired the Airflow retrain → promote flow.
  • Helped shape training, evaluation and the cold-start strategy.

Results and outlook

On the test sample: RMSE ≈ 0.90 and MAE ≈ 0.68, roughly 30 seconds of training per run on about 12,000 ratings and fast predictions. That is MLOps maturity level 1: an automated lifecycle.

The next step would be level 2 with CI/CD via GitHub Actions, monitoring with Prometheus and Grafana and drift detection with Evidently; the vision is model lineage, bias checks and a feature store.

Screenshots

Training pipeline: Airflow orchestrates ETL, training, evaluation and promotion; FastAPI trains, MLflow stores runs and versions.
Inference pipeline: the demo app calls /predict, and the model comes from the MLflow registry via the @production alias.
System status: MLflow and Airflow at a glance; the health check confirms the loaded production model, responding in 0.01 s.
Data and cold start: a MovieLens 20M sample with a 99.98% sparse matrix – 97.6% of users have fewer than five ratings.
Problem and solution: collaborative filtering with SVD, fallbacks, daily retraining and containers.