Movie Recommender
End-to-end MLOps pipeline for movie recommendations.
- ratings (MovieLens 20M)
- 20M
- RMSE on the test sample (MAE ≈ 0.68)
- 0.90
- training per run
- ≈ 30 s
Product tour video
Overview
Over seven weeks of the MLOps programme with Liora, our team of three built a production-like recommender – from raw data to serving – and wrapped it in a Streamlit app with presentation and demo modes. The data is MovieLens 20M (20 million ratings, 138,493 users), developed on a sample. The model is collaborative filtering with TruncatedSVD; new users with fewer than five ratings get the most popular movies, unknown movies fall back to the global mean.
Architecture
- Data: download CSV, split into batches, append to MySQL – each retraining adds about 70,000 new ratings.
- Training: the Airflow DAG retrain_on_new_batch (daily) picks a batch, ingests it, trains via FastAPI – Airflow doesn't run training itself – compares RMSE and MAE with the production model, promotes the better one and archives.
- Registry: MLflow stores runs, metrics and versions; the current model carries the @production alias.
- Serving: FastAPI (with Basic Auth) loads the production model; the Streamlit app shows recommendations, single predictions, movie info, popular titles and system status.
- Operations: Docker Compose with five services – API, Streamlit, MLflow, Airflow, MySQL – and an API health check.
My contributions
- Designed the Streamlit app: presentation mode and live demo.
- Implemented FastAPI endpoints with Basic Auth and MLflow model loading.
- Built the CSV-to-MySQL ingest with chunking and idempotent upserts.
- Logged metrics and artifacts to MLflow and wired the Airflow retrain → promote flow.
- Helped shape training, evaluation and the cold-start strategy.
Results and outlook
On the test sample: RMSE ≈ 0.90 and MAE ≈ 0.68, roughly 30 seconds of training per run on about 12,000 ratings and fast predictions. That is MLOps maturity level 1: an automated lifecycle.
The next step would be level 2 with CI/CD via GitHub Actions, monitoring with Prometheus and Grafana and drift detection with Evidently; the vision is model lineage, bias checks and a feature store.