Skip to content
Frank David Lee DE

All projects

Completed

Star-Rating-Prediction

Predicting star ratings from customer reviews.

Period
Jun – Sep 2025
Role
Final project for the Data Scientist certificate, team of three
Status
Completed
Stack
  • Python
  • scikit-learn
  • TensorFlow/Keras
  • NLP
  • TF-IDF
  • Streamlit
Live prediction: sentiment plus initial and refined star rating for any text.
reviews
≈ 14,000
models compared
17
weighted F1 (stacking)
0.725

Product tour video

No sound, with captions. All data in the video is made up.

Overview

Predicting the stars from a review's text, based on around 14,000 English-language Trustpilot reviews of the online retailer Temu, collected with a custom Python scraper. As a team of three, we wrote a technical spec and pivoted from regression to multi-class classification; only the text is used, no user or order data. A Streamlit interface shows sentiment plus initial and refined rating for any text, keeps an exportable log and lets you switch between a classic and a deep learning model.

Data and method

The data is heavily imbalanced: about 52% 1★, 29% 5★ and few ratings in between, so we measure weighted F1 rather than accuracy alone. Features: TF-IDF (5,000 terms) plus eleven simple text features such as length, punctuation and capitalisation.

We compared eleven classic models with scikit-learn – from decision tree to XGBoost, voting and stacking – and six Keras models: deep MLP, CNN, LSTM, CNN-LSTM, BiLSTM with attention and a transformer.

Results

The best classic model is stacking (weighted F1 0.725, accuracy 0.733); among the neural models the deep MLP with TF-IDF leads (AUC ≈ 0.88). The transformer clearly lags at this data size – more complexity isn't automatically better.

The middle classes stay hard (macro F1 0.45). An optional 20% correction on 3★ reduces the pull towards the middle; a colour-coded sample of 100 reviews shows openly where the model is off. Together, the views surface recurring supply-chain pain points such as late deliveries and refund issues.

Key takeaway: deep learning needs more compute and more data; classic models train and infer cheaper and faster and work well on smaller, well-prepared data. Choosing a model means balancing accuracy, explainability and operating cost. (We initially explored AliExpress and Wish too, then focused on Temu because of local compute limits.)

Screenshots

An initial 3★ becomes 5★ after the probability-based adjustment – bars: per-star probability, pie: sentiment.
Eleven classic models compared: stacking wins with weighted F1 0.725 and accuracy 0.733.
Six Keras models: the deep MLP with TF-IDF reaches the highest AUC (≈ 0.88); the transformer lags at this data size.
Sample of 100 reviews: dark green exact, light green fixed by the adjustment, yellow ±1 star, red ≥ 2 stars off.
Imbalanced data: about 52% 1★ and 29% 5★, few 2–4★ – hence weighted F1 and the 3★ correction.