Summary of "Top-N Recommender System Architectures"

High-level goal

Top-N recommender systems produce a finite ranked list of the best N items to show a user (for example, multi-page music recommendations). The real objective is to surface items users will love, not to predict exact ratings.

Key architectural concepts and pipeline

A typical Top-N recommendation pipeline has several stages:

Data store of user interests
- Stores explicit ratings or implicit signals (purchases, plays).
- Usually a large, distributed NoSQL or cache system (Cassandra, MongoDB, memcache).
- Optional normalization (mean-centering, z-scores) can make signals comparable, but real data is often sparse, limiting effective normalization.
Candidate generation
- Produce a manageable set of items likely to interest the user based on past behavior.
- Example: item-based collaborative filtering — find items similar to those the user liked (e.g., Star Trek → Star Wars).
- Score candidates using source item ratings and similarity strengths; low-scoring candidates may be filtered early.
Candidate ranking
- Combine duplicate candidates (boost items that appear repeatedly).
- Sort candidates by score to form the ranked list.
- More advanced approaches use learning-to-rank models (machine learning) to optimize order.
- Ranking can incorporate additional signals such as average review scores or popularity boosts.
Filtering and business rules
- Remove items the user already saw/rated, offensive content, low-quality items; enforce the N cutoff.
- Apply stop lists and other policy-based filters.
Presentation
- Final list is handed to the display layer (widget) for the user.
- Recommendation logic typically runs in a distributed recommendation web service that the frontend calls during page render.

Representative architectures discussed

Item-based collaborative filtering (Amazon’s 2003 approach)
- Compute item similarities offline; at runtime, start from the user’s liked items and expand.
- Architecturally simple; the main work is building and storing the item similarity database.
Precomputed user-item predicted ratings matrix
- Store a predicted rating for every user-item pair; runtime work is retrieval + sort.
- Supports evaluation of rating-prediction accuracy but is storage- and compute-heavy (inefficient at scale).
- May be acceptable for small catalogs; researchers often favor this approach because they measure rating-prediction accuracy, though that objective does not always align with Top-N goals.

Practical considerations and critiques

Emphasize Top-N relevance (items users will love) rather than global rating prediction.
Data sparsity limits normalization and modeling effectiveness.
Many practical variations and optimizations exist: multi-stage pipelines, hybrid signals, learning-to-rank, and business filters.

Sources / speakers

Course instructor (unnamed lecturer presenting recommendation-system architectures).
Amazon’s 2003 item-based collaborative filtering paper (referenced as the canonical example).