Implemented The Paper “Recommending What Video to Watch Next: A Multitask Ranking System"

[vishesh9131]

Implemented YouTube Multitask Ranking System from Paper using CoreRec

Welcome to the CoreRec example showcase! This discussion walks through the ex1.py script, demonstrating how to build a hybrid recommendation system using CoreRec's collaborative filtering and content-based filtering engines. This example serves as a guide for implementing recommendation systems with CoreRec.

📁 Corerec’s Methods Used Here

Core Components

• Engines
  • unionizedFilterEngine
    • mf_base: Abstract base class for matrix factorization recommenders.
    • matrix_factorization_recommender.py: Implementation of a matrix factorization recommender.
  • contentFilterEngine
    • tfidf_recommender.py: Content-based recommender using TF-IDF features.
    • hybrid.py: Combines multiple recommenders into a hybrid system.

🔍 Detailed Walkthrough

1. Importing Necessary Libraries

import logging
from typing import List, Optional
from scipy.sparse import csr_matrix
from corerec.engines.unionizedFilterEngine.mf_base.matrix_factorization_base import MatrixFactorizationBase
from corerec.engines.unionizedFilterEngine.matrix_factorization_recommender import MatrixFactorizationRecommender
from corerec.engines.contentFilterEngine.tfidf_recommender import TFIDFRecommender
from corerec.engines.hybrid import HybridEngine
from sklearn.feature_extraction.text import TfidfVectorizer

2. Configuring Logging

# Configure logging
logging.basicConfig(level=logging.INFO)

3. Data Preparation

a. User-Item Interaction Matrix

Step 1: Data Preparation

interaction_data = [
    [1, 0, 3, 0],
    [0, 2, 0, 4],
    [3, 0, 0, 5],
    [0, 1, 2, 0],
]
interaction_matrix = csr_matrix(interaction_data)

b. Defining User and Item IDs

user_ids = [0, 1, 2, 3]  # Zero-based indexing
item_ids = [0, 1, 2, 3]  # Zero-based indexing

c. Item Features for Content-Based Filtering

item_features = [
    ["action", "thriller"],
    ["comedy"],
    ["action", "adventure"],
    ["documentary"],
]

4. TF-IDF Feature Extraction

item_features_str = [" ".join(features) for features in item_features]
vectorizer = TfidfVectorizer()
tfidf_matrix = vectorizer.fit_transform(item_features_str)

5. Initializing Recommenders

a. Content-Based Recommender

tfidf_recommender = TFIDFRecommender(feature_matrix=tfidf_matrix)

b. Collaborative Filtering Recommender

collaborative_recommender = MatrixFactorizationRecommender(
    num_factors=20,
    learning_rate=0.01,
    reg_user=0.02,
    reg_item=0.02,
    epochs=20,
    early_stopping_rounds=5,
    n_threads=4
)

6. Fitting the Models

collaborative_recommender.fit(interaction_matrix, user_ids, item_ids)
tfidf_recommender.fit(data=None)  # Adjust based on actual implementation

7. Creating a Hybrid Engine

hybrid_engine = HybridEngine(
    collaborative_engine=collaborative_recommender,
    content_engine=tfidf_recommender,
    alpha=0.5
)

8. Generating Recommendations

a. Recommendation Function

def get_recommendations(user_id: int, exclude_items: Optional[List[int]] = None, top_n: int = 10) -> List[int]:
    recommendations = hybrid_engine.recommend(user_id, top_n=top_n, exclude_items=exclude_items)
    return recommendations

b. Example Usage

user_id = 0  # Example user ID
exclude_items = [0, 1]  # Items to exclude
recommended_items = get_recommendations(user_id, exclude_items, top_n=2)
print(f"Recommended items for user {user_id}: {recommended_items}")

Expected Output:

Recommended items for user 0: [2, 3]

🛠️ Handling Common Issues

TypeError: Can’t instantiate abstract class

Solution: Ensure all abstract methods from MatrixFactorizationBase are implemented.

Numpy AxisError: axis 1 is out of bounds

Solution: Convert feature_matrix to a 2D array using .toarray() if needed.

🧩 Extending the Example

•	Incorporate Advanced Recommenders: Implement ALS or deep learning-based models.
•	Feature Engineering: Use embeddings or metadata for richer features.
•	Scale: Optimize for larger datasets with parallel processing.
•	Metrics: Add Precision@K, Recall@K, etc.
•	Deployment: Integrate into a web application or API.

🔗 Additional Resources

•	[CoreRec Documentation](https://github.com/vishesh9131/CoreRec/discussions/new?category=show-and-tell#)
•	[Matrix Factorization Concepts](https://github.com/vishesh9131/CoreRec/discussions/new?category=show-and-tell#)
•	[TF-IDF Explained](https://github.com/vishesh9131/CoreRec/discussions/new?category=show-and-tell#)
•	[Scikit-learn’s TfidfVectorizer](https://github.com/vishesh9131/CoreRec/discussions/new?category=show-and-tell#)

💬 Join the Discussion

Questions or feedback? Contribute to the discussion or open an issue on CoreRec’s GitHub.