Nearest-neighbor collaborative filtering provides a successful means of generating recommendations for web users. However, this approach suffers from several shortcomings, including data sparsity and noise, the cold-start problem, and scalability.
In this work, we present a novel method for recommending items to users based on expert opinions. Our method is a variation of traditional collaborative filtering: rather than applying a nearest neighbor algorithm to the user-rating data, predictions are computed using a set of expert neighbors from an independent dataset, whose opinions are weighted according to their similarity to the user. This wisdom of the few method promises to address some weaknesses in traditional collaborative filtering, while maintaining comparable accuracy.
We validate our approach by predicting a subset of the Netflix data set. We use ratings crawled from a web portal of expert reviews [Rotten Tomatoes], measuring results both in terms of prediction accuracy and recommendation list precision.
Finally, we explore the ability of our method to generate useful recommendations, by reporting the results of a user-study where users prefer the recommendations generated by our approach.
[Keywords: collaborative filtering, cosine similarity, experts, nearest-neighbors, recommender system, top-n recommendations]
…4. Results: Based on the previously described data, we measure how well the 169 experts predict the ratings of the 10,000 Netflix users. In order to validate our approach, we set up two different experiments: in the first experiment, we measure the mean error and coverage of the predicted recommendations. In the second experiment, we measure the precision of the recommendation lists generated for the users.
| Method | MAE | Coverage |
|---|---|---|
| Critics’ Choice | 0.885 | 100% |
| Expert-CF | 0.781 | 97.7% |
| Neighbor-CF | 0.704 | 92.9% |
…As shown in §4, our approach does not outperform a “naive” neighbor-CF approach. However, our focus when using external data sources is not as much on prediction accuracy as it is on addressing some of the common problems found in traditional CF recommender systems. In the next paragraphs, we shall describe a few of these problems and discuss how they might be addressed by using a limited set of external experts to generate the predictions (ie. ‘wisdom of the few’):
-
Data Sparsity: In a standard collaborative recommender system, the user-rating data is very sparse. Although dimensionality reduction techniques offer some help, this problem is still a source of inconsistency and noise in the predictions. Using the wisdom of the few addresses this issue since domain experts are more likely to have rated a large percentage of the items, as shown in Section 2.
-
Noise and Malicious Ratings: Users introduce noise when giving their feedback to a recommender system, both in the form of careless ratings21 and malicious entries,22, 23 which will affect the quality of predictions. Experts are expected to be more consistent and conscious with their ratings, thus reducing noise. In addition, an expert data set can be immune to malicious, profile-injection attacks as it is an easy to control and stable data set.
-
Cold Start Problem: In a CF system, new items lack rating data and can not be recommended; the same is true when a new user enters the system.24 Motivated expert users typically rate a new item entering the collection as soon as they know of its existence and therefore minimize item cold-start. In addition, experts should create a less sparse and noisy dataset which should improve the user cold-start problem, as shown in our user study.
-
Scalability: Computing the similarity matrix for N users in an M-item collection is an 𝒪(N2M) problem. This matrix needs to be updated on a regular basis, as new items and/or users enter the system. Therefore, CF based approaches typically suffer from scalability limitations. While there are several ways to address this issue—such as k-means clustering,25 scalability is still an open research problem in CF systems. The wisdom of the few approach is less sensitive to scale, as it creates recommendations from a very reduced set of experts (eg. 169 experts vs. 500,000 potential neighbors in the Netflix database).
-
Privacy: Privacy in CF recommender systems is a growing concern and still an area of research.26 In order to maintain and update the similarity matrix, the system has to transmit all user ratings to a central node where the matrix is computed. This step is not needed in our approach, since the similarity matrix only includes expert data and the target user. In expert-CF, the current experts ratings can be easily transmitted thanks to the reduced size of the matrix, such that all computation is performed locally on the client. This advantage is particularly relevant in a mobile scenario.