Most of the recommendation techniques use the historical data of customers and their actions such as views, purchases, reviews etc. There are mainstream methods such as collaborating filtering and content based filtering. There are also the newer methods (some of them are variations of the existing methods) such as cluster models, search-based methods and item-to-item collaborative system.
Collaborative Filtering:
It emphasizes mostly on the user behavior and creates an vector for sentiment of the actions of the users with respect to the multiple items that he corresponding user is associated with. Using this type of vectors for various customers, the engine computes two closely matched customers. This matching can be as simple using the cosine of angle between the two vectors or by using machine learning algorithms such as KNN. The major disadvantage of this method is that it is expensive. In a worst case scenario, the complexity would be O(nm), where m is the number of customers and n is the set of items associated with each customer.
Cluster Models:
In these types of models, instead of treating the customers individually, they are grouped and the recommendations are similar to the people belonging to the same group. This reduces the complexity of the model considerably in comparison with the traditional filtering. This also highly useful when the database of the customers become too large.3
Item-to-Item Filtering:3
This method focuses more on the similarities between items rather than the similarities between customers and hence a matrix of products is created and the data will the similarity measures between those products. So when the system knows the purchasing trends of the customer, it picks the set of similar products from the product similarity matrix. The key to this method is that the product similarity matrix can be created offline and thence the performance of this method will be better than the traditional collaborative filtering. The ways to compute the item similarity are as follows,
- Pearson r correlation based similarity
- Cosine based similarity
- Adjusted cosine similarity
Listed below are some of the key challenges faced while building recommender systems:
1. Insufficient Data
The key to a good recommender system is the access to data. One of the biggest challenges that recommender systems are facing is that they need a lot of data to make effective recommendations.
The working of a recommendation system in general is illustrated below.
The first step is to obtain data either through a form or a catalog and then analyze user behavior. Following this the algorithms described in the previous sections are applied to get recommendations.
As we can infer from this process, the more item and user data the system has access to better the recommendations provided. The recommendation success stories of companies like Google, Amazon, and Netflix etc. which have access to abundant user is a testimony to this fact.
However this problem proves to be a vicious circle. In order to get good recommendations, you need a lot of users, so you can get a lot of data for the recommendations [1].
2. Changing Trends
Another challenge that recommender systems are being faced with is that the past behavior of the user need not necessarily be indicative of their current preferences. As trends change, recommendation systems will have a hard time keeping up.
Let us consider the example of the fashion industry. In this context providing item based recommendations is not feasible because there are too many aspects to consider, such as the size, texture, fabric, brand etc. Moreover, each of these aspects are subjective and each customer may prioritize them differently. Any algorithmic approach will struggle to cope with this.
3. Shift in User Intentions
A key consideration that recommender systems do not take into account is that user preferences and their intention might vary. For instance one day a user might visit Amazon to purchase a book for himself. On another day the user might visit amazon to purchase a gift for a friend.
Recommendation systems cannot be built to consider these factors. This could impact the efficiency of the recommendations provided.
4. The X Factor
In our previous blog on the major players in the field of recommendation systems we described the Netflix challenge where a prize money of one million was offered to developers who could come up with a collaborative filtering algorithm.
An issue faced during this challenge, and in the broader context of providing recommendations is related to eccentric items [1]. These are items that users have strong preferences towards, they either totally love or absolutely hate them
For instance in the Netflix challenge, movies like ‘Altered States’ and ‘Dune’ for hard to provide recommendations on. In such cases predicting user behavior is an imposing prospect.
References :
[1]http://readwrite.com/2009/01/28/5_problems_of_recommender_systems#awesm=~oB097DxOjVhKF4
[2] limitations of recommendation systems : http://tmsp.umd.edu/position%20papers/riedl-statement.pdf
[3]http://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf
Posted by prempv | Filed under 05_Recommendation engines