Jan 10, 2013

Smart TV and Data Mining

Smart TVs are coming. In this year's CES conference, the largest consumer electronics show each year in Las Vegas, every TV manufacturer is bragging about the "smartness" of their TV. All the TVs are now have Internet connectivity, have camera, display content from your smart phone, and play movies from youTube and Netflix. Even more, smart TVs have connection to Facebook, allow you to make Skype calls, and answers you voice command like Siri does on the iPhone.

The most interesting function for me is their capability of making movie recommendations.
 This means every TV has to remember and capture user’s viewing history, and possibly combine with the server’s knowledge about other viewers’ view data. All the information will be sent and stored in  TV maker's server. They will chunk the data, and build movie recommendation engine exactly like Netflix has done. 

This is a big advance for data mining. This is possible only after TV is connect to the Internet and store data in the cloud. Traditional TV does not have the computing power nor enough memory to do data mining. 

Now we are seeing data mining coming to our home. 

Jan 9, 2013

Product Recommendation by Amazon

Amazon is quite secretive about its technology. While researchers from other companies publish many papers on their approaches, we seldom see papers from Amazon. However, we can still infer about its technology by looking at their products in action. In this blog, we take a look at how Amazon makes recommendation to it users.

Users who purchase on Amazon typically get the following two "recommendations" at the bottom of product page: (1)"Frequently bought together" and (2) "Customers who bought this item also bought".

Strictly speaking, showing what are frequent bought together does not require complex recommender systems. This is a simple counting of frequent itemsets, a very fundamental technique taught on the first day of data mining class. This technique is also called Frequent Pattern Mining. The key challenge here is getting all of those "bought together" sets quickly from billions of transactions.

However, Amazon does provide more personalized recommendation, similar to what Netflix does. After you log in, there is a"recommended for you" page where Amazon's recommendation engine is in full action.

How does it Amazon make this recommendation? Based on a 2003 article published on IEEE Internet Computing (Jan/Feb issue), the company uses item-item similarity methods from collaborative filtering. At that time, this was state-of-the-art method, and Amazon was pioneering the field of recommender system.

It seems this same implementation has been in use in Amazon until today. As we can see from the picture to the left. The snapshot was taken in January 2013.  The first product (Girl’s 7-16 Jacket) is recommended because the user purchased a somewhat similar item (Girls 2-6x Princess Jacket). Similar thing is true for the second recommended product. In other words, item-item similarity is a major technology used by Amazon's recommendation engine.  

The field of recommender systems have seen great advance since 2006's Netflix Prize contest. From 2006 until 2009 when the prize was awarded,  many methods have been invented to tackle recommendation problem. Among them, the most widely adopted method today is Matrix Factorization (with SVD as a special implementation). It was shown that this method generated better results than item-item similarity approach. Netflix adopted matrix factorization method after 2007, and has been using it in its production system until today. 

Both item-item similarity and matrix factorization approach have been eclipsed by other approaches in the last 2 years. Netflix itself has moved into a machine-learning based ranking model, and others (such as getJar, see an early blog) have explored neighborhood-based methods.

Would Amazon adopt more sophisticated methods to make its recommendation? Given its business is doing so well with simple methods, this probably will not happen soon.

Jan 3, 2013

Applications of Supervised learning

Machine learning, specifically supervised learning (which is used equivalent to classification), has become so versatile that it can be applied to a wide range of situations. On the surface, binary classification does not sound that interesting -- It only gives a “yes” or “no” answer.  But it gains power when you can associate a probability with each “yes” or “no” answer. With probability, you can score people based on their likelihood of buying, likelihood of defaulting, or likelihood of churning.

Thus machine learning becomes truly powerful when it is “statistical learning”, where our prediction of “yes” or “no” is associated with a probability (a number ranging from 0 to 1, which can be scaled to generate a score). 

Here are some applications we can apply binary classification to:

1. Click prediction, which can be applied to (1) search ranking (2) online ads ranking
     If you can rank the probability of user click, then you can present the search results based on that probability. The documents or products what have high probably of being clicked will ranked higher. Similarly, when we decide which display ads toshow to the user, we can use click probability. 

2. Fraud detection
     Our task is deciding whether a transaction is fraud. This is a simple binary classification: fraud or not? When the probability crosses a threshold (say 0.8), we can classify a transaction as a fraud.

3. Select sales prospects to call based on their probability of responding.
    This is also a simple binary classification problem: Predict whether a prospect will respond to not.

Additional applications of binary classification are: 
4.   Accept or reject an application for  loan or credit card, or an insurance claim
5.    Customer churn
6.    Most valuable customer
7.    Cancer detection
8.    Quality control: Car is good to go out or not
9.    Product category classification
10.    User type classification
11.    Sentiment classification in social media: Positive or negative? 
12.    Document classification (applied in legal search, LinkedIn recommendation, and web search) 

Thus data mining field is now dominated by methodology of machine learning. Many people equate data mining to machine learning. This is not surprising given the wide applications we see.