Pages

Jul 8, 2013

Text Mining: Named Entity Detection

An interesting task of text mining is detecting entities in the text. Such entities could be a person, a company, a product, or a location. Since an entity is associated with a special name, it is also called Named Entity. For example, the following text contains 3 named entities:
       Apple has hired Paul Deneve as vice president, reporting to CEO Tim Cook.
The first term “Apple” indicates a company, and the second and third are persons.

Named entity detection (NER) is an important component in  social media analysis. It helps us to understand user sentiment on specific products. NER is also important for product search for E-commerce companies. It helps us to understand user search query related to certain products.

To map each name to an entity, one solution is using a dictionary of special names. Unfortunately, this approach has two serious problems. The first problem is that our dictionary is not complete. New companies are created and new products are sold every day. It is hard to keep track all the new names. The second problem is the ambiguity of associating a name to an entity. The following example illustrates this:
As Washington politicians argue about the budget reform, it is a good time to look back at George Washington’s time.
In this text, the first mention of “Washington” refers to a city, while the second mention refers to a person. The distinction of these two entities comes from their context.

To resolve ambiguity in entity mapping,  we can create certain rules to utilize the context. For example, we can create the following rules:
  1. When ‘Washington’ is followed by ‘politician’, then it refers to a city.
  2. When ‘Washington’ is preceded by ‘in’, then it refers to a city.
  3. When ‘Washington’ is preceded by ‘George’, then it refers to a person.
But such rules could be too many. For example, each of the following phrases would generate a different rule: “Washington mentality”, “Washington atmosphere”, “Washington debate” as well as “Washington biography” and “Washington example”. The richness of natural language makes the number of rules exploding and still susceptible to exceptions.

Instead of manually creating rules, we can apply machine learning. The advantage of machine learning is that it creates patterns automatically from examples. No rule needs to be manually written by humans. The machine learning algorithm takes a set of training examples, and chunk out its own model (that is comparable to rules). If we get new training data, we can re-train the machine learning algorithm and generate a new model quickly.

How does the machine learning approach work? I will discuss it in the next post. 

132 comments:

  1. Thank you for this post. Very interesting.

    ReplyDelete
  2. I think it is amazing that the search engine will try to interpret and answer my question. It sounds like program that was playing jeopardy. What was it called? Watson?
    As the technology grows it will reduce the really odd search results that pop up. To see more info please visit http://essayhogwarts.com/dissertation-proposal/.

    ReplyDelete
  3. Interesting post. I will wait for your next post on machine learning.

    ReplyDelete
  4. After I originally left a comment I seem to have clicked on the -Notify me when new comments are added- checkbox aand now whenever a comment is added I recieve 4 emaols with the same comment.
    Perhaps there iis a means you are able to remove me from that service?
    Many thanks!

    Also visit my homepage ... bristle hair btush - -

    ReplyDelete
  5. Sometimes single mothers and couples may encounter such situations wherein childhood
    becomes impossible for them. Try to think of
    what areas in your neighborhood could use a little bit
    of sprucing up - As you try to decide what services are needed, you
    can actively advertise and promote it to try to get others involved.

    Closely linked to this strategy is the technology strategy.


    Feel free to surf to my blog post; Adoption Network
    Law Center webblog - -

    ReplyDelete
  6. This article has covered the topic quite well. Very informational and interesting. Thanks for sharing this knowledge with us.

    new year, new year images, new year wallpaper, new year quotes, new year wishes, new year sms, new year greetings, whatsapp status

    ReplyDelete
  7. We are offering website services & web design services for your business website also offering E-commerce website....

    ReplyDelete
  8. Hello. Now You can share mother's day quotes,images & messages.
    Mother's day quotes
    Mothers day messages
    Happy mother's day messages

    ReplyDelete
  9. In the particular article You might understand how for getting pay per click campaign started building net applications

    ReplyDelete
  10. The war between humans, orcs and elves continues earn to die . Lead your race through a series of epic battles, using your crossbow to fend off foes and sending out units to destroy castleshappy wheels . Researching and upgrading wisely will be crucial to your success! There are 5 ages total and each one will bring you new units to train to fight in the war for you cause.
    earn to die 2
    Whatever you do, don’t neglect your home base because you cannot repair it and once it is destroyed, you lose! Age of War is the first game of the series and really sets the tone for the Age of War games . Also try out the Age of Defense series as it is pretty similar.
    In this game, you start at the cavern men’s age, then evolvetank trouble ! There is a total of 5 ages, each with its units and turrets. Take control of 16 different units and 15 different turrets to defend your base and destroy your enemy.
    The goal of the game also differs depending on the level. In most levels the goal is to reach a finish line or to collect tokens. Many levels feature alternate or nonexistent goals for the player. The game controls are shown just under gold miner. Movement mechanisms primarily include acceleration and tilting controls. cubefield
    It consists of a total of 17 levels and the challenge you face in each level increases as you go up. unfair mario The game basically has a red ball that has to be moved across the various obstacles in its path to the goal. slitherio

    ReplyDelete

  11. Great! Thanks for sharing the information. That is very helpful for increasing my knowledge in this fiel
    Red Ball | | duck life | Slitherio
    Red Ball 2 | Red Ball 3 | Red Ball 4

    ReplyDelete
  12. good Preserve in head if you carry out the guidelines in Click Here you with interesting content material. nice.

    ReplyDelete
  13. good all the add-ons you’d prefer, hence by using a build will sportsdevil not working It is usually compatible with multiple operating devices and great.

    ReplyDelete
  14. good creative way we need to present. In simple way official website link your google android right now !! We give you the formal VivaVideo Download Free nice.

    ReplyDelete

  15. Wonderful blog! I found it while searching on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Many thanks.
    2048 game | five nights at freddy's 4 | five nights at freddy's 3 | fireboy and watergirl | fireboy and watergirl 4||red ball | age of war

    ReplyDelete
  16. Can one jailbreak iOS 10 / iOS 10.0.2 / 10.0.1? If not, what is latest on iOS 10 / 10.0.2 jailbreak status for iPhone, iPad and iPod touch devices? you can get answer of this Question as now you can get iOS 10 jailbreak.

    ReplyDelete
  17. good you can hook up your cell to your Computer either Home windows or Xender PC Download You can hook up your clever mobile phone to Computer applying nice.

    ReplyDelete
  18. Visit us now to view & share beautiful collection of Best Images of
    happy chocolate day 2017

    ReplyDelete
  19. good all these functions you most likely want this application ITube with a couple of tricks by which we could nice.

    ReplyDelete
  20. Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write.
    Thanks for sharing !

    ReplyDelete
  21. Great blog here with all of the valuable information you have. Keep up the good work you are doing here.
    Easter Images
    mothers day poems
    Fathers Day greetings
    birthday wishes for sister
    Wishes Quotes

    ReplyDelete
  22. good info with all your close friends about how you can https://vivavideodownload.co the APK file and saving it in your android storage. best.

    ReplyDelete

  23. Very well written admin and I love to read more info, so keep updating we will happy to visit again and again to your blog... Best
    French Open

    ReplyDelete
  24. The best collection of Happy Mothers Day 2017 Images, Photos, Pictures, Wallpapers, Mothers Day Quotes, Wishes, Messages, Greetings,
    Happy Mothers Day Poems & Sayings, Mothers Day Messages, SMS, Text Messages,
    happy fathers day images
    happy fathers day image
    fathers day pictures 2017
    fathers day cards 2017
    father day images free
    happy fathers day quotes
    happy fathers day message
    happy fathers day to my husband
    happy fathers day wishes
    father's day quote
    eid mubarak 2017
    Mothers Day WhatsApp Status & Facebook Status, DP Images,
    Mothers Day FB Profile Pics, FB Timeline Cover Photos etc.

    ReplyDelete
  25. Web Cam Sex Chat with live cams girls largest Adult Sex Cams community and chat, Uncensored Adult Chat. Start chatting with amateurs, exhibitionists, pornstars, Cam girls are ready for live sex via cam. webcam chat for adults with flirty chicks. Join us and get much more in private live camsex!

    ReplyDelete
  26. Thumbzilla xxx videos Watch the best hottest thumbzilla videos porn on the internet via our website and blog. We publish new fresh pics videos and blog articles every day. You are welcome to stay here and enjoy our production of xxx porno.

    ReplyDelete
  27. porn toys Light up your hot passion with so many sex toys both for her and for him ... do not make sex a dull thing....

    ReplyDelete
  28. Kratom Canada Buy kratom and ethnobotanicals in Canada with free shipping! Wholesale, bulk ethnobotanicals and entheogens in Toronto, Vancouver, Montreal. Canada Kratom

    ReplyDelete
  29. JAV VR Watch 100s of full Japanese Adult Videos streaming at JAVRedlight.com including in VR! Just what you want on demand.

    ReplyDelete
  30. JAV VR Watch 100s of full Japanese Adult Videos streaming at JAVRedlight.com including in VR! Just what you want on demand.

    ReplyDelete



  31. pakistani young ladies dependably take genuine about her wellness with this they are accomplished

    and mannered escorts.Indian Escorts in

    Dubai
    you can get your most loved young ladies since we have distinctive sort of escorts in

    dubai people groups need to invest their energy with pakistani escorts for the most part high

    upper class need administrations from pakistani young ladies.

    https://www.scoop.it/t/dubai-escorts-by-anjalimodel9
    https://vk.com/dubaiescorts.verified
    https://twitter.com/MyDubaiModel
    http://dubaicallgirls.co/indianescorts.html

    call and whatsapp number +971561616995

    ReplyDelete
  32. Welcome to Dubai Escorts Agency.
    Pakistani Escort In Dubai
    Dubai Escorts
    Call Girls In Dubai
    Indian Escort In Dubai
    https://vk.com/dubaiescorts.verified
    https://www.scoop.it/t/dubai-escorts-by-anjalimodel9
    https://twitter.com/MyDubaiModel

    ReplyDelete
  33. Indian And Pakistani Escort Girls In Dubai Companions and Elite Sweetheart Experience Administrations.
    Regardless of whether The Model Girls are Really at That Level or Recently Advertised That Way,
    The Fortunate Respectable Who Have This Gigantic Choice of Cute Models.
    Indian Escort In Dubai
    https://vk.com/dubaiescorts.verified
    https://www.scoop.it/t/dubai-escorts-by-anjalimodel9
    https://twitter.com/MyDubaiModel

    ReplyDelete
  34. I like the post format as you create user engagement in the complete article. It seems round up of all published posts. Thanks for gauging the informative posts.
    cara menggugurkan kandungan

    ReplyDelete
  35. giay da bong gia re Với phương châm “Mong 1 người đến vạn lần, hơn vạn người đến 1 lần”, Giaydabongtot.com không ngừng nỗ lực, cải tiến đáp ứng nhu cầu mua giày đá bóng của khách hàng. Giaydabongtot.com đảm bảo sẽ cung cấp sản phẩm chân thực so với mô tả ở trên Website

    ReplyDelete
  36. tu bep nhua Chúng tôi là đơn vị hàng đầu trong phân phối các sản phẩm bếp điện từ, tủ bếp nhựa chất lượng với giá thành hợp lý nhất hiện nay. Các sản phẩm của chúng tôi được khách hàng đánh giá rất cao và tin tưởng sử dụng nhiều

    ReplyDelete
  37. mu noi nam Mũ Nồi Xinh là thương hiệu mũ nồi hàng đầu tại Việt Nam. Chúng tôi đã cung cấp hàng triệu sản phẩm chất lượng cho thời trang Việt. Đến với chúng tôi bạn sẽ được tư vấn lựa chọn cho mình mẫu mũ nồi phù hợp nhất với mình

    ReplyDelete
  38. mũ lưỡi trai Khiêm xách tay chuyên các loại mũ lưỡi trai, nón snapback, nón kết, .... chất lượng hàng đầu, mẫu mã đa dạng phù hợp với phong cách sống của nhiều người. Hãy đến ngay với chúng tôi để được tư vấn cũng như lựa chọn cho mình loại mũ phù hợp nhất

    ReplyDelete
  39. mature cam girls VibraToy.Online is absolutely free to use for everyone and all new members get 10 credits free. Since the introduction of the new live adult chat service, it has become a hive of hot and horny conversations. It has some of the best milf chat options for members, which has fetched it a high viewership in the course of time.

    ReplyDelete
  40. email shemales The site is a site with the theme of men meeting Shemales (transsexuals) to email and hookup with.

    ReplyDelete
  41. indian escorts london Kensington Girls has the best selection of Indian escorts for you in London. These escorts are extremely open minded and love what they do. Exotic escorts are exactly what these lovey ladies are, they like to have fun and are only having fun if you are as well.

    ReplyDelete
  42. This site Find Las Vegas female escorts in Las Vegas at ListedEscorts.com with new verified listings posted daily, including pics, prices, reviews and extra filters. Call for hottest and the most beautiful Las Vegas escorts available 24/7 direct to your room in 30 minutes. Search the Largest directory of call girls in Las Vegas and have fun in the sin city of the world. You deserve the best experience for your money and it is our goal to make this happen. If you’re looking for quality companionship during your time in Las Vegas, you need look no further.

    ReplyDelete
  43. rent a man My name is Manny and I am an independent male escort, based in Perth, available for today's professional and modern women. I'm well spoken with impeccable tastes and a great conversationalist. I pride myself on being polite, courteous, romantic and offering first class client satisfaction. If you are the type of mature woman that wants to feel loved and appreciated, either after a long hard day of work or simply because you believe you deserve that special treatment, then you have come to the right place! I offer all intimate male escort and gigolo services a woman may desires.

    ReplyDelete
  44. transgender girlfriends Do you want to know how to meet T-Girls? Do you find your efforts are often misunderstood by the T-Girls you contact? Perhaps you joined a transgender dating site but can’t get a reply. No need to worry. I have a unique understanding of what makes t-girls and their admirers tick. Often, it is just a matter of semantics or fine-tuning the approach that will make the difference with your transgender romantic efforts.

    ReplyDelete
  45. local sexts learn how to simply text girls and get them to have sex with you. It’s true, it can really be done. It’s actually a lot easier than you might imagine. I know personal, meeting girls online and then texting (sexting) them can lead to hot sex. There’s nothing hotter than meeting acute girl online and then fucking her the same night.. Maybe you don’t roll that way, maybe you want to settle down and fall in love.

    ReplyDelete
  46. transgender social network TGGuide.com's Transgender Chat and Social Network provides free social and support services for the trans community including transwomen, transmen, supporters and anyone who self-identifies on the transgender spectrum.

    ReplyDelete
  47. Acompanhantes Porto Alegre Acompanhantes Porto Alegre - Prazer360 - o site das acompanhantes de Porto Alegre, Garotas de programas, guia noturno e de motéis.....

    ReplyDelete
  48. make money adult site Adult website providing free XXX adult videos; free live adult movie theater; free live XXX adult cams; free eBooks on making money in the adult industry; free adult XXX games; adult XXX product reviews.

    ReplyDelete
  49. domino qq terbaik Vegas, the program follows Mariano's attempts to become a professional poker player and features professional player Daniel Negreanu tutoring him. The

    ReplyDelete
  50. hot lesbian porn videos Over 86 millions of best porn movies ever with most important xxx porn categories and hottest pornstars fucked hard and sucks big cock . Biggest porn tube starring amazing horny sluts with hungry pussies needs deep hard penetration . Naughty Teens , Horny Milfs , Sexy Matures or Kinky Shemales taking big hard cock in mouth , pussy and asshole .

    ReplyDelete
  51. Hardcore free porn videos amateur anal lesbian Asian hentai

    ReplyDelete
  52. Porn tube Young teen and amateur porn videos...

    ReplyDelete
  53. We deliver professional web design and website development agency service, offering excellent bespoke website design and development. We use WordPress and WooCommerce as standard. On demand, we also use for E-Commerce Magento in delivering excellent web design and build for eCommerce sites.

    if you want more just look here "web design"

    ReplyDelete
  54. Android application is one among the numerous applications intended for text to audio. Also Blackberry has its own particular application for a sheltered drive which is helping in limiting the street mischances. So hold up not and get an application for yourself too!from text to speech

    ReplyDelete
  55. Thanks for this wonderful information. You posts are really nice and interesting. Writing style is also very amazing Beautiful Independence Day Wallpapers

    ReplyDelete
  56. merchant account for pharmacy IPAYTOTAL provide online pharmacy merchant accounts to online pharmacy websites and online supplement websites. We have lots of experience in the field of high-risk merchant accounts and we know how to do best for your business to provide you with the best credit card processing rates and terms.

    ReplyDelete
  57. Kartal Escort Bayan The best quality Istanbul Kartal dating website. Please visit us to say hello.

    ReplyDelete
  58. bitcoin merchant services We provide merchant accounts for companies who want to accept bitcoin on their website from their customers . We want to let all crypto currency users know about our services and what we can offer them.We offer a checkout solution designed to accommodate Bitcoin where customers checkout in cryptocurrency, automatically convert received payments to USD or EUR and withdraw it to your bank account.

    ReplyDelete