[SOLVED] twitter/facebook comments classification into various categories

Issue

I have some comments dataset which I want to classify into five categories :-

jewelries, clothes, shoes, electronics, food & beverages

So if someones talking about pork, steak, wine, soda, eat : its classified into f&b

Whereas if somebodys talking about say – gold, pendent, locket etc : its classified into jewelries

I want to know , what tags/tokens should I be looking for in a comment/tweet so as to classify it into any of these categories. Finally which classifier to use. I just need some guidance and suggestions , Ill take it from there.

Please help. Thanks

Solution

Well this is kind of a big subject.

You mentioned Python, so you should have a look at the NLTK library which allows you to process natural language, such as your comments.

After this step, you should have a classifier which will map the words you retrieved to a certain class. NTLK also have tools for classification which is linked to knowledge databases. If you are lucky, the categories you are looking for are already available; otherwise you may have to build them yourself. You can have a look at this example which uses NTLK and the WordNet database. You can have access to the Synset, which seems to be pretty broad; and you can also have a look at the hypersets (see for example list(dog.closure(hyper)) ).

Basically you should consider using a multiclassifier on the whole tokenized text (comments on Facebook and tweets are usually short. You might also decide to only consider FB comments below 200 characters, your choice). The choice of a multiclassifier is motivated by non-orthogonality of your classification set (clothes, shoes and jewelries can be the same object; you could have electronic jewelry [ie smartwatches], etc.). This is a fairly simple setup but it’s an interesting first step, whose strengths and weaknesses will allow you to iterate easily (if needed).

Good luck!

Answered By – Flavian Hautbois

Answer Checked By – Pedro (BugsFixing Volunteer)

Leave a Reply

Your email address will not be published.