Improving Topical Social Media Sentiment Analysis by Correcting Unknown Words Automatically

Abstract

In the digital world, social media has become one of the most popular communication mediums that allow users to share their views on various topics in their social network. For example, Twitter users are allowed to share their thoughts on various topic by sending tweets with a maximum length of 140 characters. Hence, social media driven information contains opinions and sentiments on various topics of interest which are extremely useful for companies to design marketing strategies. Sentiment Analysis is widely used to assist people to understand the massive amount of data available online and identify the polarity of the topical based social media opinions. However, social media platforms’ users come from all over the world and have variation in terms of informal language and short notation used on social media platforms. Therefore, the identification on the polarity of topical social media has become more challenging and the accuracy on the polarity of topical social media opinions might be influenced. This paper investigates the effectiveness of applying different spelling correction algorithms, such as Levenshtein distance and Peter Norvig’s algorithm for spelling correction of unknown words found in social media such as Twitter, before carrying out sentiment analysis. The evaluation of spelling correction algorithms on sentiment analysis is carried out by comparing the polarities of manually annotated tweets with the polarities obtained from the sentiment analysis algorithm. Based on the results obtained, there are slight improvements in term of percentage of matched polarity, where 1.6% improvement by using the Levenshtein distance-based algorithm and 2.0% improvement by using the Peter Norvig’s algorithm.