Abstract
Literature has shown the prominence of extracting social media data to understand public opinion. However, there are little works on how these opportunities can be realized and the challenges in exploiting the opportunities in the transportation industry. Further, data quality and availability using social media may vary according to different demographics due to population size and languages used. Additionally, most of the related prior studies that show the opportunities of using social media data were conducted in North and South America. With this proposition, we seek to investigate the challenges of using Twitter data with text mining techniques for understanding users’ opinions and sentiment through a case study of using the data to assess public transportation service performance specifically in a Malaysian context. Our findings indicate that social media data can only be useful in generating reasonable insights if users could input informative words for forming discussed topics to derive opinion, and incline towards a certain sentiment with adjectives. The findings also identified the need for a more proficient dictionary to classify multilingual tweets. Our research provides original evidence proving the potential of using social media data to assess public transportation services performance which may vary depending on the demographics of the social media users.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Windasari, I.P., Uzzi, F.N., Satoto, K.I.: Sentiment analysis on Twitter posts: an analysis of positive or negative opinion on GoJek. In: 4th International Conference on Information Technology, Computer, And Electrical Engineering. IEEE 2017, October 2017
Almselati, A., Rahmat, R., Jaafar, O.: An overview of urban transport in Malaysia. Soc. Sci. 6(1), 24–33 (2011)
Borhan, M., Syamsunur, D., Mohd Akhir, N., Mat Yazid, M., Ismail, A., Rahmat, R.: Predicting the use of public transportation: a case study from Putrajaya, Malaysia. Sci. World J. 2014, 1–9 (2014)
Hovy, D., Johannsen, A., Søgaard, A.: User review sites as a resource for large-scale sociolinguistic studies. In: Proceedings of the 24th International Conference on World Wide Web, pp. 452–461, May 2015
Cozza, V., Petrocchi, M., Spognardi, A.: Mining implicit data association from Tripadvisor Hotel Reviews. In: EDBT/ICDT Workshops, pp. 56–61, March 2018
Collins, C., Hasan, S., Ukkusuri, S.: A novel transit rider satisfaction metric: rider sentiments measured from online social media data. J. Public Transp. 16(2), 21–45 (2013)
Schweitzer, L.: Planning and social media: a case study of public transit and stigma on Twitter. J. Am. Plann. Assoc. 80(3), 218–238 (2014)
Haghighi, N.N., Liu, X.C., Wei, R., Li, W., Shao, H.: Using Twitter data for transit performance assessment: a framework for evaluating transit riders’ opinions about quality of service. Public Transport 10(2), 363–377 (2018). https://doi.org/10.1007/s12469-018-0184-4
Mendez, J., Lobel, H., Parra, D., Herrera, J.: Using Twitter to infer user satisfaction with public transport: the case of Santiago, Chile. IEEE Access 7, 60255–60263 (2019)
Hoffer, J.A., Venkataraman, R., Topi, H.: Modern database management (Global Edition). ISBN-13: 978–1292263359. Pearson. 2019 (2019)
Manning, C.D., Schütze, H., Raghavan, P.: Introduction to Information Retrieval. Cambridge University Press (2008)
Casas, I., Delmelle, E.: Tweeting about public transit — Gleaning public perceptions from a socialmedia microblog. Case Stud. TransportPolicy 5(4), 634–642 (2017)
Statista: Twitter: number of monthly active users 2010–2019 (2019). https://www.statista.com/statistics/282087/number-of-monthly-active-twitter-users/. Accessed 5 Jan 2020
Datareportal. Digital 2021: Malaysia. https://datareportal.com/reports/digital-2021-malaysia. Accessed 24 May 2021
‘Public transport in the Klang Valley (2019). https://en.wikipedia.org/wiki/Public_transport_in_the_Klang_Valley. Accessed 7 Jan 2020
Brabham, D.: motivations for participation in a crowdsourcing application to improve public engagement in transit planning. J. Appl. Commun. Res. 40(3), 307–328 (2012)
Evans-Cowley, J., Griffin, G.: Microparticipation with social media for community engagement in transportation planning. Transp. Res. Record J. Transp. Res. Board 2307(1), 90–98 (2012)
Grant-Muller, S., Gal-Tzur, A., Kuflik, T., Minkov, E., Shoor, I., Nocera, S.: Enhancing transport data collection through social media sources: methods, challenges and opportunities for textual data. IET Intel. Transport Syst. 9(4), 407–417 (2015)
Blei, D., Ng, A., Jordan, M.: Latent Dirichlet Allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Weng, J., Lim, E.P., Jiang, J., He, Q.: Twitterrank: finding topic-sensitive influential twitterers. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 261–270, February 2010
Yang, M., Rim, H.: Identifying interesting Twitter contents using topical analysis. Expert Syst. Appl. 41(9), 4330–4336 (2014)
Hong, L., Davison, B.D.: Empirical study of topic modeling in Twitter. In Proceedings of the First Workshop on Social Media Analytics, pp. 80–88, July 2010
‘WordNet | A Lexical Database for English’, https://wordnet.princeton.edu/. Accessed 7 Jan 2020
Ren, Y., Wang, R., Ji, D.: A topic-enhanced word embedding for Twitter sentiment classification. Inf. Sci. 369, 188–198 (2016)
Ali, F., et al.: Transportation sentiment analysis using word embedding and ontology-based topic modeling. Knowl.-Based Syst. 174, 27–42 (2019)
Qi, B., Costin, A., Jia, M.: A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services. Travel Behav. Soc. 21, 10–23 (2020)
Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., Kappas, A.: Sentiment strength detection in short informal text. J. Am. Soc. Inf. Sci. Technol. 61(12), 2544–2558 (2010). https://doi.org/10.1002/asi.21416
Abu Bakar, M., Mohd Ariff, N., Hui, E.: Exploratory data analysis of Twitter’s rhythm in Malaysia. In: AIP Conference Proceedings 2019 (2019)
Octoparse. https://www.octoparse.com/. Accessed 25 May 2021
Huberman, B.A., Romero, D.M., Wu, F.: Social networks that matter: Twitter under the microscope. arXiv preprint arXiv:0812.1045 (2008)
Acknowledgments
This paper was supported by Partnership Grant CR-UM-SST-DCIS-2018–01 and RK004–2017 between Sunway University and the University of Malaya.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Chua, H.N., Liao, A.W.Q., Low, Y.C., Lee, A.S.H., Ismail, M.A. (2022). Challenges of Mining Twitter Data for Analyzing Service Performance: A Case Study of Transportation Service in Malaysia. In: Abramowicz, W., Auer, S., Stróżyna, M. (eds) Business Information Systems Workshops. BIS 2021. Lecture Notes in Business Information Processing, vol 444. Springer, Cham. https://doi.org/10.1007/978-3-031-04216-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-04216-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04215-7
Online ISBN: 978-3-031-04216-4
eBook Packages: Computer ScienceComputer Science (R0)