A Comparative Study of Machine Learning Models for Sentiment Analysis of Dana App Reviews

Yudianto Sujana

Abstract

Sentiment analysis of user reviews has become increasingly important for mobile app developers, as it can provide valuable insights into customer satisfaction and guide the improvement of app features. In this study, we compared the performance of three machine learning models - Support Vector Machine, Neural Network, and Bidirectional Long Short-Term Memory - in classifying the sentiment of user reviews for the Dana mobile application. Our results showed that the Bi-LSTM model outperformed the other models, achieving the highest accuracy, precision, recall, and F1-score. The superior performance of the Bi-LSTM model can be attributed to its ability to capture long-term dependencies and contextual information within the review text, which is crucial for accurate sentiment analysis. These findings highlight the effectiveness of deep learning techniques in handling the complexities of language and sentiment analysis, particularly in the context of user-generated content. The insights from this study can inform the development of more accurate and efficient sentiment analysis tools for mobile app reviews, ultimately benefiting both app developers and users.

Keywords

Bi-LSTM; Deep Learning; Machine Learning; Sentiment Analysis

Full Text:

PDF

References

Alshdaifat, E., Alshdaifat, D., Alsarhan, A., Hussein, F., & El-Salhi, S. M. F. S. (2021). The Effect of Preprocessing Techniques, Applied to Numeric Features, on Classification Algorithms’ Performance. Data, 6(2), 11. https://doi.org/10.3390/data6020011

Baccouche, A., Garcia-Zapirain, B., & Elmaghraby, A. (2018). Annotation Technique for Health-Related Tweets Sentiment Analysis. 2018 IEEE International Symposium on Signal Processing and Information Technology, ISSPIT 2018, 382–387. https://doi.org/10.1109/ISSPIT.2018.8642685

Balakrishnan, V., Selvanayagam, P. K., & Yin, L. P. (2020). Sentiment and Emotion Analyses for Malaysian Mobile Digital Payment Applications. ACM International Conference Proceeding Series, 67–71. https://doi.org/10.1145/3388142.3388144

Brauwers, G., & Frasincar, F. (2023). A Survey on Aspect-Based Sentiment Classification. ACM Computing Surveys, 55(4), 1–37. https://doi.org/10.1145/3503044

Chakraborty, I., Kim, M., & Sudhir, K. (2022). Attribute Sentiment Scoring with Online Text Reviews: Accounting for Language Structure and Missing Attributes. Journal of Marketing Research, 59(3), 600–622. https://doi.org/10.1177/00222437211052500

Chollet, F., & others. (2015). Keras.

Fu, B., Lin, J., Liy, L., Faloutsos, C., Hong, J., & Sadeh, N. (2013). Why people hate your App - Making sense of user feedback in a mobile app store. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Part F128815, 1276–1284. https://doi.org/10.1145/2487575.2488202

Handani, S. W., Saputra, D. I. S., Hasirun, Arino, R. M., & Ramadhan, G. F. A. (2019). Sentiment Analysis for Go-Jek on Google Play Store. Journal of Physics: Conference Series, 1196(1), 012032. https://doi.org/10.1088/1742-6596/1196/1/012032

Hermanto, Kuntoro, A. Y., Asra, T., Pratama, E. B., Effendi, L., & Ocanitra, R. (2020). Gojek and Grab User Sentiment Analysis on Google Play Using Naive Bayes Algorithm And Support Vector Machine Based Smote Technique. Journal of Physics: Conference Series, 1641(1), 012102. https://doi.org/10.1088/1742-6596/1641/1/012102

Islam, M. R. (2014). Numeric rating of Apps on Google Play Store by sentiment analysis on user reviews. 2014 International Conference on Electrical Engineering and Information & Communication Technology, 1–4. https://doi.org/10.1109/ICEEICT.2014.6919058

JoMingyu. (2019). google-play-scraper. https://github.com/JoMingyu/google-play-scraper

Kandhro, I. A., Jumani, S. Z., Ali, F., Shaikh, Z. U., Arain, M. A., & Shaikh, A. A. (2020). Performance Analysis of Hyperparameters on a Sentiment Analysis Model. Engineering, Technology & Applied Science Research, 10(4), 6016–6020. https://doi.org/10.48084/etasr.3549

Kristiyanti, D. A., Putri, D. A., Indrayuni, E., Nurhadi, A., & Umam, A. H. (2020). E-Wallet Sentiment Analysis Using Naïve Bayes and Support Vector Machine Algorithm. Journal of Physics: Conference Series, 1641(1), 012079. https://doi.org/10.1088/1742-6596/1641/1/012079

Pai, H.-T., Lai, H.-W., Wang, S.-L., Wu, M.-F., & Chuang, Y.-T. (2017). Recommendations for mobile applications. Proceedings of the 1st International Conference on Internet of Things and Machine Learning, 1–6. https://doi.org/10.1145/3109761.3109771

Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021). Data and its (dis)contents: A survey of dataset development and use in machine learning research. Patterns, 2(11), 100336. https://doi.org/10.1016/j.patter.2021.100336

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel V. and Thirion, B., Grisel, O., Blondel, M., Prettenhofer P. and Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

Ranjan, S., & Mishra, S. (2020). Comparative Sentiment Analysis of App Reviews. 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 1–7. https://doi.org/10.1109/ICCCNT49239.2020.9225348

Ruder, S., Peters, M. E., Swayamdipta, S., & Wolf, T. (2019). Transfer Learning in Natural Language Processing. Proceedings of the 2019 Conference of the North, 15–18. https://doi.org/10.18653/v1/N19-5004

Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673–2681. https://doi.org/10.1109/78.650093

Singla, Z., Randhawa, S., & Jain, S. (2017). Statistical and sentiment analysis of consumer product reviews. 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 1–6. https://doi.org/10.1109/ICCCNT.2017.8203960

DOI: https://doi.org/10.20961/ijie.v7i2.93132