Evaluasi Komparatif Naive Bayes dan SVM pada Sentimen Berbahasa Indonesia
DOI:
https://doi.org/10.54367/jtiust.v11i1.6776Keywords:
sentiment analysis, Naive Bayes, SVM, TF-IDF, IndonesianAbstract
Purpose: This study compares Multinomial Naive Bayes and Support Vector Machine for three-class sentiment classification of Indonesian text. Design/methods/approach: A controlled synthetic dataset of 3,000 online-learning comments was generated with balanced positive, neutral, and negative labels. The data include formal and informal expressions, negation, mixed sentiment, and spelling variation. Preprocessing consisted of case folding, cleaning, normalization, stopword removal, and negation handling. Unigram and bigram features were weighted using TF-IDF. Models were tuned through five-fold cross-validation and evaluated on a stratified 20% test set using accuracy, macro precision, macro recall, macro F1-score, and confusion matrices. Findings/results: Naive Bayes achieved 82.83% accuracy and 82.82% macro F1, whereas SVM achieved 82.83% accuracy and 82.82% macro F1. The error analysis indicates that mixed-polarity sentences, negation, and neutral expressions are the most challenging patterns. Conclusions: Both models provide equivalent classification performance in this controlled experiment, although their modeling assumptions differ. Because the dataset is synthetic, the findings should be validated using real-world data.References
B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment classification using machine learning techniques,” in Proc. EMNLP, 2002, pp. 79–86, doi: 10.3115/1118693.1118704
W. Medhat, A. Hassan, and H. Korashy, “Sentiment analysis algorithms and applications: A survey,” Ain Shams Engineering Journal, vol. 5, no. 4, pp. 1093–1113, 2014, doi: 10.1016/j.asej.2014.04.011.
A. McCallum and K. Nigam, “A comparison of event models for Naive Bayes text classification,” in AAAI Workshop on Learning for Text Categorization, 1998, pp. 41–48.
C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, pp. 273–297, 1995, doi: 10.1007/BF00994018.
B. Wilie et al., “IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding,” in Proc. AACL-IJCNLP, 2020, pp. 843–857.
K. Kowsari, K. J. Meimandi, M. Heidarysafa, S. Mendu, L. Barnes, and D. Brown, “Text classification algorithms: A survey,” Information, vol. 10, no. 4, Art. no. 150, 2019, doi: 10.3390/info10040150.
A. Tripathy, A. Agrawal, and S. K. Rath, “Classification of sentiment reviews using n-gram machine learning approach,” Expert Systems with Applications, vol. 57, pp. 117–126, 2016, doi: 10.1016/j.eswa.2016.03.028.
A. D. Khoirunnisa, A. F. Hidayatullah, and M. R. K. Perdana, “Sentiment analysis of Indonesian social media text: A systematic review,” selected Indonesian NLP literature, 2021.
F. Z. Tala, “A study of stemming effects on information retrieval in Bahasa Indonesia,” M.Sc. thesis, Universiteit van Amsterdam, 2003.
M. A. Fauzi, “Random forest approach for sentiment analysis in Indonesian language,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 12, no. 1, pp. 46–50, 2018.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A benchmark dataset and pre-trained language model for Indonesian NLP,” in Proc. COLING, 2020, pp. 757–770.
F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
