Sistem Chatbot Helpdesk WhatsApp Berbasis Large Language Model Gemini dengan Fine-Tuning Menggunakan Retrieval-Augmented Generation dan Qdrant Vector Database (Studi Kasus: Universitas Multi Data Palembang)

Muhamad Rifqi Virgiansyah; Muhammad Rizky Pribadi

Authors

Muhamad Rifqi Virgiansyah Universitas Multi Data Palembang
Muhammad Rizky Pribadi Universitas Multi Data Palembang https://orcid.org/0000-0001-7440-9667

Keywords:

Chatbot, Large Language Model, Retrieval-Augmented Generation

Abstract

Universitas Multi Data Palembang encounters significant challenges in handling a high volume of student inquiries via WhatsApp, resulting in prolonged response times and limited service availability outside operational hours. This study aims to design and implement an intelligent 24-hour helpdesk chatbot system to enhance academic service efficiency. The research employs the Research and Development (R&D) method. The system architecture integrates the Google Gemini Large Language Model (LLM), with performance optimized through the Retrieval-Augmented Generation (RAG) approach and Qdrant Vector Database. This technique enables the chatbot to access an internal knowledge base constructed from official university documents in real-time, thereby minimizing hallucinations. Performance evaluation using the RAGAS framework demonstrates a substantial improvement in Answer Correctness, achieving a score of 0.89, and a Faithfulness score of 0.98 compared to the model without RAG. Furthermore, User Acceptance Testing (UAT) involving 64 respondents yielded an average score of 4.26 (Very Good Category), indicating that the system is feasible for implementation to support rapid and accurate academic information services.

References

D. M. Alfiansyah, W. Wiilys, L. Setiyani, D. F. Wati, and D. Dedih, “Pengembangan Chatbot Berbasis Web untuk Layanan Informasi di Horizon University,” bit-Tech, vol. 7, no. 3, pp. 1068–1077, Apr. 2025, doi: 10.32877/bt.v7i3.2318.

A. Z. Amrullah, A. S. Anas, and Primajati. Gilang, “Implementasi Chatbot sebagai Virtual Assistant Penerimaan Mahasiswa Baru pada Universitas Bumigora,” Jurnal Bumigora Information Technology (BITe), vol. 4, no. 1, pp. 17–26, Jun. 2022, doi: 10.30812/bite.v4i1.1664.

Y. Tribber, K. Kusnadi, and M. Asfi, “Implementasi Retrieval Augmented Generation untuk Layanan Informasi Kampus dengan Chatbot Berbasis AI,” Prosiding SISFOTEK, vol. 8, no. 1, pp. 594–600, Nov. 2024, Accessed: Jun. 25, 2025. [Online]. Available: https://seminar.iaii.or.id/index.php/SISFOTEK/article/view/560

T. Q. Ramadhani, N. Q. Nada, and N. D. S, “Penerapan Metode Retrieval-Augmented Generation (RAG) Pada Chatbot E-Commerce Berbasis Gemini Ai,” Jurnal Ilmiah ILKOMINFO - Ilmu Komputer & Informatika, vol. 8, no. 2, pp. 301–313, Jul. 2025, doi: 10.47324/ilkominfo.v8i2.384.

N. Rachmat and D. P. Kesuma, “Implementasi LLM Gemini Pada Pengembangan Aplikasi Chatbot Berbasis Android,” Jurnal Ilmu Komputer (JUIK), vol. 4, no. 1, p. 40, Feb. 2024, doi: 10.31314/juik.v4i1.2831.

A. Basri and E. Ernawati, “Pemanfaatan Chatbot AI Untuk Mendukung Penyelesaian Tugas Akademik Mahasiswa Matematika: Studi Kasus Universitas Muslim Maros,” Prosiding Seminar Nasiona FKIP Universitas Muslim Maros, vol. 2, no. 1, pp. 55–60, Jul. 2025. [Online]. Available: https://ejournals.umma.ac.id/index.php/semnas/article/view/2818

M. D. A. Muhajir, N. Prastiti, and M. Koeshardianto, “IMPLEMENTASI CHATBOT MENGGUNAKAN FRAMEWORK LANGCHAIN BERBASIS LARGE LANGUAGE MODEL PADA WEBSITE DISPERPUSIP JAWA TIMUR,” Jurnal Sistem Informasi dan Bisnis Cerdas, vol. 17, no. 2, pp. 69–79, Aug. 2024. [Online]. Available: https://sibc.upnjatim.ac.id/index.php/sibc/article/view/364

D. W. Putra, A. P. Nugroho, and E. W. Puspitasari, “Game Edukasi Berbasis Android Sebagai Media Pembelajaran Untuk Anak Usia Dini,” JIMP - Jurnal Informatika Merdeka Pasuruan, vol. 1, no. 1, pp. 46–58, Mar. 2016, doi: 10.37438/jimp.v1i1.7.

H. T. I. Koten, B. S. Rintyarna, and R. W. P. Pamungkas, “Desain Arsitektur Chatbot Berbasis Hybrid Model Menggunakan Whatsapp Untuk Sistem Layanan Kesehatan,” Jurnal ELTIKOM : Jurnal Teknik Elektro, Teknologi Informasi dan Komputer, vol. 6, no. 2, pp. 142–151, Dec. 2022, doi: 10.31961/eltikom.v6i2.610.

DataReportal, “Digital 2024: Indonesia,” DataReportal – Global Digital Insights, Feb. 21, 2024. https://datareportal.com/reports/digital-2024-indonesia

L. R. Hidayat, I. G. P. S. Wijaya, and R. Dwiyansaputra, “OPTIMALISASI LAYANAN SISTEM INFORMASI MAHASISWA DENGAN INTEGRASI TELEGRAM: CHATBOT RETRIEVAL-AUGMENTED-GENERATION BERBASIS LARGE LANGUAGE MODEL,” Jurnal Teknologi Informasi, Komputer, dan Aplikasinya (JTIKA), vol. 7, no. 1, pp. 121–131, Mar. 2025, doi: 10.29303/jtika.v7i1.459.

S. A. Talaohu, R. Soekarta, and M. Surahmanto, “Implementasi LLM Pada Chatbot PMB Universitas Muhammadiyah Sorong Menggunakan Metode RAG Berbasis Website,” Framework: Jurnal Ilmu Komputer Dan Informatika, vol. 3, no. 2, pp. 1–11, Aug. 2025, doi: https://doi.org/10.33506/framework.v3i02.4790.

S. Khadafi, R. A. Saputra, and R. Uttungga, “Implementasi Chatbot Informasi Akademik Menggunakan Jaro Winkler pada Program Studi Sistem Informasi-ITATS,” in Seminar Nasional Sains dan Teknologi Terapan, Surabaya: Institut Teknologi Adhi Tama Surabaya, 2024, pp. 1–11.

A. Pratama and D. Sisephaputra, “Implementasi Chatbot Menggunakan Metode Retrieval Augmented Generation (RAG) untuk Layanan Pertanyaan Akademik,” Journal of Intelligent Systems and Computation, 2024.

S. A. Talaohu, R. Soekarta, and M. Surahmanto, “Implementasi LLM Pada Chatbot PMB Universitas Muhammadiyah Sorong Menggunakan Metode RAG Berbasis Website,” Framework : Jurnal Ilmu Komputer Dan Informatika, vol. 3, no. 2, pp. 1–11, Aug. 2025, doi: https://doi.org/10.33506/framework.v3i02.4790.

S. Khadafi, R. A. Saputra, and R. Uttungga, “Implementasi Chatbot Informasi Akademik Menggunakan Jaro Winkler pada Program Studi Sistem Informasi-ITATS,” in Seminar Nasional Sains dan Teknologi Terapan, Surabaya: Institut Teknologi Adhi Tama Surabaya, 2024, pp. 1–11. [Online]. Available: https://ejurnal.itats.ac.id/sntekpan/article/view/6575/0

G. Guntoro, L. Costaner, and L. Lisnawita, “Aplikasi Chatbot untuk Layanan Informasi dan Akademik Kampus Berbasis Artificial Intelligence Markup Language (AIML),” Digital Zone: Jurnal Teknologi Informasi dan Komunikasi, vol. 11, no. 2, pp. 291–300, Nov. 2020, doi: 10.31849/digitalzone.v11i2.5049.