Effectiveness Of An Ojs-Based Semester Lesson Plan Validation System: A User Experience Review
Indonesia Indonesia Indonesia 
Article Sidebar
-
Cohen's Kappa, Mixed Methods, Semester Learning Plan (RPS), System Effectiveness, User Experience, Validation System
Abstract
Background. Quality assurance in higher education instructional planning is often hindered by inefficient validation processes for Semester Learning Plans (RPS). Digital innovations inspired by academic workflow systems offer potential solutions for improving efficiency and reliability in these processes.
Purpose. This study aimed to evaluate the effectiveness and user experience of the RPS Validation and Evaluation System (SVeR), a digital platform modeled after the workflow of the Open Journal System (OJS). Specifically, it investigated whether SVeR could enhance the validation process and be positively received by lecturers.
Method. Using an explanatory sequential mixed methods design, 17 lecturers from the PGMI Program at UIN Mahmud Yunus Batusangkar participated. The quantitative phase measured inter-rater reliability using Cohen's Kappa (?) statistics and assessed user perceptions through a rating questionnaire. The qualitative phase analyzed user feedback via thematic analysis.
Results. Quantitative findings indicated a very high level of effectiveness, with an average Kappa coefficient of ? = 0.939, alongside a highly positive user acceptance, reflected in an average rating of 8.88 out of 10. Thematic analysis revealed three major advantages of SVeR: simplification of workflow, increased efficiency, and a positive impact on quality assurance.
Conclusion. The study concludes that SVeR is both objectively effective and well-received by users, suggesting its potential as a model for digitalizing academic quality assurance processes in higher education.
Full text article
References
’Azah, N., Juraidah, J., & Fateh, H. (2025). Enhancing Quality Assurance in Higher Education through Strategic Library Management and Digital Literacy Initiatives. JELS: Journal of Education and Learning Sciences, 05(02), 45–59. https://doi.org/https://doi.org/10.56404/jels.v5i2.174
Afifah, N. (2019). Relevansi Kurikulum PGMI Terhadap Capaian Pembelajaran Mahasiswa PGMI IAIN Metro Nurul Afifah Institut Agama Islam Negeri Metro. Ar-Riayah?: Jurnal Pendidikan Dasar, 3(1), 61–82. https://doi.org/10.29240/jpd.v3i1.863
Agissa, A. W., Sobari, M., Rohimiansyah, M. I., & Marlina, L. (2025). Peran Manajemen Strategis dalam Peningkatan Mutu Pendidikan Islam di Madrasah. Journal of Innovative and Creativity, 5(3), 31750–31760. https://doi.org/https://doi.org/10.31004/joecy.v5i3.4815
Aini, Q., Handayani, I., Husna, F., & Lestari, N. (2020). Utilization Of Scientific Publication Media To Improve The Quality Of Scientific Work. Aptisi Transactions On Management (ATM), 4(1), 1–12. https://doi.org/https://doi.org/10.33050/atm.v4i1.711
Al Madani, M. (2025). Is OJS Suitable for Managing Conferences? Open Journal Theme. https://openjournaltheme.com/conference-platform-vs-ojs/
Altman, M., Castro, E., Crosas, M., Durbin, P., Garnett, A., & Whitney, J. (2015). Open Journal Systems and Dataverse Integration--Helping Journals to Upgrade Data Publication for Reusable Research. Code4Lib Journal, 30, 8. http://search.ebscohost.com/login.aspx?direct=true&db=lxh&AN=111927103&site=ehost-live
Aripradono, H. W., Nursyamsi, I., Wahab, A., & Sultan, Z. (2024). Educational Technology for Digital Transformation of Higher Education Institutions into Entrepreneurial Universities. Policy and Governance Review, 8(2), 303–322. https://doi.org/DOI: https://doi.org/10.30589/pgr. v8i3.1019
Arya, K. Y., Tripathy, P., Awasthi, S., & Shahi, S. (2024). A Comparative Study of Traditional and Digital Approaches to Chemistry Education. African Journal of Biomedical Research, 27(4), 9874–9879. https://doi.org/OIhttps://doi.org/10.53555/AJBR.v27i4S.5561
Asyibli, B., Romadhon, A. M., Zohriah, A., & Muin, A. (2025). Quality Management of Indonesian Education Services in the Digital Era: A Systematic Literature Review. Didaktika: Jurnal Kependidikan, 14(4), 7367–7382. https://doi.org/https://doi.org/10.58230/27454312.3121
Awashreh, R., Ghunaimi, H. Al, & Hassiba, A. (2025). A comparison of traditional , online , and hybrid learning models in accounting and finance education?: Student perceptions and academic outcomes. International Journal of Advanced and Applied Sciences, 12(6), 21–34. https://doi.org/https://doi.org/10.21833/ijaas.2025.06.003
Bagozzi, R. P. (2007). The Legacy of the Technology Acceptance Model and a Proposal for a Paradigm Shift. Journal of the Association for Information Systems, 8(4), 244–254. https://doi.org/DOI: 10.17705/1jais.00122
Bano, A., Ali, M., & John, S. (2019). Issues of Syllabus Designing Practices and Quality Assurance at Higher Education Level. Global Social Sciences Review (GSSR), IV(IV), 135–145. https://doi.org/http://dx.doi.org/10.31703/gssr.2019(IV-IV).18
Basyah, N. A., Fahmi, I., Ali, I., Rusli, Z., & Majid, R. A. (2025). The Acceptance of Technology Model Management Systems: A Systematic Review. Jurnal Penelitian Pendidikan IPA, 11(10), 23–32. https://doi.org/10.29303/jppipa.v11i10.7394
Biggs, J. (2014). Constructive alignment in university teaching. HERDSA Review of Higher Education, 1(July), 5–22.
Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319–340. http://www.jstor.org/stable/249008
Delatorre-diaz, L., Ramírez-pérez, H. X., & Escobar-castillejos, D. (2025). The e ectiveness of curriculum standardization in data analysis and tools proficiency for undergraduate education: a case study. Frontiers in Education, 10(April), 1–8. https://doi.org/doi: 10.3389/feduc.2025.1537174
Duval, E. (2004). Learning Technology Standardization?: Making Sense of it All. Computer Science and Information Systems, 1(1), 33–43. https://doi.org/https://doi.org/10.2298/CSIS0401033D
Eberly, M. B., Newton, S. E., Wiggins, R. A., Eberly, M. B., Newton, S. E., & Wiggins, R. A. (2001). The Syllabus as a Tool for Student-Centered Learning. The Journal of General Education, 50(1), 56–74. https://doi.org/https://doi.org/10.1353/jge.2001.0003
Garavalia, L. S., Hummel, J. H., Wiley, L. P., & Huitt, W. G. (1999). Constructing the Course Syllabus?: Faculty and Student Perceptions of Important Syllabus Components. Journal on Excellence in College Teaching, 10(1), 5–21.
Gupta, P. K., Raturi, S., & Strzelecki, A. (2026). Determinants of educational technology adoption among university teachers: extending the UTAUT framework. Discover Education, February, 1–36. https://doi.org/https://doi.org/10.1007/s44217-026-01192-8
Henim, S. R., & Sari, R. P. (2020). Evaluasi User Experience Sistem Informasi Akademik Mahasiswa pada Perguruan Tinggi Menggunakan User Experience Questionnaire. Jurnal Komputer Terapan, 6(1), 69–78. https://doi.org/https://doi.org/10.35143/jkt.v6i1.3582
Houdyshell, M., & Ziegler, N. (2021). Graduate students share their experiences of building helping skills?: A case study. Journal of Curriculum Studies Research, 3(2), 57–78. https://doi.org/https://doi.org/10.46303/jcsr.2020.4
Husein, H. M., Sukardi, M., & Jarti, N. (2025). Pengembangan Sistem Informasi Akademik Berbasis Web untuk Meningkatkan Efisiensi Administrasi. Jurnal Nasional Komputasi Dan Teknologi Informasi (JNKTI), 8(6), 3209–3211. https://doi.org/https://doi.org/10.32672/jnkti.v8i6.10008
Jovanovic, M., & Dimitrijevic, D. (2023). Higher Education Curriculum Management System. FACTA UNIVERSITATIS: Teaching, Learning and Teacher Education, 7(1), 65–76. https://doi.org/https://doi.org/10.22190/FUTLTE221117007J
Karmila, E., Zainuri, A., & Yasir, M. G. (2025). Strategi Manajemen dalam Peningkatan Mutu Pendidikan Dasar Islam: Studi Kasus di MI Hijriyah II Palembang. Jurnal Wahana Didaktika, 23(2), 296–306. https://doi.org/https://doi.org/10.31851/wahanadidaktika.v23i2.18611
Khechine, H., Raymond, B., & Augier, M. (2020). The adoption of a social learning system: Intrinsic value in the UTAUT model. British Journal of Educational Technology, 0(0), 1–20. https://doi.org/10.1111/bjet.12905
Koteczki, R. (2025). Exploring Generation Z’s Acceptance of Artificial Intelligence in Higher Education: A TAM and UTAUT-Based PLS-SEM and Cluster Analysis. Education Sciences, 15(8), 1044. https://doi.org/https://doi.org/10.3390/ educsci15081044
Landis, J. R., & Koch, G. G. (2013). The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1), 159–174. http://www.jstor.org/stable/2529310
Layyinah, S. Q., Hikmatyar, A., Aqsho, K. N., & Nuryaman, N. (2016). Pengembangan profesionalisme lptk dalam mencetak tenaga pendidik. National Education Conference, 56–62.
Lestari, S., & Yusuf, F. N. (2025). Aligning Assessment Practices with Learning Objectives?: A Case of EFL Classes in Indonesia. Indonesian Journal of English Language Teaching and Applied Linguistics, 10(1), 145–163. https://doi.org/http://dx.doi.org/10.210 93/ijeltal.v10i1.1973
Malik, B. H., Khalid, M., Maryam, M., Ali, M. N., Yousaf, S., & Mehmood, M. (2019). IoT Testing-as-a-Service?: A New Dimension of Automation. International Journal of Advanced Computer Science and Applications, 10(5), 364–371.
Maraza-quispe, B., & Alejandro-oviedo, O. M. (2019). Towards a Standardization of Learning Behavior Indicators in Virtual Environments. International Journal of Advanced Computer Science and Applications, Vol. 10, No. 5, 2019, 11(11), 146–152.
Maulida, N. M., Prastya, S. E., Hidayat, A., & Pratama, R. N. (2024). Evaluasi Kepuasan Pengguna Sistem Informasi Akademik (SIAKAD) Universitas Sari Mulia Banjarmasin Menggunakan Metode End User Computing Satisfaction (EUCS). Jurnal Nasional Komputasi Dan Teknologi Informasi, 7(5), 1012–1020. https://doi.org/https://doi.org/10.32672/jnkti.v7i5.7949
Merrill, M. D. (2002). First Principles of Instruction. Educational Technology Research and Development, 50(3), 43–59. https://doi.org/https://doi.org/10.1007/BF02505024
Murdiani, D., & Umar, R. (2020). Evaluasi Kualitas Sistem Jurnal Elektronik berbasis Open Journal System Menggunakan ISO/IEC 25010. BACA: Jurnal Dokumentasi Dan Informasi, 41(1), 75–86. https://doi.org/DOI: https://doi.org/10.14203/j.baca.v41i1.588
Nahar, B. S. (2016). Comparative Study of Traditional vs. Digital Reference Services. Internastional Journal for Advanced Research in Science and Technology, 6(0), 168–174.
Nirwanto, N. (2019). End-user Satisfaction as an Impact of the System Quality , Information Quality , and Top Management Support , upon the Perceived Usefulness of Technology Utilization. Osaka Conference Prograrn, 1–18.
Normelindasari, D., & Solichin, A. (2020). Effect of System Quality , Information Quality , and Perceived Usefulness on User Satisfaction of Webstudent Applications to Improve Service Quality for Budi Luhur University Students. International Conference on Management, Economics and Business (ICMEB 2019) Effect, 120(Icmeb 2019), 77–82.
Papanthymou, A., & Darra, M. (2022). Quality in Higher Education?: Defining the Conceptual Contents and their Relative Predominance. Higher Education Studies, 12(4), 18–36. https://doi.org/10.5539/hes.v12n4p18
Prasetya, B., Mahardhika, D. P., & Usino, W. (2025). Analysis of the Effect of System Quality, Information Quality, and Service Quality on User Satisfaction of T24 Application With Perceived Usefulness as an Intervening Variable. Indonesian Interdisciplinary Journal of Sharia Economics (IIJSE), 8(3), 7453–7471. https://doi.org/https://doi.org/10.31538/iijse.v8i3.6912
Putra, S. P., & Ramli, M. (2025). Peran, Tantangan dan Stratedi Pusat Penjaminan Mutu di Perguruan Tinggi Islam di Indonesia. JURNAL MAPPESONA, 8(2), 55–65.
Rafa, A., Triandini, N., & Kuswantoro, A. (2022). Digitalisasi Administrasi Akademik untuk Pelayanan Efektif dan Efisien. In Administrasi Perkantoran (pp. 256–272).
Ragil, Y. A., Rahmawati, Y., & Supena, A. (2025). Implementasi Kebijakan Penyelenggaraan Pendidikan Profesi Guru ( PPG ) untuk Sertifikasi Guru PAUD. Jurnal Edukasi AUD, 11(1), 41–54. https://doi.org/10.18592/jea.v11i1.15243
Ramatni, A., Rahmi, S., Septiyani, rini D., Andriani, N., Alpata, A. R., & Farianti, W. T. (2025). Arah baru Manajemen Pendidikan: Dari Administrasi Konvensional ke Digitalisasi Total. Jurnal Edu Research, 6(1), 313–322. https://doi.org/https://doi.org/10.47827/jer.v6i1.531
Ridwan, M., Rusdianto, D. S., & Ananta, M. T. (2019). Pengembangan Aplikasi Service Open Journal System (OJS) (Studi Kasus?: FILKOM UB). 3(5), 2548–2964. http://j-ptiik.ub.ac.id
Riyanto, S., Subagyo, H., Nugroho, B., Afandi, S., Marlina, E., Maulana, A. H. A., Sari, R. keumala, & Rishadi. (2017). Pengantar Open Journal System (OJS). November. https://doi.org/10.13140/RG.2.1.4093.1280
Rizman, M., Mohd, S., Johari, J., & Ruslan, F. A. (2024). Assessing Student Performance in Digital Signal Processing (DSP) Subjects?: A Comparative Study of Traditional and Online Learning Environments. Journal of Electrical and Electronic System Research, 24(April), 18–24.
Rohman, M., Rakhmawati, L., Haryudo, S. I., Indriyanti, A. D., Tjahyaningtijas, R. R. H. P. A., & Puspitaningayu, P. (2024). Effectiveness and Efficiency of the Open-Source Platform System OJS 3.2 Journal Management for Electronic-Based Article Publication?: Author and Reviewer Perspectives. Vocational Education International Conference (VEIC 2023), Veic 2023, 922–928. https://doi.org/10.2991/978-2-38476-198-2
Rosli, M. S., Saleh, N. S., Ali, A., Bakar, S. A., & Tahir, L. M. (2022). A Systematic Review of the Technology Acceptance Model for the Sustainability of Higher Education during the COVID-19 Pandemic and Identified Research Gaps. Sustainability, 14(8), 1–25. https://doi.org/https://doi.org/10.3390/su141811389
Rouffet, C., Beuningen, C. Van, & Graaff, R. De. (2023). Constructive alignment in foreign language curricula: an exploration of teaching and assessment practices in Dutch secondary education. The Language Learning Journal ISSN:, 51(3), 344–358. https://doi.org/10.1080/09571736.2022.2025542
Rumore, M. M. (2016). The Course Syllabus: Legal Contract or Operator’s Manual? American Journal of Pharmaceutical Education, 80(10), 1–7. https://doi.org/10.5688/ajpe8010177
Rusmiati, M. N., Ashifa, R., & Herlambang, Y. T. (2023). Analisis Problematika Implementasi Kurikulum Merdeka di Sekolah Dasar. Naturalistic: Jurnal Kajian Penelitian Dan Pendidikan Dan Pembelajaran, 7(2), 1490–1499. https://doi.org/https://doi.org/10.35568/naturalistic.v7i2.2203
Siagian, B. R., Budianto, C., & Ramadhan, M. D. (2025). Analisis Kualitas Sistem Informasi Manajemen Akademik Menggunakan Model DeLone dan McLean. Jurnal Pendidikan Tambusai, 9(2), 22248–22254. https://doi.org/https://doi.org/10.31004/jptam.v9i2.30145
Slamet, T. S., & Akil, M. H. U. (2025). Critical Review of the Technology Acceptance Model in Information Systems Research. International Journal of Strategic Information Systems and Knowledge Applications, 1(1), 42–51.
Slattery, J. M., & Carlson, J. F. (2014). Preparing An Effective Syllabus: Current Best Practices. College Teaching, 53(4), 159–164. https://doi.org/10.3200/CTCH.53.4.159-164
Sofia, U., & Musrini, B. M. (2021). DeLone and McLean Model of Academic Information System Success. ELECTROTEHNIC?, ELECTRONIC?, AUTOMATIC? (EEA), 69(2), 92–101. https://doi.org/https://doi.org/10.46904/eea.21.69.2.11080011
Susilawati, S., Rahmat, J., & Oktori, A. R. (2022). Implementasi Kurikulum Berbasis KKNI Dalam Upaya Penguatan Mutu Lulusan Pada Program Studi PGMI IAIN Curup. AR-RIAYAH: Jurnal Pendidikan Dasar, 6(1), 105–114. https://doi.org/10.29240/jpd.v6i1.4427
Tabatadze, B. (2026). Prospects of Using AI Technologies in Open Journal Systems (OJS). International Journal of Software Engineering, 14(1), 22–35. https://doi.org/10.31578/jtst.v8i2.163
Tan, X., Lin, X., & Zhuang, R. (2024). Development and validation of a secondary vocational school students ’ digital learning competence scale. Smart Learning Environments, 11(37), 1–29. https://doi.org/10.1186/s40561-024-00325-6
Thompson, B. (2007). The Syllabus as a Communication Document: Constructing and Presenting the Syllabus. Communication Education, 56(1), 54–71. https://doi.org/http://dx.doi.org/10.1080/03634520601011575
Willinsky, J. (2005). Open Journal Systems: An example of open source software for journal management and publishing. Library Hi Tech, 23(4), 504–519. https://doi.org/https://doi.org/10.1108/07378830510636300
Authors
Copyright (c) 2026 Safrizal Safrizal, Desri Jumiarti , Syamsul Bahri
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
