Deep Neural Network-Based Student Performance Prediction with Hessian-Free Optimization

Andy Irawan, Zainal Abidin, Mohammad Jamhuri

Abstract


Predicting student graduation predicates is important for academic monitoring and timely intervention in higher education. This study investigates graduation predicate prediction using deep neural networks under three feature-group settings: academic-only, non-academic-only, and combined academic–non-academic features. A multilayer perceptron with three hidden layers was trained using SGD with momentum, RMSProp, Adam, and a damped Hessian-free optimization procedure. Two tasks were considered: a four-class graduation predicate classification task and a binary risk-screening task in which Sufficient was treated as the positive risk class. The results show that the combined feature group achieved the best multiclass performance, with an accuracy of 0.8478 and a weighted F1-score of 0.8274. Hessian-free optimization consistently produced the best results across all feature-group scenarios, with the clearest gain appearing in the non-academic-only setting. In the additional risk-screening analysis, non-academic variables provided meaningful but limited predictive signal, and Major emerged as the strongest individual predictor. These findings show that combining academic and non-academic information improves graduation predicate prediction and that Hessian-free optimization is an effective training strategy for deep neural classification in educational data.


Keywords


deep neural networks; educational data mining; graduation predicate prediction; Hessian-free optimization; risk screening

Full Text:

PDF

References


[1] Chaka, Chaka. "Educational Data Mining, Student Academic Performance Prediction, Prediction Methods, Algorithms and Tools: An Overview of Reviews." Journal of e-Learning and Knowledge Society 18(2) (2022), 58–69. doi: 10.20368/1971-8829/1135578.

[2] Alnasyan, Bayan, Mohammed Basheri, and Madini O. Alassafi. "The Power of Deep Learning Techniques for Predicting Student Performance in Virtual Learning Environments: A Systematic Literature Review." Computers and Education: Artificial Intelligence 6 (2024), 100231. doi: 10.1016/j.caeai.2024.100231.

[3] Villegas-Espinoza, Alexander E. J., and Jorge Isaac Necochea-Chamorro. "Using Deep Learning in Student Performance Prediction: A Systematic Review." TEM Journal 14(3) (2025), 2472–2482. doi: 10.18421/TEM143-51.

[4] Yaacob, Wan Fatimah Wan, et al. "Supervised Data Mining Approach for Predicting Student Performance." Indonesian Journal of Electrical Engineering and Computer Science 16(3) (2019), 1584–1592. doi: 10.11591/ijeecs.v16.i3.pp1584-1592.

[5] Shahiri, A. M., W. Husain, and N. A. Rashid. "A Review on Predicting Student’s Performance Using Data Mining Techniques." Procedia Computer Science 72 (2015), 414–422. doi: 10.1016/j.procs.2015.12.157.

[6] Widaningsih, Sri. "Perbandingan Metode Data Mining Untuk Prediksi Nilai dan Waktu Kelulusan Mahasiswa Prodi Teknik Informatika dengan Algoritma C4.5, Naïve Bayes, KNN dan SVM." Jurnal Tekno Insentif 13(1) (2019), 16–25. doi: 10.36787/jti.v13i1.78.

[7] Wibowo, A., and A. Rohman. "Prediksi Predikat Kelulusan Mahasiswa Menggunakan Naive Bayes dan Decision Tree pada Universitas XYZ." EXPERT: Jurnal Manajemen Sistem Informasi dan Teknologi 12(2) (2022), 104. doi: 10.36448/expert.v12i2.2810.

[8] Nurrohmat, M. A. "Aplikasi Pemrediksi Masa Studi dan Predikat Kelulusan Mahasiswa Informatika Universitas Muhammadiyah Surakarta Menggunakan Metode Naive Bayes." Khazanah Informatika: Jurnal Ilmu Komputer dan Informatika 1(1) (2015), 29–34. doi: 10.23917/khif.v1i1.1179.

[9] Rudy Hendrawan, I. N., et al. "Klasifikasi Lama Studi dan Predikat Kelulusan Mahasiswa menggunakan Metode Naïve Bayes." Jurnal Eksplora Informatika 11(1) (2022), 50–56. doi: 10.30864/eksplora.v11i1.606.

[10] Indahyanti, U., N. L. Azizah, and H. Setiawan. "Pendekatan Ensemble Learning Untuk Meningkatkan Akurasi Prediksi Kinerja Akademik Mahasiswa." Jurnal Sains dan Informatika 8(2) (2022), 160–169. doi: 10.34128/jsi.v8i2.459.

[11] Kaensar, Chayaporn, and Worayoot Wongnin. "Predicting New Student Performances and Identifying Important Attributes of Admission Data Using Machine Learning Techniques with Hyperparameter Tuning." Eurasia Journal of Mathematics, Science and Technology Education 19(12) (2023), em2369. doi: 10.29333/ejmste/13863.

[12] Li, Kam Cheong, Billy Tak-Ming Wong, and Thomas Hon-Tung Chan. "Predictive Analytics for University Student Admission: A Literature Review." In: Blended Learning: Education in a Smart Learning Environment. Springer, 2023, 250–259. doi: 10.1007/978-3-031-35731-2_22.

[13] Mengash, Hanan Abdullah. "Using Data Mining Techniques to Predict Student Performance to Support Decision Making in University Admission Systems." IEEE Access 8 (2020), 55462–55470. doi: 10.1109/ACCESS.2020.2981905.

[14] Mustakim, and Giantika Oktaviani. "Algoritma K-Nearest Neighbor Classification Sebagai Sistem Prediksi Predikat Prestasi Mahasiswa." Jurnal Sains, Teknologi, dan Industri 13(2) (2016), 195–202. doi: 10.24014/sitekin.v13i2.1688.

[15] Nasution, N. B., et al. "Prediksi Lama Studi dan Predikat Kelulusan Mahasiswa Menggunakan Algoritma Supervised Learning." G-Tech: Jurnal Teknologi Terapan 7(2) (2023), 386–395. doi: 10.33379/gtech.v7i2.2077.

[16] Zulfiqar, La Ode Mohamad, Nurul Renaningtias, and M. Y. Fathoni. "Educational Data Mining in Graduation Rate and Grade Predictions Utilizing Hybrid Decision Tree and Naïve Bayes Classifier." In: Proceedings of the International Conferences on Information System and Technology 2019. SCITEPRESS, 2020, 151–157. doi: 10.5220/0009907101510157.

[17] Ofori, Francis, Elizaphan Maina, and Rhoda Gitonga. "Using Machine Learning Algorithms to Predict Students’ Performance and Improve Learning Outcome: A Literature Based Review." Journal of Information and Technology 4(1) (2020), 33–55. Available online.

[18] Martens, James. "Deep Learning via Hessian-Free Optimization." In: Proceedings of the 27th International Conference on Machine Learning. 2010, 735–742. doi: 10.5555/3104322.3104416.

[19] Pearlmutter, Barak A. "Fast Exact Multiplication by the Hessian." Neural Computation 6(1) (1994), 147–160. doi: 10.1162/neco.1994.6.1.147.

[20] Shewchuk, Jonathan Richard. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain. Tech. rep. Carnegie Mellon University, 1994. Available online.

[21] Rane, Nitin Liladhar, et al. "Techniques and Optimization Algorithms in Deep Learning: A Review." In: Applied Machine Learning and Deep Learning: Architectures and Techniques. Deep Science Publishing, 2024, 59–79. doi: 10.70593/978-81-981271-4-3_3.

[22] Korbit, Mikalai, et al. "Exact Gauss-Newton Optimization for Training Deep Neural Networks." Neurocomputing 658 (2025), 131738. doi: 10.1016/j.neucom.2025.131738.

[23] Jamhuri, Mohammad, et al. "Inexact Generalized Gauss–Newton–CG for Binary Cross-Entropy Minimization." Jurnal Riset Mahasiswa Matematika 5(2) (2025), 102–122. doi: 10.18860/jrmm.v5i2.34739.

[24] Jamhuri, Mohammad, et al. "Neural Networks Optimization via Gauss–Newton Based QR Factorization on SARS-CoV-2 Variant Classification." Systems and Soft Computing 7 (2025), 200195. doi: 10.1016/j.sasc.2025.200195.

[25] Jamhuri, Mohammad, Imam Mukhlash, and Mohammad Isa Irawan. "Performance Improvement of Logistic Regression for Binary Classification by Gauss-Newton Method." In: Proceedings of the 2022 5th International Conference on Mathematics and Statistics. 2022, 12–16. doi: 10.1145/3545839.3545842.

[26] Bañeres, David, et al. "An Early Warning System to Detect At-Risk Students in Online Higher Education." Applied Sciences 10(13) (2020), 4427. doi: 10.3390/app10134427.

[27] Foster, Elizabeth, and Richard Siddle. "The Effectiveness of Learning Analytics for Identifying At-Risk Students in Higher Education." Assessment & Evaluation in Higher Education 45(6) (2020), 842–854. doi: 10.1080/02602938.2019.1682118.




DOI: https://doi.org/10.18860/jrmm.v5i4.37951

Refbacks

  • There are currently no refbacks.