AI-assisted adaptive problem-based learning in a programming course: its effects on students' problem-solving and debugging skills

Septriyan Anugrah; Nadya Faranda; Nofri Hendri; Meldi Ade Kurnia Yusri

doi:10.32698/03302

AI-assisted adaptive problem-based learning in a programming course: its effects on students' problem-solving and debugging skills

Septriyan Anugrah⁽¹⁾, Nadya Faranda⁽²⁾, Nofri Hendri⁽³⁾, Meldi Ade Kurnia Yusri⁽⁴⁾,
(1) Universitas Negeri Padang

Indonesia
(2) Universitas Negeri Padang

Indonesia
(3) Universitas Negeri Padang

Indonesia
(4) Universitas Negeri Padang

Indonesia

Corresponding Author

DOI : https://doi.org/10.32698/03302

Full Text:

Language : en

Abstract

This study examined differences in problem-solving and debugging skills between students who participated in AI-assisted adaptive Problem-Based Learning (PBL) and those who participated in the Case Method in a programming course. A quasi-experimental nonequivalent pretest-posttest control-group design involved 69 students from the Department of Curriculum and Educational Technology, Universitas Negeri Padang. The experimental group (n = 35) followed Arends' PBL with data-informed adaptive grouping and structured use of ChatGPT, Gemini, and Claude during the investigation phase, while the control group (n = 34) followed the Case Method without generative AI. Multivariate analysis of covariance (MANCOVA) tested the two posttest scores while controlling for both pretest scores. The results showed a significant multivariate difference between the instructional conditions, Pillai's Trace = 0.456, F(2, 64) = 26.780, p < 0.001, partial η² = .456. Follow-up tests also showed significant differences in problem solving (partial η² = 0.335) and debugging (partial η² = 0.435). Adjusted means were higher for the experimental group on both skills. The findings suggest that an instructional package integrating PBL, adaptive grouping, and structured AI assistance has the potential to support problem-solving and debugging skills. However, because the three components were implemented simultaneously, their individual effects cannot be separated.

Keywords

adaptive grouping, generative AI, problem-based learning, problem solving, debugging

References

Ahmadzadeh, M., Elliman, D., & Higgins, C. (2007). The impact of improving debugging skill on programming ability. Innovation in Teaching and Learning in Information and Computer Sciences, 6(4), 72–87. https://doi.org/10.11120/ital.2007.06040072

Arends, R. I. (2012). Learning to teach (9th ed.). In McGraw-Hill.

Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based learning: A meta-analysis. Learning and Instruction, 13(5), 533–568. https://doi.org/10.1016/S0959-4752(02)00025-7

Fitzgerald, S., McCauley, R., Hanks, B., Murphy, L., Simon, B., & Zander, C. (2008). Debugging: Finding, fixing and flailing, a multi-institutional study of novice debuggers. Computer Science Education, 18(2), 93–116. https://doi.org/10.1080/08993400802114508

Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., & Koedinger, K. R. (2022). Ethics of AI in education: Towards a community-wide framework. International Journal of Artificial Intelligence in Education, 32(3), 504–526. https://doi.org/10.1007/s40593-021-00239-1

Hooshyar, D., Weng, X., Sillat, P. J., Tammets, K., Wang, M., & Hämäläinen, R. (2024). The effectiveness of personalized technology-enhanced learning in higher education: A meta-analysis with association rule mining. . . Computers & Education. https://doi.org/10.1016/j.compedu.2024.105169

Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2020.100001

Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2021.100017

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences. https://doi.org/10.1016/j.lindif.2023.102274

Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023). Studying the effect of AI code generators on supporting novice learners in introductory programming. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–23. https://doi.org/10.1145/3544548.3580919

Liu, X., Liu, S., Chen, S., Wang, Y., Zhang, B., & Kang, J. (2022). Effects of problem-based learning instructional intervention on critical thinking in higher education: A meta-analysis. Thinking Skills and Creativity. https://doi.org/10.1016/j.tsc.2022.101069

Manuaba, I. B. A. P., No, Y., & Wu, C.-C. (2022). The effectiveness of problem based learning in improving critical thinking, problem-solving and self-directed learning in first-year medical students: A meta-analysis. PLOS ONE, 17(11). https://doi.org/10.1371/journal.pone.0277339

McCauley, R., Fitzgerald, S., Lewandowski, G., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: A review of the literature from an educational perspective. Computer Science Education, 18(2), 67–92. https://doi.org/10.1080/08993400802114581

Mirata, V., & Bergamin, P. (2023). Role of organisational readiness and stakeholder acceptance: An implementation framework of adaptive learning for higher education. Educational Technology Research and Development, 71, 1567–1593. https://doi.org/10.1007/s11423-023-10248-7

Mirata, V., Hirt, F., Bergamin, P., & van der Westhuizen, C. (2020). Challenges and contexts in establishing adaptive learning in higher education: Findings from a Delphi study. International Journal of Educational Technology in Higher Education, 17(32). https://doi.org/10.1186/s41239-020-00209-y

Mogavi, R. H., Deng, C., Kim, J. J., Zhou, P., Kwon, Y. D., Metwally, A. H., Tlili, A., Bassanelli, S., Bucchiarone, A., Gujar, S., Nacke, L. E., & Hui, P. (2024). ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptions. Computers in Human Behavior: Artificial Humans, 2(1). https://doi.org/10.1016/j.chbah.2023.100027

Normadhi, N. B. A., Shuib, L., Md Nasir, H. N., Bimba, A., Idris, N., & Balakrishnan, V. (2019). Identification of personal traits in adaptive learning environment: Systematic literature review. Computers & Education, 130, 168–190. https://doi.org/10.1016/j.compedu.2018.11.005

Plooy, D., E., Casteleijn, D., & Franzsen, D. (2024). Personalized adaptive learning in higher education: A scoping review of key characteristics and impact on academic performance and engagement. Heliyon, 10(21). https://doi.org/10.1016/j.heliyon.2024.e39630

Qian, Y., & Lehman, J. (2018). Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Transactions on Computing Education, 18(1), 1–24. https://doi.org/10.1145/3077618

Sun, D., Boudouaia, A., Zhu, C., & Li, Y. (2024). Would ChatGPT-facilitated programming mode impact college students’ programming behaviors, performances, and perceptions? An empirical study. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-024-00446-5

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(15). https://doi.org/10.1186/s40561-023-00237-x

Yew, E. H. J., & Goh, K. (2016). Problem-based learning: An overview of its process and impact on learning. Health Professions Education, 2(2), 75–79. https://doi.org/10.1016/j.hpe.2016.01.004

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education where are the educators? International Journal of Educational Technology in Higher Education, 39(16). https://doi.org/10.1186/s41239-019-0171-0

Zhang, X., Zhang, B., & Zhang, F. (2022). Student-centered case-based teaching and online-offline case discussion in postgraduate courses of computer science. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-022-00374-2