Assessing the Quality of Mathematics Test─ Enumeration Rules to Measure Students’ Computational Thinking Skills

Suparman Suparman, Dadang Juandi, Turmudi Turmudi, Bambang Avip Priatna Martadiputra, Nana Diana

Abstract

Computational thinking (CT), one of the most fundamental thinking processes, empowers students in solving mathematical complex problems. In fact, most of Indonesian students, particularly in senior high school, yet have poor CT skills in mathematics. The quality of mathematics test can be one of the potential predictors to describe students’ CT skills accurately. The test, however, which involves enumeration rules as a content to measure Indonesian students’ CT skills relatively has not been existing. A cross-sectional study using descriptive-quantitative method was used to design and produce a qualified mathematics test to measure students’ CT skills. Three mathematical-essay problems were assessed by four experts in mathematics education, and administrated to 35 eleventh-grade students at a public senior high school in Bandung city. A number of quantitative analyses, such as Aiken’s V, Kendall’s W test, Pearson correlation, Cronbach alpha, discrimination index, and difficulty index were applied to assess the test’s quality. Results of this study showed high content and construct validity through expert assessment, as well as strong criterion-related validity based on student performance. High reliability was confirmed through Cronbach’s alpha, indicating consistent results across all test items. In addition, item analysis revealed very good discrimination indices and recommended difficulty levels, ensuring the test effectively differentiates student skills. The test provides a model for educators and curriculum designers in mathematics education seeking to improve classroom assessments, particularly in assessing students’ CT skills.

Keywords

Computational Thinking, Enumeration Rules, Mathematics Test, Reliability, Validity

References

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings, educational and psychological measurument. Educational and Psychological Measurement, 45(1), 131–142.

Akveld, M., & Kinnear, G. (2024). Improving mathematics diagnostic tests using item analysis. International Journal of Mathematical Education in Science and Technology, 55(10), 2478–2505. https://doi.org/10.1080/0020739X.2023.2167132

Arriza, L., Retnawati, H., & Ayuni, R. T. (2024). Item analysis of high school specialization mathematics exam questions with item response theory approach. BAREKENG: Jurnal Ilmu Matematika Dan Terapan, 18(1), 151–162. https://doi.org/10.30598/barekengvol18iss1pp0151-0162

Bland, J. M., & Altman, D. G. (1997). Statistics notes: Cronbach’s alpha. BMJ, 314, 571–572.

Creswell, W. J., & Creswell, J. D. (2018). Research Design: Qualitative, Quantitative adn Mixed Methods Approaches. Sage Publications Inc.

Ebell, R. L., & Friesbie, D. A. (1991). Essentials of educational measurement. Prentice-Hall, Inc. https://ebookppsunp.files.wordpress.com/2016/06/robert_l-ebel_david_a-_frisbie_essentials_of_edbookfi-org.pdf

Fitrah, M., Sofroniou, A., Ofianto, Judijanto, L., & Widihastuti. (2024). Reliability and separation index analysis of mathematics questions integrated with the cultural architecture framework using the Rasch model. Journal of Education and E-Learning Research, 11(3), 499–509. https://doi.org/10.20448/jeelr.v11i3.5861

Gall, J. P., Gall, M. D., & Borg, W. R. (2014). Applying educational research: How to read, do, and use research to solve problems of practice. Longman publishing Inc.

Garcia, M. L. B., Santos, K. C. P., & Vistro-Yu, C. P. (2025). Comparing two psychometric approaches: The case of item analysis for a classroom test in mathematics. International Journal of Education and Practice, 13(1), 327–344. https://doi.org/10.18488/61.v13i1.4060

Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051

Helsa, Y., Suparman, Juandi, D., Turmudi, & Ghazali, M. B. (2023). A meta-analysis of the utilization of computer technology in enhancing computational thinking skills: Direction for mathematics learning. International Journal of Instruction, 16(2), 735–758. https://doi.org/10.29333/iji.2023.16239a

Hilda, A. M., & Siswanto, R. D. (2021). Android application development: Permutation of the same elements based on realistic mathematics education. Mathematics Teaching-Research Journal, 13(4), 170–180.

Istiqlal, M., Istiyono, E., Widihastuti, Sari, D. K., Danni, R., & Safitri, I. (2024). Construction of mathematics cognitive test instrument of computational thinking model for Madrasah Aliyah students. Nazhruna: Jurnal Pendidikan Islam, 7(2), 475–492. https://doi.org/10.31538/nzh.v7i2.4425

Jatmiko, M. A., Herman, T., & Dahlan, J. A. (2021). Desain didaktis materi kaidah pencacahan untuk Siswa SMA kelas XI [Didactical design of enumeration rules for eleventh-grade students]. Hipotenusa Journal of Research Mathematics Education (HJRME), 4(1), 35–54. https://doi.org/10.36269/hjrme.v4i1.464

Kamber Hamzić, D., Trumić, M., & Hadžalić, I. (2025). Construction and analysis of test in triangle and circle trigonometry. International Electronic Journal of Mathematics Education , 20(1), 1–9. https://doi.org/10.29333/iejme/15734

Kartianom, K., Retnawati, H., & Hidayati, K. (2024). Assessing the fairness of mathematical literacy test in Indonesia: Evidence from gender-based differential item function analysis. Journal of Pedagogical Research, 8(3), 191–208. https://doi.org/10.33902/JPR.202426420

Kaup, C. F., Pedersen, P. L., & Tvedebrink, T. (2023). Integrating computational thinking to enhance students’ mathematical understanding. Journal of Pedagogical Research, 7(2), 127–142. https://doi.org/10.33902/JPR.202318531

Khan, G.-A. N., Ishrat, N., & Khan, A. Q. (2015). Using Item analysis on essay types questions given in summative examination of medical college students: Facility value, Discrimination index. International Journal of Research in Medical Sciences, 3(1), 178–182. https://doi.org/10.5455/2320-6012.ijrms20150131

Lamanna, L., Gea, M. M., & Batanero, C. (2022). Do secondary school students’ strategies in solving permutation and combination problems change with instruction? Canadian Journal of Science, Mathematics and Technology Education, 22(3), 602–616. https://doi.org/10.1007/s42330-022-00228-z

Maharani, S., Kholid, M. N., Pradana, L. N., & Nusantara, T. (2019). Problem-solving in the context of computational thinking. Infinity: Journal of Mathematics Education, 8(2), 109–116.

Masfingatin, T., & Maharani, S. (2019). Computational thinking: Students on proving geometry theorem. International Journal of Scientific and Technology Research, 8(9), 2216–2223.

Matitaputty, C., Nusantara, T., Hidayanto, E., & Sukoriyanto. (2022). Examining the pedagogical content knowledge of in-service mathematics teachers on the permutations and combinations in the context of student mistakes. Journal on Mathematics Education, 13(3), 393–414. https://doi.org/10.22342/jme.v13i3.pp393-414

Matlock-Hetzel, S. (1997). Basic Cconcepts in item and test analysis. Educational Research Association, 1, 1–15.

Munawarah, Thalhah, S. Z., Angriani, A. D., Nur, F., & Kusumayanti, A. (2021). Development of instrument test computational thinking skills IJHS/JHS based RME approach. Mathematics Teaching-Research Journal, 13(4), 202–220.

Nitko, A. J., & Brookhart, S. M. (2014). Educational Assessment of Students Sixth Edition. In Pearson New International Edition. Pearson Education, Inc.

Pei, C. (Yu), Weintrop, D., & Wilensky, U. (2018). Cultivating computational thinking practices and mathematical habits of mind in lattice land. Mathematical Thinking and Learning, 20(1), 75–89. https://doi.org/10.1080/10986065.2018.1403543

Rahmadani, N., & Hidayati, K. (2023). Quality of mathematics even semester final assessment test in class VIII using R program. Jurnal Pendidikan Matematika, 17(3), 397–416.

Rutherford, A. (2011). ANOVA and ANCOVA: A GLM approach. John Willey & Sons, Inc.

Shida, N., Abdullah, A. H., Osman, S., & ... (2023). Validation of mathematics test to assess polytechnic students’ problem solving. JPM: Jurnal Pendidikan Matematika, 17(2), 265–278. https://ejournal.unsri.ac.id/index.php/jpm/article/view/20171

Sukirman, S., Ibharim, L. F. M., Said, C. S., & Murtiyasa, B. (2021). A strategy of learning computational thinking through game based in virtual reality: Systematic review and conceptual framework. Informatics in Education, 21(1), 179–200. https://doi.org/10.15388/infedu.2022.07

Suparman, Juandi, D., & Turmudi. (2025). Development of Ucing Sumput as a digital educational game to enhance students’ mathematics achievement. Mathematics Teaching-Research Journal, 17(2), 7–35.

Suparman, Juandi, D., Turmudi, & Wahyudin. (2024). Computational thinking in mathematics instruction integrated STEAM education : Global trend and students ’ achievement in the last two decades. Beta: Jurnal Tadris Matematika, 17(2), 101–134. https://doi.org/10.20414/betajtm.v17i2.643

Suparman, S., Juandi, D., Turmudi, T., Martadiputra, B. A. P., Helsa, Y., Masniladevi, M., & Suherman, D. S. (2025). Computational thinking in mathematics instruction integrated to STEAM education: A systematic review and meta-analysis. TEM Journal, 14(1), 949–963. https://doi.org/10.18421/TEM141-84

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

Taylor, R. (1990). Interpretation of the correlation coefficient: A basic review. Journal of Diagnostic Medical Sonography, 6(1), 35–39. https://doi.org/10.1177/875647939000600106

Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://doi.org/10.1007/s10956-015-9581-5

Wing, J. M. (2006). Computational thinking. ACM SIGCSE Bulletin, 39(1), 195–196. https://doi.org/10.1145/1227504.1227378

Zhao, W., & Shute, V. J. (2019). Can playing a video game foster computational thinking skills? Computers and Education, 141, 1–40. https://doi.org/10.1016/j.compedu.2019.103633

DOI: https://doi.org/10.20961/paedagogia.v28i3.109581

Refbacks

  • There are currently no refbacks.