Hide
Show all
Email this article 
Profiling Multicomponent Chemical Reasoning: A Learning Analytics Approach to Applied and Socio-Chemical Dimensions
Abstract
Scientific reasoning in chemistry involves the ability to apply conceptual knowledge in problem-solving, as well as to evaluate issues within broader social, ethical, and environmental contexts. However, conventional assessments often fail to capture this multidimensionality by reducing performance to a single final score. This study uses an integrated learning analytics approach to analyze students’ reasoning performance across two core domains of chemistry learning—applied reasoning and socio-chemical reasoning. A quantitative descriptive design was employed, involving 56 pre-service chemistry teachers who completed four open-ended essay questions, two in each reasoning domain. Student responses were scored using an analytical rubric assessing conceptual accuracy, logical coherence, and justification relevance. Data were analyzed using single-domain and multicomponent strategies, including quadrant profiling, trajectory mapping, clustering, and distribution analysis. Visual tools such as radar charts, spaghetti plots, contour density plots, and alluvial diagrams were used to depict students’ reasoning profiles. Results revealed that most students demonstrated moderate reasoning abilities, although notable inconsistencies were observed between the domains. Individual trajectories exhibited non-linear variations, highlighting diverse cognitive patterns. Clustering and heatmaps indicated distinct learner segments, while alluvial diagrams illustrated transitions between reasoning levels across domains. These findings suggest that students’ reasoning abilities are varied and dynamic. It is concluded that chemistry reasoning is multidimensional and should be assessed through integrated, data-driven methods. The study recommends the adoption of formative, analytics-supported assessments to inform differentiated instruction and promote deeper conceptual and ethical engagement in chemistry education.
Keywords
Full Text:
PDFReferences
[1] D. King, “New perspectives on context-based chemistry education: Using a dialectical sociocultural approach to view teaching and learning,” Stud. Sci. Educ., vol. 48, no. 1, pp. 51–87, Mar. 2012, doi: https://doi.org/10.1080/03057267.2012.655037.
[2] D. G. Ramadhani, S. Yamtinah, S. Saputro, and S. Widoretno, “Analysis of the relationship between students’ argumentation and chemical representational ability: A case study of hybrid learning oriented in the environmental chemistry course,” Chem. Teach. Int., vol. 5, no. 4, pp. 397–411, 2023, doi: https://doi.org/10.1515/cti-2023-0047.
[3] J. Pernaa, V. Kämppi, and M. Aksela, “Supporting the relevance of chemistry education through sustainable ionic liquids context: A research-based design approach,” Sustainability, vol. 14, no. 10, p. 6220, 2022, doi: https://doi.org/10.3390/su14106220.
[4] J. L. Spencer et al., “Cultural relevance in chemistry education: Snow chemistry and the Iñupiaq community,” J. Chem. Educ., vol. 99, no. 1, pp. 363–372, 2021, doi: https://doi.org/10.1021/acs.jchemed.1c00480.
[5] J. E. Wissinger et al., “Integrating sustainability into learning in chemistry,” J. Chem. Educ., vol. 98, no. 4, pp. 1061–1063, 2021, doi: https://doi.org/10.1021/acs.jchemed.1c00284.
[6] V. Talanquer, “Importance of understanding fundamental chemical mechanisms,” J. Chem. Educ., vol. 95, no. 11, pp. 1905–1911, 2018, doi: https://doi.org/10.1021/acs.jchemed.8b00508.
[7] J. T. Laverty et al., “Characterizing college science assessments: The three-dimensional learning assessment protocol,” PLoS One, vol. 11, no. 9, p. e0162333, 2016, doi: https://doi.org/10.1371/journal.pone.0162333.
[8] J. Sjöström, I. Eilks, and V. Talanquer, “Didaktik models in chemistry education,” J. Chem. Educ., vol. 97, no. 4, pp. 910–915, 2020, doi: https://doi.org/10.1021/acs.jchemed.9b01034.
[9] V. Talanquer, “Chemistry education: Ten facets to shape us,” J. Chem. Educ., vol. 90, no. 7, pp. 832–838, 2013, doi: https://doi.org/10.1021/ed300881v.
[10] J. H. Carmel, D. G. Herrington, L. A. Posey, J. S. Ward, A. M. Pollock, and M. M. Cooper, “Helping students to ‘do science’: Characterizing scientific practices in general chemistry laboratory curricula,” J. Chem. Educ., vol. 96, no. 3, pp. 423–434, 2019, doi: https://doi.org/10.1021/acs.jchemed.8b00912.
[11] C. Harris, J. Krajcik, J. W. Pellegrino, and A. H. DeBarger, “Designing knowledge-in-use assessments to promote deeper learning,” Educ. Meas. Issues Pract., vol. 38, no. 2, pp. 53–67, 2019, doi: https://doi.org/10.1111/emip.12253.
[12] J. R. Maeyer and V. Talanquer, “The role of intuitive heuristics in students’ thinking: Ranking chemical substances,” Sci. Educ., vol. 94, no. 6, pp. 963–984, 2010, doi: 1 https://doi.org/0.1002/sce.20397.
[13] L. Smith, J. R. Paddock, J. Vaughan, and D. W. Parkin, “Promoting nursing students’ chemistry success in a collegiate active learning environment: ‘If I have hope, I will try harder,’” J. Chem. Educ., vol. 95, no. 11, pp. 1929–1938, 2018, doi: https://doi.org/10.1021/acs.jchemed.8b00201.
[14] Y. Jin, C. A. Rodriguez, L. Shah, and G. T. Rushton, “Examining the psychometric properties of the Redox Concept Inventory: A Rasch approach,” J. Chem. Educ., vol. 97, no. 12, pp. 4235–4244, 2020, doi: https://doi.org/10.1021/acs.jchemed.0c00479.
[15] K. Adu-Gyamfi and B. Anim-Eduful, “Interaction effect of gender, across school-type on upper-secondary students’ development of experimental reasoning on organic qualitative analysis,” J. Balt. Sci. Educ., vol. 21, no. 3, pp. 351–364, 2022, doi: https://doi.org/10.33225/jbse/22.21.351.
[16] K. Adu-Gyamfi and I. A. Asaki, “Teachers’ conceptual difficulties in teaching senior high school organic chemistry,” Contemp. Math. Sci. Educ., vol. 3, no. 2, p. ep22019, 2022, doi: https://doi.org/10.30935/conmaths/12382.
[17] L. N. Amsad, S. Liliasari, A. Kadarohman, and R. E. Sardjono, “Students’ difficulties in solving synthesis organic compound problems,” Unnes Sci. Educ. J., vol. 8, no. 3, 2019, doi: https://doi.org/10.15294/usej.v8i3.32424.
[18] D. L. Danipog and M. B. Ferido, “Using art-based chemistry activities to improve students’ conceptual understanding in chemistry,” J. Chem. Educ., vol. 88, no. 12, pp. 1610–1615, 2011, doi: https://doi.org/10.1021/ed100009a.
[19] B. Wei, “Reconstructing a school chemistry curriculum in the era of core competencies: A case from China,” J. Chem. Educ., vol. 96, no. 7, pp. 1359–1366, 2019, doi: https://doi.org/10.1021/acs.jchemed.9b00211.
[20] J. Fischmann, P. Verboon, and R. v. Geel, “Elementary tutorial for multilevel analysis using visualizations,” 2021, doi: https://doi.org/10.31234/osf.io/7nzx2.
[21] L.-T. Tsai and C.-C. Yang, “Hierarchical effects of school-, classroom-, and student-level factors on the science performance of eighth-grade Taiwanese students,” Int. J. Sci. Educ., vol. 37, no. 8, pp. 1166–1181, 2015, doi: https://doi.org/10.1080/09500693.2015.1022625.
[22] J. Wu, Y.-H. Lee, and J. J. H. Lin, “Using iMCFA to perform the CFA, multilevel CFA, and maximum model for analyzing complex survey data,” Front. Psychol., vol. 9, 2018, doi: https://doi.org/10.3389/fpsyg.2018.00251.
[23] J. W. K. Yeung, “The dynamic relationships between educational expectations and science learning performance among students in secondary school and their later completion of a STEM degree,” Behav. Sci., vol. 14, no. 6, p. 506, 2024, doi: https://doi.org/10.3390/bs14060506.
[24] B. T. Moges, M. M. Gebremeskel, S. A. Tilwani, and Y. Assefa, “Student engagement in a differentiated higher education system in Ethiopia: A multilevel analysis,” J. Appl. Res. High. Educ., vol. 16, no. 5, pp. 1341–1354, 2024, doi: https://doi.org/10.1108/jarhe-11-2023-0507.
[25] O. Ersan and M. C. Rodriguez, “Socioeconomic status and beyond: A multilevel analysis of TIMSS mathematics achievement given student and school context in Turkey,” Large-Scale Assessments Educ., vol. 8, no. 1, 2020, doi: https://doi.org/10.1186/s40536-020-00093-y.
[26] B. Setiaji, P. H. Santoso, K. N. Aziz, H. Retnawati, and M. Khairudin, “Using multilevel modelling to evaluate science literacy and technology course of the Indonesian non-science students,” J. Pendidik. Ipa Indones., vol. 12, no. 1, pp. 96–111, 2023, doi: https://doi.org/10.15294/jpii.v12i1.41457.
[27] S. A. Priyambada, M. ER, B. N. Yahya, and T. Usagawa, “Profile-based cluster evolution analysis: Identification of migration patterns for understanding student learning behavior,” IEEE Access, vol. 9, pp. 101718–101728, 2021, doi: https://doi.org/10.1109/access.2021.3095958.
[28] V. Revathi, K. B. Ghutugade, R. Vannapuram, and B. P. S. Prasanna, “K-means algorithm for clustering of learners performance levels using machine learning techniques,” Rev. D Intell. Artif., vol. 35, no. 1, pp. 99–104, 2021, doi: https://doi.org/10.18280/ria.350112.
[29] C. Romero and S. Ventura, “Educational data mining and learning analytics: An updated survey,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 10, no. 3, 2020, doi: https://doi.org/10.1002/widm.1355.
[30] T. Yıldırım, “Trends in PhD theses in Turkish chemistry education (1999–2019),” Eurasian J. Educ. Res., vol. 20, no. 89, pp. 1–40, 2020, doi: https://doi.org/10.14689/ejer.2020.89.10.
[31] M. Schultz, J. Lai, J. P. Ferguson, and S. Delaney, “Topics amenable to a systems thinking approach: Secondary and tertiary perspectives,” J. Chem. Educ., vol. 98, no. 10, pp. 3100–3109, 2021, doi: https://doi.org/10.1021/acs.jchemed.1c00203.
[32] S. York, R. Lavi, Y. J. Dori, and M. Orgill, “Applications of systems thinking in STEM education,” J. Chem. Educ., vol. 96, no. 12, pp. 2742–2751, 2019, doi: https://doi.org/10.1021/acs.jchemed.9b00261.
[33] B. Flynn et al., “Future directions for systems thinking in chemistry education: Putting the pieces together,” J. Chem. Educ., vol. 96, no. 12, pp. 3000–3005, 2019, doi: https://doi.org/10.1021/acs.jchemed.9b00637.
[34] P. G. Mahaffy et al., “Beyond ‘inert’ ideas to teaching general chemistry from rich contexts: Visualizing the chemistry of climate change (VC3),” J. Chem. Educ., vol. 94, no. 8, pp. 1027–1035, 2017, doi: https://doi.org/10.1021/acs.jchemed.6b01009.
[35] D. J. C. Constable, C. Jiménez‐González, and S. A. Matlin, “Navigating complexity using systems thinking in chemistry, with implications for chemistry education,” J. Chem. Educ., vol. 96, no. 12, pp. 2689–2699, 2019, doi: https://doi.org/10.1021/acs.jchemed.9b00368.
[36] H. Tümay, “Systems thinking in chemistry and chemical education: A framework for meaningful conceptual learning and competence in chemistry,” J. Chem. Educ., vol. 100, no. 10, pp. 3925–3933, 2023, doi: https://doi.org/10.1021/acs.jchemed.3c00474.
[37] M. A. Al-Hagery, M. A. Alzaid, T. S. Alharbi, and M. A. Alhanaya, “Data mining methods for detecting the most significant factors affecting students’ performance,” Int. J. Inf. Technol. Comput. Sci., vol. 12, no. 5, pp. 1–13, 2020, doi: 1 https://doi.org/0.5815/ijitcs.2020.05.01
[38] M. Usher, A. Hershkovitz, and A. Forkosh‐Baruch, “From data to actions: Instructors’ decision making based on learners’ data in online emergency remote teaching,” Br. J. Educ. Technol., vol. 52, no. 4, pp. 1338–1356, 2021, doi: https://doi.org/10.1111/bjet.13108.
[39] Y. Tsai, O. Poquet, D. Gašević, S. Dawson, and A. Pardo, “Complexity leadership in learning analytics: Drivers, challenges and opportunities,” Br. J. Educ. Technol., vol. 50, no. 6, pp. 2839–2854, 2019, doi: https://doi.org/10.1111/bjet.12846.
[40] M. v. Geel, T. Keuning, A. J. Visscher, and J. Fox, “Assessing the effects of a school-wide data-based decision-making intervention on student achievement growth in primary schools,” Am. Educ. Res. J., vol. 53, no. 2, pp. 360–394, 2016, doi: https://doi.org/10.3102/0002831216637346.
[41] M. Botvin, A. Hershkovitz, and A. Forkosh‐Baruch, “Data-driven decision-making in emergency remote teaching,” Educ. Inf. Technol., vol. 28, no. 1, pp. 489–506, 2022, doi: https://doi.org/10.1007/s10639-022-11176-4.
[42] A. S. Musyarofah, A. L. Anggraini, S. Sudarti, R. D. Handayani, M. Jamhari, and H. Haeruddin, “Analysis of the comparison of science literacy skills of students at MTS Nurul Huda Situbondo in solving PISA science problems,” Edukasia, vol. 4, no. 2, pp. 2507–2516, 2023, doi: https://doi.org/10.62775/edukasia.v4i2.619.
[43] K. A. Kırkıç and F. Uludağ, “STEM attitudes of students as predictor of secondary school technology and design course achievement,” Probl. Educ. 21st Century, vol. 79, no. 4, pp. 585–596, 2021, doi: https://doi.org/10.33225/pec/21.79.585.
[44] M. Mejía‐Rodríguez and L. Kyriakidēs, “What matters for student learning outcomes? A systematic review of studies exploring system‐level factors of educational effectiveness,” Rev. Educ., vol. 10, no. 3, 2022, doi: https://doi.org/10.1002/rev3.3374.
[45] A. Dewi and Y. Yahdi, “Research trends on socio-scientific issues in chemistry learning: A systematic review,” J. Pendidik. Mipa, vol. 26, no. 1, pp. 457–475, 2025, doi: https://doi.org/10.23960/jpmipa.v26i1.pp457-475.
[46] N. Kravchenko and N. Mikheeva, “Theoretical construct of young learners’ intercultural competence and its practical application,” Bull. South Ural State Univ. Ser. Educ. Educ. Sci., vol. 15, no. 3, pp. 32–41, 2023, doi: https://doi.org/10.14529/ped230303.
[47] F. Daniels, L. P. Fakude, N. S. Linda, and R. R. M. Modeste, “Nurse educators’ experiences of case-based education in a South African nursing programme,” Curationis, vol. 38, no. 2, 2015, doi: https://doi.org/10.4102/curationis.v38i2.1523.
[48] T. M. Moore et al., “Development of an abbreviated form of the Penn Line Orientation Test using large samples and computerized adaptive test simulation,” Psychol. Assess., vol. 27, no. 3, pp. 955–964, 2015, doi: https://doi.org/10.1037/pas0000102.
[49] S. Xu, A. Song, X. Li, J. Nie, C. Zhang, and Y. Wang, “Performance of the automated COBAS AmpliPrep/COBAS TaqMan HIV-1 test on a genetically diverse panel of specimens from China: Comparison to the COBAS Amplicor HIV-1 monitor test, v1.5,” Intervirology, vol. 53, no. 4, pp. 221–228, 2010, doi: https://doi.org/10.1159/000299064.
[50] J. Afonso, R. Ramírez‐Campillo, F. M. Clemente, F. Büttner, and R. Andrade, “The perils of misinterpreting and misusing ‘publication bias’ in meta-analyses: An education review on funnel plot-based methods,” Sport. Med., vol. 54, no. 2, pp. 257–269, 2023, doi: https://doi.org/10.1007/s40279-023-01927-9.
[51] L. Padilla, I. T. Ruginski, and S. H. Creem-Regehr, “Effects of ensemble and summary displays on interpretations of geospatial uncertainty data,” Cogn. Res. Princ. Implic., vol. 2, no. 1, 2017, doi: https://doi.org/10.1186/s41235-017-0076-1.
[52] R. P. Hickson, I. E. Annis, L. A. Killeya‐Jones, and G. Fang, “Opening the black box of the group‐based trajectory modeling process to analyze medication adherence patterns: An example using real‐world statin adherence data,” Pharmacoepidemiol. Drug Saf., vol. 29, no. 3, pp. 357–362, 2019, doi: https://doi.org/10.1002/pds.4917.
[53] M. Stieff, M. Hegarty, and G. Deslongchamps, “Identifying representational competence with multi-representational displays,” Cogn. Instr., vol. 29, no. 1, pp. 123–145, 2011, doi: https://doi.org/10.1080/07370008.2010.507318.
[54] H. Wu and P. Shah, “Exploring visuospatial thinking in chemistry learning,” Sci. Educ., vol. 88, no. 3, pp. 465–492, 2004, doi: https://doi.org/10.1002/sce.10126.
Refbacks
- There are currently no refbacks.
