Design and Evaluation of a Reflective Learning Questionnaire for Enhancing Instruction in a Data Analytics and Machine Learning Course

Abstract

This prototype research aimed to: (1) design a reflective learning questionnaire for the Data Analytics and Machine Learning (DAML) course that evaluates both topic-level understanding and the ability to use data-analysis tools; (2) assess the preliminary instrument quality in terms of internal consistency, its relationship with academic achievement, and its ability to distinguish learners based on their understanding level; and (3) propose practical guidelines for using the questionnaire to enhance teaching and support personalized learning in technical-education contexts. This research was a quantitative prototype investigation using a post-course questionnaire as the data-collection tool. The sample was selected purposively and 19 out of 23 undergraduate students in Manufacturing System Engineering program enrolling in this course filled in the questionnaire voluntarily. Research tools included (1) the reflective learning questionnaire on clustering, association rules, regression, classification and image processing, (2) course grade, and (3) a comprehension test. Data analysis was done through using descriptive statistics (mean and standard deviation) and internal consistency (Cronbach's ). The results were as follows: (1) the questionnaire successfully separated the level of understanding between students among topics (regression was the most understood while image processing was the least); (2) students' self-evaluation correlated positively with their academic achievement, especially in topics involving real-world scenarios; and (3) the questionnaire exhibited acceptable overall reliability with a Cronbach's  of 0.82, whereby it can be used as a good diagnostic tool to facilitate personalized learning and digital-based teaching in the Data Analytics and Machine Learning course.