Investigation of the effect of ranking tasks of student understanding of key astronomy topics

Abstract

This research concerns the development and testing of a new type of introductory astronomy curriculum material called ranking tasks. Ranking tasks are a novel form of conceptual exercise in which students are presented with (usually) pictures or diagrams that describe up to six slightly different variations of a basic physical situation. Students are then asked to make a comparative judgment identifying the order or ranking of the various situations based on some physical outcome or result. These exercises are easily incorporated as collaborative group activities into the traditional lecture-based classroom. This study developed design guidelines for ranking tasks based on several learning theories and classroom pilot studies. A single-group repeated measures experiment was then conducted using eight key introductory astronomy topics with 250 students at the University of Arizona in the Fall of 2004. Our research questions were: * Do in-class collaborative ranking task exercises result in student conceptual gains when incorporated into traditional lecture-based instruction? * Are these gains significant in terms of effect size measures commonly used in education research? * What value do students perceive in the use of in-class ranking tasks exercises? The study found that average assessment test scores across the eight astronomy topics increased from 32% on the start-of-semester pretest to 61% after traditional lecture, to 77% after the ranking task exercises. A mixed factors ANOVA confirmed a significant rise in test scores after the ranking tasks (alpha = 0.05). The average normalized gain on the post-ranking task tests was 0.41, with a Cohen’s d effect size of 0.62 which is described in the literature as moderately large. Interestingly, we found that the normalized gain from the ranking tasks was equal to the entire previous gain from traditional instruction. A repeated-factor ANOVA found that the use of ranking tasks equally benefited both genders and also both high and low-scoring median groups on the pretest. A Lickert-scale attitude survey found that 83% of the students participating in the 16 week study believed that the ranking task exercises helped their understanding of core astronomy concepts. Based on these results, we assert that adding collaborative ranking task exercises to a traditional classroom instruction can significantly improve student understanding of key introductory astronomy concepts.