The construction and evaluation of a dynamic computerised adaptive test for the measurement of learning potential

Abstract

Recent political and social changes in South Africa have created the need for culture-fair tests for cross-cultural measurement of cognitive ability. This need has been highlighted by the professional, legal and research communities. For cognitive assessment, dynamic assessment is more equitable because it involves a test-train-retest procedure, which shows what performance levels individuals are able to attain when relevant training is provided. Following Binet’s thinking, dynamic assessment aims to identify those individuals who are likely to benefit from additional training. The theoretical basis for learning potential assessment is Vygotsky’s concept of the zone of proximal development. This thesis describes the development, standardisation and evaluation of the Learning Potential Computerised Adaptive Test (LPCAT), for measuring learning potential in the culturally diverse South African population by means of nonverbal figural items. In accordance with Vygotsky’s view, learning potential is defined as a combination of present performance and the extent to which performance is increased after relevant training. This definition allows for comparison of individuals at different levels of initial performance and with different measures of improvement. Computerised adaptive testing based on item response theory, as used in the LPCAT, is uniquely suitable for increasing both measurement accuracy and testing efficiency of dynamic testing, two aspects that have been identified as problematic. The LPCAT pretest and the post-test are two separate adaptive tests, hence eliminating the role of memory in post-test performance. Several multicultural groups were used for item analysis and test validation. The results support the LPCAT as a culture-fair measure of learning potential in the nonverbal general reasoning domain. For examinees with a wide range of ability levels, LPCAT scores correlate strongly with academic performance. For African examinees, poor proficiency in English (the language of teaching) hampers academic performance. The LPCAT ensures the equitable measurement of learning potential, independent of language proficiency and prior scholastic learning and can be used to help select candidates for further training or developmental opportunities.