The evaluation of a digital information literacy program

Abstract

The thesis reports on the evaluation of a digital information literacy program (DILP) to determine the program’s effectiveness in enhancing students’ digital information literacy skills. The program was originally designed and developed for the South African student, as member of Generation Y, but was adapted to suit the demographics and characteristics of Generation Z. New learning technologies were incorporated to enhance students’ learning experience. One of the characteristics of information literacy programs that illustrate best practice is the evaluation of the program itself to judge it’s effectiveness and validate the program as a learning tool. A review of the literature confirmed the paucity of the evaluation of such programs using assessment of student learning through outcomes assessment instruments, based on information literacy competency standards, designed with proven validity and reliability. The literature review found no evidence of the evaluation of the effectiveness of such programs through meaningful assessment of student learning using outcomes assessment in South Africa. For these reasons, the evaluation of the DILP was undertaken. To evaluate the effectiveness of the DILP, a non-randomised quasi-experimental research design, focusing on a single-group pre-test/post-test design which incorporated a combined quantitative and qualitative research approach was used. The primary research instrument was a pre- and post-test. A group of students, belonging to Generation Y and Z, completed a pre-test, worked through the DILP and completed a post-test. Telephonic and e-mail interviews were used to collect further data. The statistical analysis is presented by using descriptive statistics (stacked bar charts for the quantitative data and pie charts for the qualitative data). Inferential statistics were used to reach conclusions beyond the immediate data presented in the charts. The final step was to judge the overall effectiveness of the DILP. The difference between the means was statistically significant, indicating that the DILP was effective in enhancing the digital information literacy skills of students. Based on this research, additional research could be the evaluation of a DILP designed specifically for “digital natives”; the development of online outcomes assessment instruments for web-based tutorials with proven validity and reliability and research in the area of integrating emerging learning technologies with such programs, evaluating their effectiveness.