Using Mixed Methods to Explore the Validity of a Secondary School Monitoring System: A Case From South Africa

Abstract

The issue of quality education is a critical topic of discussion, for South Africa facing the challenge of implementation amid a plethora of progressive policies. The purpose of the research reported here is to describe the procedures undertaken to explore the feasibility of implementing a monitoring system in the South African context. The research was guided by two main research questions. The first or the main research question is how appropriate is the Middle Years Information System as a monitoring system in the South African context? This non experimental study used a mixed-methods design, rooted in pragmatism, to explore validity and reliability issues of using Middle Year Information system as a possible monitoring system that would provide a balanced view of the school’s contribution to academic gains made by students. The samples included in the study were from National Department of Education officials (two officials from curriculum and assessment); Provincial Department of Education officials (one mathematics specialist, one language specialist, and one specialist from the Gauteng Department of Education Office for Standards in Education); specialists in the field of language, mathematics, and psychology; and 11 schools. In particular, content-related validity (including curriculum validity), construct-related validity, and predictive validity were examined, whereas inferences drawn with regard to reliability were done by means of internal consistency reliability. The second main research question extends the first research question. If Middle Year Information System is valid, with South African adaptations, and reliable, then what factors on a school, classroom, and student-level could have an effect on student performance. Thus, the second main research question is which factors could have an effect on student performance and therefore inform the design of the monitoring system?