A methodology for the evaluation of management information systems at public technical and vocational education and training colleges in South Africa

Abstract

The support and promotion of public Technical and Vocational Education and Training (TVET) Colleges is fundamental in addressing South Africa’s intermediate-level and artisanal skills as shortages in these areas contribute to considerable unemployment in South Africa. These institutions have been earmarked by the South African government for extensive growth. Therefore, efficient and effective management and accurate decision-making within these institutions are essential. The evaluation of the management information systems (MISs) within these institutions, which provide data and information to inform institutional short-term and long-term management decision-making and day-to-day operations, should take place on a regular basis to so enhance the reliability and accuracy of the data and information. The problem is that no evidence of a methodology (artefact) for the evaluation of MISs at public TVET Colleges in South Africa could be found in the literature. Therefore, the rationale for this study is to develop a methodology for the evaluation of MISs at public TVET Colleges in South Africa. Hence the main research question for the study was formulated as: What are the components that constitute a methodology for the evaluation of a MIS at a public TVET College in South Africa? The study was conducted according to a design science paradigm. Design science is underpinned by a pragmatic philosophical paradigm which considers thought as a tool for prediction, problem solving and action. The Design Science Research Process (DSRP) model informed the research process utilised to develop the artefact for this problem centred initiated study. The iterated activities of the DSRP model which include: design, demonstrate, evaluate and communicate, contributed to the refinement of the methodology (artefact). The artefact mainly underwent experimental evaluation to demonstrate its applicability. The methodology (artefact) was empirically evaluated at three cluster-random selected public TVET Colleges after all colleges, with similar MIS maturity levels, were clustered into groups. The study contributed to the extant knowledge base of: theory building, on different levels. The main theoretical contribution is the final evaluated methodology (DSR artefact) which enables IT practitioners and MIS managers at public TVET Colleges in South Africa to evaluate their MISs on a regular basis. The methodology (artefact) presents a theory for design and action which satisfies the conditions of importance, parsimony and novelty on a micro-level. The study furthermore contributed to the extant literature on the theory of MIS success evaluation by contributing to theory on the measurement of MIS success constructs and measuring of the relationships between the constructs. Another theoretical contribution is the innovative evidence-based method by which the public TVET Colleges were clustered. The clustering method was used to ensure a more rigorous sample selection technique than purposive or convenient sample selection of cases and is generalisable to other knowledge domain contexts. The research study furthermore produced results of interest to both technology-focused and management-focused audiences. For technology-focused audiences the processes by which the artefact was constructed and evaluated are described, thus establishing repeatability of the study and building the knowledge base for further research extensions by future design science researchers. The rigour of the artefact design process was complemented by a thorough presentation of the experimental design of the artefact’s field test in three public TVET College environments which provides sufficient detail for management audiences to determine if sufficient organisational resources exist for utilisation of the artefact.