Assessment of the quality of science education textbooks : conceptual framework and instruments for analysis

Abstract

Science and technology are constantly transforming our day-to-day living. Science education has become of vital importance to prepare learners for this everchanging world. Unfortunately, science education in South Africa is hampered by under-qualified and inexperienced teachers. Textbooks of good quality can assist teachers and learners and facilitate the development of science teachers. For this reason thorough assessment of textbooks is needed to inform the selection of good textbooks. An investigation revealed that the available textbook evaluation instruments are not suitable for the evaluation of the physical science textbooks in the South African context. An instrument is needed that focusses on science education textbooks and which prescribes the criteria, weights, evaluation procedure and rating scheme that can ensure justifiable, transparent, reliable and valid evaluation results. This study utilised elements from the Analytic Hierarchy Process (AHP) to develop such an instrument and verified the reliability and validity of the instrument’s evaluation results. Development of the Instrument for the Evaluation of Science Education Textbooks started with the formulation of criteria. Characteristics that influence the quality of textbooks were identified from literature, existing evaluation instruments and stakeholders’ concerns. In accordance with the AHP, these characteristics or criteria were divided into categories or branches to give a hierarchical structure. Subject experts verified the content validity of the hierarchy. Expert science teachers compared the importance of different criteria. The data were used to derive weights for the different criteria with the Expert Choice computer application. A rubric was formulated to act as rating-scheme and score sheet. During the textbook evaluation process the ratings were transferred to a spreadsheet that computed the scores for the quality of a textbook as a whole as well as for the different categories. The instrument was tested on small scale, adjusted and then applied on a larger scale. The results of different analysts were compared to verify the reliability of the instrument. Triangulation with the opinions of teachers who have used the textbooks confirmed the validity of the evaluation results obtained with the instrument. Future investigations on the evaluation instrument can include the use of different rating scales and limiting of criteria.