A methodology for evaluating intelligent tutoring systems

Abstract

This dissertation proposes a generic methodology for evaluating intelligent tutoring systems (ITSs), and applies it to the evaluation of the SQL-Tutor, an ITS for the database language SQL. An examination of the historical development, theory and architecture of intelligent tutoring systems, as well as the theory, architecture and behaviour of the SQL-Tutor sets the context for this study. The characteristics and criteria for evaluating computer-aided instruction (CAl) systems are considered as a background to an in-depth investigation of the characteristics and criteria appropriate for evaluating ITSs. These criteria are categorised along internal and external dimensions with the internal dimension focusing on the intrinsic features and behavioural aspects of ITSs, and the external dimension focusing on its educational impact. Several issues surrounding the evaluation of ITSs namely, approaches, methods, techniques and principles are examined, and integrated within a framework for assessing the added value of ITS technology for instructional purposes.