Development and validation of an integrated model for evaluating e-service quality, usability and user experience (e-SQUUX) of Web-based applications in the context of a University web portal

Abstract

Developments in Internet technology and pervasive computing over the past two and half decades have resulted in a variety of Web-based applications (WBAs) that provide products and services to online users or customers. The Internet is used not only to transfer information via the web but is increasingly used to provide electronic services including business transactions, information-delivery and social networking, as well as e-government, e-health and e-learning. For such organisations, e-service quality, usability and user experience are considered to be critical determinants of their products’ or services’ success. Many studies to model these three concepts separately have been undertaken as part of broader studies of software quality or service quality modelling. However, to the current researcher’s knowledge, none of the studies have focussed on proposing an evaluation model that integrates and combines the three of them. This research is an effort to fill that gap. The primary purpose of this mixed-methods research was to develop a conceptual integrated model for evaluating e-service quality, usability and user experience (e-SQUUX) of WBAs and then contextualise it to evaluation of a University web portal (UWP). This was undertaken using an exploratory sequential research design. During a qualitative phase, an extensive extensive systematic literature review of 264 relevant sources relating to dimensions of e-service quality, usability and user experience, was undertaken to derive an integrated conceptual e-service quality, usability and user experience (e-SQUUX) Model for evaluating WBAs. The model was then empirically refined through a sequential series of validations, thus developing various versions of the e-SQUUX Model. First, it was content validated by a set of four expert reviewers. Second, during the quantitative phase, in the context of a University web portal, a questionnaire survey was conducted that included a comprehensive pilot study with 29 partipants, prior to the main survey. The main survey data from 174 particiapants was used to determine a validated model, using Exploratory factor analysis (EFA), followed by producing a structural model, using partial least square – structural equation modelling (PLS-SEM). This version consisted of the components of the final e-SQUUX Model. Consequently, the research enriches the body of knowledge on IS and HCI by providing the e-SQUUX Model as an evaluation tool. For designers, developers and managers of UWPs, the model serves as a customisable set of evaluation criteria and also provides specific recommendations for design. In line with the Exploratory sequential design of mixed methods research, the findings of the qualitative work in this research influenced the subsequent quantitative study, since the potential Likert-scale questionnaire items were derived from the definitions and meanings of the components that emanated from the qualitative phase of the study. Consequently, this research is an exemplar for developing an integrated evaluation model for specific facets or domains, and of its application in a particular context, in this case, a University web portal. Keywords: e-service quality, usability, user experience, evaluation model, integrated model, exploratory factor analysis, partial least square – structural equation modelling (PLS-SEM), mixed methods research, Exploratory sequential design, quantitative study, qualitative study, validation, Web-based applications, University web portal