Measurement model equivalence in web- and paper-based surveys

Abstract

The aim of this research is to investigate whether web-based and paper-based organisational climate surveys can be regarded as equivalent techniques of data collection. Due to the complex geographical placement of various units of the participating organisation and limited internet access, both paper-based and web-based questionnaires were used. Overall, 1295 employees participated in the survey; of these, 899 used paper questionnaires and 396 used the web-based questionnaire. 9Confi rmatory factor analysis (CFA) in a Structural Equation Modelling (SEM) framework was used to test the tenability of a series of increasingly restrictive models, using goodness-of-fi t tests. 10The SurveyTracker software survey package was used for the web survey, and SPSS with LISREL was used for the statistical analysis. Several measurement models were tested; four models showed very good fi ts for both the web-based and paper-based surveys for fit measures such as RMSEA, NFI, NNFI, PNFI CFI, IFI, RFI and BIC. The four path diagrams also allowed the researcher to investigate the two groups of participants’ responses and fi t measures across diff erent aspects of the organisation, such as strategic issues (4 dimensions), human resource issues (4 dimensions), operational issues and job satisfaction (5 dimensions), and leadership and transformational issues (7 dimensions). 11The results indicate that hypotheses H1 to H3 are all tenable. It can therefore be concluded that the web-based and paper-based surveys can be considered equal with respect to similar factor structure, with equal factor loadings and equal variances of the factors and equal covariances between the factors. The results may therefore be combined in a single analysis without compromising measurement validity.