A critical review on the reporting of surveys in transdisciplinary research : a case study in Information Systems

Abstract

Variability of goals and evolving research methodologies are fundamental characteristics of transdisciplinary research. This integration of research strategies from different fields complicates the evaluation of transdisciplinary research since the variability of goals drives variability of criteria and quality indicators. The aim of this research is to investigate the implications of using research methods across disciplinary boundaries by drilling down into the use of one research strategy in one research context (Information Systems) and a related sub-context (Human-Computer Interaction). Surveys with questionnaires as data-capturing tools were selected as an established research method which is widely used in transdisciplinary research. Questionnaires are one of the most established data capturing tools and yet the validity of questionnaire-based findings have often been questioned. The main problem areas have been identified as the sampling of the data, the questionnaire design and the interpretation of the results. This paper looks into questionnaire reporting practices - an essential determinant in the validity and reliability of survey-based research. The field of Information Systems and Human- Computer Interaction has been chosen as the research context. Information Systems research is by nature interdisciplinary in focusing on social and organisational issues regarding the development and use of software in organisations. Human-Computer Interaction studies address the challenges of making computers and computations useful, usable, and universally accessible to humans. Both Information Systems and Human- Computer Interaction studies address complex, heterogeneous, real-world problems, thereby meeting the first criteria to be classified as transdisciplinary research. The research design entails document analysis of papers presented at conferences in Computer Science and Information Systems over a three-year period to identify trends in the reporting of survey results, especially the questionnaire design. Transdisciplinary research methodology facilitates the application of research methods across fields. However, if the constraints of the method are not recognised the validity of the results may be compromised in a plethora of ways. While fusion of methods are encouraged on a theoretical level in transdisciplinary research the findings of this study are a warning about the dangers of interdisciplinary application of research strategies without due diligence in observing best practices in the parent discipline. The paper aims to advance the discussion on research design and practice beyond disciplinary research and should be of interest to researchers and practitioners who deal with multidisciplinary, interdisciplinary, and transdisciplinary research.