White Rose University Consortium logo
University of Leeds logo University of Sheffield logo York University logo

Estimating the robustness of questionnaire results: lessons from a mixed-mode survey of expectations for teleworking & road-based business travel

Bonsall, PW and Shires, JD (2009) Estimating the robustness of questionnaire results: lessons from a mixed-mode survey of expectations for teleworking & road-based business travel. Transportation, 36 (1). 47 - 64 . ISSN 0049-4488

Full text available as:

Abstract

The robustness of questionnaire results to various forms of bias are explored in the context of a dual-mode (web and hardcopy) survey of employers’ anticipations of levels of employee commuting and business travel activity under a range of future ICT scenarios. The questionnaire incorporated several innovative features which, together with the dual-mode format, allowed an unusually wide range of analyses. For example: the robustness of respondents’ opinions was tested by examining the effect of incorporating alternative versions of a briefing text, one being very positive and one very negative, about the role of ICT; instrument bias was identified via detailed comparison of the results from the two versions of the questionnaire; and the impact of exogenous factors which are often ignored or taken as constant was assessed via special supplementary questions. Analysis showed that the robustness of opinions and expectations varied and was influenced by respondent characteristics, and that results from the two versions of the questionnaire differed significantly. It is concluded that opinions and expectations are less robust, and questionnaire results are more subject to bias and myopic interpretation, than is generally recognised and that web-based surveys seem particularly vulnerable to sampling bias. Methods are suggested for measuring robustness, for reducing bias and for validating and contextualising results. The use of contrasting briefing texts is recommended as a means of establishing the robustness of opinions and expectations while supplementary questions are recommended for validating and contextualising SP and SE exercises.

Item Type: Article
Institution: The University of Leeds
Academic Units: The University of Leeds > Faculty of Environment (Leeds) > Institute for Transport Studies (Leeds)
Depositing User: Symplectic Publications
Date Deposited: 23 Jan 2012 12:40
Last Modified: 08 Feb 2013 17:36
Published Version: http://dx.doi.org/10.1007/s11116-008-9185-5
Status: Published
Publisher: Springer Verlag
Identification Number: 10.1007/s11116-008-9185-5
URI: http://eprints.whiterose.ac.uk/id/eprint/43656

Actions (repository staff only: login required)