Sources of error in web surveys: the role of respondent, design and device in data quality

This thesis investigates how the design of response scales, the type of device used to respond to the survey, and respondent characteristics affect measurement error in web surveys. Chapter 1 explores measurement error associated with different response formats in the National Student Survey, a cross-sectional web survey in the UK. Respondents who accessed the survey on a mobile device were randomly allocated to radio-button grids and drop-down response scale designs. Respondents using a PC viewed all questions in a radio-button format. Applying the radio-button design to tablets, smartphones and PCs reduces measurement error between devices. Drop-down lists were shown to reduce straightlining in grids on smartphones but increase primacy effects and affect response distributions. Chapter 2 looks at scale direction effects in an online panel experiment administering survey questions with either ‘forward’ or ‘reversed’ ordered response scales. The ‘forward’ scale design results in higher selection of the high/positive responses whereas effects for the ‘reversed’ scale are less pronounced. Respondent age, education, gender and extraversion trait are associated with scale direction effects, suggesting the role of satisficing and anchoring mechanisms. The ‘forward’ scale design reduces selection of high/positive responses among conscientious respondents. Chapter 3 explores the use of mobile devices for survey completion in an online cohort study. I find that respondents with a higher need for cognition and higher extraversion are more likely to use a PC for survey completion or switch between smartphones and PCs. More agreeable respondents are likely to use smartphones or tablets in addition to PCs for survey completion. When controlling for these respondent characteristics, I find that item non-response is higher for mobile than PC respondents. Collectively, these findings provide practical implications for survey designers with regards to scale design and mobile optimization, but also emphasize the role of respondent characteristics when predicting measurement error.