Research Article
A Study on Factors Affecting Response Rate, Response Speed, and Data Quality in E-Mail Surveys
Published: January 2003 · Vol. 32, No. 2 · pp. 475-498
Full Text
Abstract
This study investigates which factors affect respondent reactions—namely response rate, response speed, and response quality—in email surveys. The six factors hypothesized to influence respondent reactions in this study are: personalization, the nature of the sponsoring organization, respondent incentives, questionnaire format, advance notification, and follow-up letters. To test the validity of the six hypotheses, an experiment was conducted with email users. For this purpose, 17,480 individuals identified through a specialized email search engine were assigned to 64 experimental groups through randomization procedures and exposed to different values of the independent variables, and respondent reactions were observed. The experiment was conducted over a total of 44 days, resulting in 652 responses for an overall response rate of 3.73%. The data analysis using ANOVA and chi-square tests revealed that five of the six factors—personalization, nature of the sponsoring organization, respondent incentives, questionnaire format, and follow-up letters—excluding advance notification, showed effects on respondent reactions that were at least partially consistent with the hypotheses. However, none of the six factors showed the hypothesized effects on response speed. Specifically, as hypothesized, personalization had a positive effect on response rate and response quality. Having an academic institution as the sponsoring organization also showed positive effects on response rate and response quality compared to a corporate sponsor. Meanwhile, respondent incentives did not affect response rate or response speed, but showed a positive effect on response quality consistent with the hypothesis, although the evidence was partial. Questionnaire format, as hypothesized, was found to influence response rate, and there was partial evidence of its effect on response quality. Additionally, sending follow-up letters increased the response rate as hypothesized, while showing no effect on response speed or response quality.
