Research Article
Detection Methods of Careless Responses in Surveys and the Effects of Their Removal
1 Seoul National University
Published: January 2020 · Vol. 49, No. 2 · pp. 331-364
DOI: https://doi.org/10.17287/kmr.2020.49.2.331
Full Text
Abstract
This study investigates the phenomenon of ‘careless responding’ that prevails in surveys. Specifically, we focus on methods of detection and the effects of screening careless responding. First, this study discusses the diverse definitions and types of careless responding. Methods of detection are introduced in the following triangular categorization: direct, unobtrusive, and statistical. The theoretical review portion provides a comparative summary of detection methods and their advantages and disadvantages to help future researchers apply a suitable one to their own study designs. Furthermore, this study conducts an empirical study to identify the impact of screening careless responding on statistical results, analyzing survey data from 3,030 employees working in a major conglomerate in Korea. The major findings are as follows: (1) According to the applied method, the target and the proportion of screened responses varied, ranging from 0.5% to 14% at maximum. Only a small proportion was detected coincidentally by two or more methods. (2) The screening of careless responding only had a slight impact on statistical figures of factor analysis, correlations, and regression. Such less than moderate impacts were in line with earlier findings and less threatening to the validation of research models. (3) Lastly, we demonstrated a multi-hurdling method that adopted two methods sequentially. To conclude, the study discusses possible applications of detection methods and avenues for future research.
