Abstract
Objectives
This study tests whether juveniles’ responses on sensitive topics such as self-reported delinquency, victimization, and substance use are comparable when teachers versus external persons supervise students while filling out online questionnaires.
Methods
Eighty classes with 1,197 students (9th grade) in eastern Switzerland were randomly assigned (at the class level) to supervision either by their teacher or by an external person (i.e., researcher). Students filled out online questionnaires about self-reported delinquency, victimization, and substance use in the classroom while being supervised either by their teacher or by an external person. Prevalence rates were compared using Chi-square tests.
Results
Only three out of 57 comparisons show significantly different outcomes. Whenever differences are found, and contrary to our expectation, sensitive experiences are more often admitted when students are supervised by their teacher. Effect sizes do not exceed 0.6, and are thus all below the limit of a small effect.
Conclusions
Using online questionnaires with teachers as supervisors may not affect validity while making surveys less expensive and intrusive.
This is a preview of subscription content, log in via an institution to check access.
References
Aquilino, W. S. (1994). Interview mode effects in surveys of drug and alcohol use: A field experiment. Public Opinion Quarterly, 58(2), 210–240.
Beebe, T. J., Harrison, P. A., McRae, J. A., Anderson, R. E., & Fulkerson, J. A. (1998). An evaluation of computer-assisted self-interviews in a school setting. Public Opinion Quarterly, 62(4), 623–632.
Bennett, S., Parpia, T., Hayes, R., & Cousens, S. (2002). Methods for the analysis of incidence rates in cluster randomized trials. International Journal of Epidemiology, 31(4), 839–846.
Bland, J. M. (2004). Cluster randomised trials in the medical literature: Two bibliometric surveys. BMC Medical Research Methodology, 4(1), 21–26.
Brener, N. D., Billy, J. O. G., & Grady, W. R. (2003). Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. Journal of Adolescent Health, 33(6), 436–457.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.
Gans, J. E., & Brindis, C. D. (1995). Choice of research setting in understanding adolescent health problems. Journal of Adolescent Health, 17(5), 306–313.
Gribble, J. N., Miller, H. G., Rogers, S. M., & Turner, C. F. (1999). Interview mode and measurement of sexual behaviors: Methodological issues. Journal of Sex Research, 36(1), 16–17.
Hallfors, D., Khatapoush, S., Kadushin, C., Watson, K., & Saxe, L. (2000). A comparison of paper vs computer-assisted self-interview for school alcohol, tobacco, and other drug surveys. Evaluation and Program Planning, 23(2), 149–155.
Hindelang, M. J., Hirschi, T., & Weis, J. G. (1979). Correlates of delinquency: The illusion of discrepancy between self-report and official measures. American Sociological Review, 44(6), 995–1014.
Kann, L., Brener, N. D., Warren, C. W., Collins, J. L., & Giovino, G. A. (2002). An assessment of the effect of data collection setting on the prevalence of health risk behaviors among adolescents. Journal of Adolescent Health, 31(4), 327–335.
Kivivuori, J. & Salmi, V. (2011). Supervision mode effects in school-based delinquency and victimization survey: Preliminary test (Research Brief No. 20/2011).
Lucia, S., Herrmann, L., & Killias, M. (2007). How important are interview methods and questionnaire designs in research on self-reported juvenile delinquency? An experimental comparison of Internet vs paper-and-pencil questionnaires and different definitions of the reference period. Journal of Experimental Criminology, 3, 39–64.
Thornberry, T. P., & Krohn, M. D. (2000). The self-report method for measuring delinquency and crime. In D. Duffee (Ed.), Measurement and analysis of crime and justice (Criminal Justice 2000, Vol. 4, pp. 33–83). Washington, DC: National Institute of Justice, Office of Justice Programs, US Department of Justice.
Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60(2), 275–304.
Trapl, E. S. (2007). Understanding adolescent survey responses: Impact of mode and other characteristics on data outcomes and quality. Cleveland: Case Western Reserve University.
Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873.
Vereecken, C. A., & Maes, L. (2006). Comparison of a computer-administered and paper-and-pencil-administered questionnaire on health and lifestyle behaviors. Journal of Adolescent Health, 38(4), 426–432.
Weisband, S., & Kiesler, S. (1996). Self-disclosure on computer forms: Meta-analysis and implications. In R. Bilger, S. Guest, & M. J. Tauber (Eds.), Proceedings of the SIGCHI conference on human factors in computing systems: Common ground (pp. 3–10). Vancouver: ACM.
Wright, D. L., Aquilino, W. S., & Supple, A. J. (1998). A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly, 62(3), 331–353.
Acknowledgements
This survey was mandated by the Government of the Canton of St. Gallen, Switzerland (Department of Security & Justice and Department of Education) and carried out by the Institute of Criminology, University of Zurich, Switzerland.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Walser, S., Killias, M. Who should supervise students during self-report interviews? A controlled experiment on response behavior in online questionnaires. J Exp Criminol 8, 17–28 (2012). https://doi.org/10.1007/s11292-011-9129-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11292-011-9129-5