Skip to main content
SpringerLink
Log in

Costs and Errors in Fixed and Mobile Phone Surveys

  • Chapter
  • First Online:
Handbook of Survey Methodology for the Social Sciences

Abstract

Due to low costs, speed, simplicity, interviewer assistance, and easy monitoring telephone interviewing had been the preferred mode of many survey practitioners for decades. However, technological developments related to information society, in particular the increasing rate of mobile-only individuals and households, are rapidly changing the survey research environment. In the first part of the chapter methodological issues of data collection by phone are delineated. After a brief history of phone surveys recent telephone use trends and their implications on survey coverage and sampling are outlined. Next, nonresponse in phone surveys is discussed. The section ends with an illustration of challenges posed by incorporating mobile phone in survey research. In the second part, phone surveys are discussed in the context of mixed modes, in particular their potential to improve coverage and response rates. Furthermore, data quality and cost issues are treated. Finally, the section introduces dual frame sampling of fixed and mobile numbers, a special type of mixed mode surveys that was developed to resolve the phone coverage problem. In the third part, the chapter deals with the optimization of telephone surveys according to costs and errors. An analytical solution for dual frame surveys and a more general postsurvey evaluation of different modes are presented. In conclusion, we recapitulate key issues of phone surveys and indicate future trends with guidelines for further research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • AAPOR Cell Phone Task Force (2010) New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers. American Association for Public Opinion Research, from http://aapor.org/AM/Template.cfm?Section=Cell_Phone_Task_Force&Template=/CM/ContentDisplay.cfm&ContentID=2818.

  • Belak, E. (2007). Vpliv uporabe mobilnih telefonov na anketno zbiranje podatkov. Magistrsko delo. Ljubljana: FDV.

    Google Scholar 

  • Berzelak, N.,Vehovar, V. & Lozar Manfreda, K. (2008). What can we achieve with 5 Euros? optimization of survey data quality using mixed-mode approaches. Paper presented at the Applied Statistics Conference, Ribno

    Google Scholar 

  • Biemer, P. P. (2010). Total survey error: Design implementation and evaluation. Public Opinion Quarterly, 74(5), 817–848.

    Article  Google Scholar 

  • Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality. Hoboken, NJ: Wiley.

    Book  Google Scholar 

  • Blumberg, S. J., & Luke, J. V. (2009). Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2008. National Center for Health Statistics, from http://www.cdc.gov/nchs/nhis.htm.

  • Bosnjak, M., Neubarth, W., Couper, M. P., Bandilla, W., & Kaczmirek, L. (2008). Prenotification in web-based access panel surveys: The influence of mobile text messaging versus e-mail on response rates and sample composition. Social Science Computer Review, 26(2), 213–223.

    Article  Google Scholar 

  • Brick, J. M., Dipko, S., Presser, S., Tucker, C., & Yangyang, Y. (2006). Nonresponse bias in a dual frame sample of cell and landline numbers. Public opinion Quarterly, 70(5, Special Issue), 780–793.

    Google Scholar 

  • Brick, J. M., Edwards, W. S., & Sunghee, L. (2007). Sampling telephone numbers and adults, interview length, and weighting in the California health interview survey cell phone pilot study. Public Opinion Quarterly, 71(5), 793–813.

    Article  Google Scholar 

  • Brick, M. J., Cervantes, I. F., Lee, S., & Norman, G. (2011). Nonsampling Errors in Dual Frame Telephone Surveys. Survey Methodology 37 (1), 1–12.

    Google Scholar 

  • Callegaro, M., Steeh, C., Buskirk, T. D., Vehovar, V., Kuusela, V., & Piekarski, L. (2007). Fitting disposition codes to mobile phone surveys: Experiences from studies in Finland, Slovenia and the USA. Journal of the Royal Statistical Society: Series A (Statistics in Society), 170(3), 647–670.

    Article  Google Scholar 

  • Chang, L. C., & Krosnick, J. A. (2009). National surveys Via Rdd telephone interviewing versus the internet. Comparing sample representativeness and response quality. Public Opinion Quarterly, 73(4), 641–678.

    Article  Google Scholar 

  • Cochran, W. G. (1978). Sampling techniques (3rd ed.). New York, NY: Wiley.

    Google Scholar 

  • de Leeuw, E. D. (1992). Data quality in mail, telephone, and face-to-face surveys. Amsterdam: TT-Publicaties.

    Google Scholar 

  • de Leeuw, E. D. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21(2), 233–255.

    Google Scholar 

  • Deming, W. E. (1950). Some theory of sampling. New York, NY: Wiley.

    Google Scholar 

  • Deming, W. E. (1953). On a probability mechanism to attain an economic balance between the resultant error of response and the bias of nonresponse. Journal of the American Statistical Association, 48(264), 743–772.

    Article  Google Scholar 

  • Dillman, D. A., Smyth, J. D., & Christian, C. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken: Wiley.

    Google Scholar 

  • Dolnicar, S., Laesser, C., & Matus, K. (2009). Online versus paper: Format effects in tourism surveys. Journal of Travel Research, 47(3), 295–316.

    Article  Google Scholar 

  • Elliott, M., Little, R. J. A., & Lewitsky, S. (2000). Subsampling callbacks to improve survey efficiency. Journal of the American Statistical Association, 95, 730–738.

    Article  Google Scholar 

  • European Commission (2008a). Special Eurobarometer 293. E-communication Household Survey. TNS Opinion and social, from http://ec.europa.eu/public_opinion/archives/ebs/ebs_293_full_en.pdf

  • European Commission (2008b). Flash Eurobarometer 251. Public attitudes and perceptions in the Euro area. Gallup Europe, Brussels [Producer]. GESIS, Cologne [Publisher]. ZA4743, data set version 2.0.0. Description: http://info1.gesis.org/dbksearch13/sdesc2.asp?no=4743&db=e&doi=10.4232/1.10007 (July 29 2010).

  • Fuchs, M. (2007). Mobile web surveys: A preliminary discussion of methodological implications. In F. G. Conrad & M. F. Schober (Eds.), Envisioning the survey interview of the future (pp. 77–94). Hoboken, NJ: Wiley.

    Chapter  Google Scholar 

  • Gabler, S., & Häder, S. (1999). Erfahrungen beim Aufbau eines Auswahlrahmens für Telefonstichproben in Deutschland. ZUMA-Nachrichten Heft, 44, 45–61.

    Google Scholar 

  • Gallup Europe (2009, February). Presentation Technical workshop. Paper presented at the Eurobarometer Technical Workshop, Budapest, Hungary.

    Google Scholar 

  • Groves, R. M. (1989). Survey errors and survey costs. Hoboken, NJ: Wiley.

    Book  Google Scholar 

  • Groves, R. M. (2005, March). Past, present, and future. Washington, DC: Paper presented at the Total Survey Error Workshop.

    Google Scholar 

  • Groves, R. M. (2006, April). Survey budgets, cost models, and responsive survey designs. Washington, DC: Paper presented at the Survey Cost Workshop.

    Google Scholar 

  • Groves, R. M., & Kahn, R. L. (1979). Surveys by telephone: A national comparison with personal interviews. New York: Academic Press.

    Google Scholar 

  • Groves, R. M., & Lepkowski, J. M. (1982). Alternative dual frame mixed mode surveys designs. Proceedings of the Survey Research Methods Section, American Statistical Association.

    Google Scholar 

  • Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opinion Quarterly, 72(2), 167–189.

    Article  Google Scholar 

  • Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken, NJ: Wiley.

    Google Scholar 

  • Hansen, M. H., Hurwitz, W. N., & Madow, W. G. (1953). Sample survey methods and theory. New York, NY: Wiley.

    Google Scholar 

  • Harter, R., Mach, T., Wolken, J., & Chapline, J. (2007, June). Determining subsampling rates for nonrespondents. Paper presented at the Third International Conference on Establishment Surveys, Montréal, Canada.

    Google Scholar 

  • Hartley, H. O. (1962). Multiple frame surveys. Proceedings of the Social Statistics Section, American Statistical Association.

    Google Scholar 

  • Karr, A. F., & Last, M. (2006). Survey Costs: Workshop Report and White Paper.

    Google Scholar 

  • Keeter, S., Kennedy, C., Clark, A., Tompson, T., & Mokrzycki, M. (2007). What’s missing from national landline RDD surveys? Public Opinion Quarterly, 71(5), 772–792.

    Article  Google Scholar 

  • Kennedy, C. (2007). Evaluating the effects of screening for telephone service in dual frame RDD surveys. Public opinion Quarterly, 71(5), 750–771.

    Google Scholar 

  • Kish, L. (1965). Survey sampling. New York, NY: Wiley.

    Google Scholar 

  • Kuusela, V., & Simpanen, M. (2002). Effects of mobile phones on phone survey practices and results. Copenhagen: ICIS Conference.

    Google Scholar 

  • Kuusela, V., Callegaro, M., & Vehovar, V. (2007). The influence of mobile telephones on telephone surveys. In J. M. Lepkowski, C. Tucker, J. M. Brick, E. D. de Leeuw, L. Japec, P. J. Javrakas, M. W. Link, & R. L. Sangster (Eds.), Advances in telephone survey methodology (pp. 87–112). Hoboken, NJ: Wiley.

    Chapter  Google Scholar 

  • Lavrakas, P. J., Shuttles, C. D., Steeh, C., & Fienberg, H. (2007). The state of surveying cell phone numbers in the United States: 2007 and beyond. Public Opinion Quarterly, 71(5), 840–854.

    Article  Google Scholar 

  • Leah, C., Keeter, S., Purcell, K., & Smith, A. (2010). Assesing the cell phone challenge. PEW research centre, from http://pewresearch.org/pubs/1601/assessing-cell-phone-challenge-in-public-opinion-surveys

  • Lepkowski, J. M. & Groves, R. M. (1984). The impact of bias on dual frame survey design. AMSE model. Proceedings of the Section on Survey Research, American Statistical Association: http://www.amstat.org/sections/SRMS/ Proceedings/papers/1984_051.pdf

  • Lepkowski, J. M., & Groves, R. M. (1986). A mean squarred error model for dual frame, mixed mode survey design. Journal of the American Statistical Association, 396(81),  .

    Google Scholar 

  • Lessler, J. T., & Kalsbeek, W. D. (1992). Nonsampling error in surveys. New York, NY: Wiley.

    Google Scholar 

  • Linacre, S. J., & Trewin, D. J. (1993). Total survey design—application to a collection of the construction industry. Journal of Official Statistics, 9(3), 611–621.

    Google Scholar 

  • Link, M. W., Battaglia, M. P., Frankel, M. R., Osborn, L., & Mokdad, A. H. (2007). Reaching the U.S. cell phone generation: Comparison of cell phone survey results with an ongoing landline telephone survey. Public Opinion Quarterly, 71(5), 814–839.

    Article  Google Scholar 

  • Lyberg, L. E., Japec, L., & Biemer, P. P. (1998). Quality improvement in surveys—A process perspective. Paper presented at the Survey Research Methods Section.

    Google Scholar 

  • Lynn, P., & Elliot, D. (2000). The british crime survey: A review of methodology. London, UK: National Centre for Social Research.

    Google Scholar 

  • McCutcheon, A. L. (2009). Mode or Mensch? Respondent sensitivity to mode changes in data collection methods. The 1st Alec Gallup Future of Survey Research Forum: Incorporating Mobile Phones in Social and Policy-oriented Surveys. Brussels.

    Google Scholar 

  • Nicolaas, G., Lynn, P. & Lound, C. (2000). Random digit dialing in the UK: viability of the sampling method revisited. Paper presented at the International Conference on Social Science Methodology, Cologne.

    Google Scholar 

  • Pew Research Center (2008, December 18). Calling cell phones in ‘08 pre-election polls. News Release. Retrieved February 20, 2009, from http://people-press.org/reports/pdf/cell-phone-commentary.pdf

  • Singer, E. (2006). Nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 637–645.

    Article  Google Scholar 

  • Statistical Office of the Republic of Slovenia. (2008). Community survey on ICT usage in households and by individuals—2008 [Survey data for Slovenia]. Ljubljana, Slovenia: Statistical Office of the Republic of Slovenia.

    Google Scholar 

  • Steeh, C., & Piekarski, L. (2008). “Accommodating new technologies: Mobile and VoIP communication.” In J. M. Lepkowski, C. Tucker, J. M. Brick, E. de Leeuw, L. Japec, P. J. Lavrakas, M. W. Link, & R. L. Sangster (Eds.), Advances in telephone survey methodology (pp. 423–448). New York, NY: Wiley.

    Google Scholar 

  • Steeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field Methods, 19(1), 59–75.

    Article  Google Scholar 

  • Tortora, R. D. (2004). Response trends in a National Random Digit Dial telephone survey. Advances in Methodology and Statistics, 1(1), 61–84.

    Google Scholar 

  • Traugot, M. V., Groves, R. M., & Lepkowski, J. M. (1987). Dual frames to reduce nonresponse. Public Opinion Quarterly, 51(4), 522–539.

    Article  Google Scholar 

  • Tucker, C., Brick, M. J., & Meekin, B. (2007). Household telephone service and usage patterns in the United States in 2004. Public Opinion Quarterly, 71(1), 3–22.

    Article  Google Scholar 

  • Vehovar, V., Belak, E., Batagelj, Z., & Čikić, S. (2004). Mobile phone surveys: The Slovenian case study. Metodološki zvezki (Advances in Methodology and Statistics), 1(1), 1–19.

    Google Scholar 

  • Vehovar, V., & Lozar Manfreda, K. (2008). Overview: Online surveys. In N. G. Fielding, R. M. Lee, & G. Blank (Eds.), The handbook of online research methods (pp. 177–194). Thousand Oaks, CA: SAGE Publications.

    Google Scholar 

  • Vehovar, V., & Slavec, A. (2011). Costs Error Optimization for Cell-Landline Dual Frame Surveys. Paper presented on the 66th conference of the American Association for Public Opinion Research.

    Google Scholar 

  • Vehovar, V., Lozar Manfreda, K., & Batagelj, Z. (2001). Sensitivity of electronic commerce measurement to the survey instrument. International Journal of Electronic Commerce, 6(1), 31–52.

    Google Scholar 

  • Vehovar, V., Berzelak, J., & Lozar Manfreda, K. (2010). Mobile phones in an environment of competing survey modes: Applying metric for evaluation of costs and errors. Social Sciences Computer Review, 28(3), 303–318.

    Article  Google Scholar 

  • Virtanen, V., Sirkiä, T., & Jokiranta, V. (2007). Reducing nonresponse by SMS reminders in mail surveys. Social Science Computer Review, 25(3), 384–395.

    Article  Google Scholar 

  • Waksberg, J. (1978). Sampling method for random digit dialing. Journal of American Statistical Association, 73(361), 40–46.

    Article  Google Scholar 

  • Weisberg, H. F. (2005). The total survey error approach: A guide to the new science of survey research. Chicago: University of Chicago Press.

    Google Scholar 

Suggested Readings

  • AAPOR Cell Phone Task Force (2010). New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers. American Association for Public Opinion Research.

    Google Scholar 

  • Groves, R. M., Biemer, P. P., Lyberg, L. E., Massey, J. T., Nicholls, W. L, I. I., & Waksberg, J. (2001). Telephone survey methodology. Hoboken, NY: Wiley.

    Google Scholar 

  • Lepkowski, J. M., Tucker, C., Brick, J. M., de Leeuw, E. D., Japec, L., Lavrakas, P. J., et al. (2008). Advances in telephone survey methodology. Hoboken, NY: Wiley.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Ljubljana Faculty of Social Sciences, Kardeljeva ploscad 5, 1000, Ljubljana, Slovenija

    Vasja Vehovar, Ana Slavec & Nejc Berzelak

Authors
  1. Vasja Vehovar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Slavec
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nejc Berzelak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vasja Vehovar .

Editor information

Editors and Affiliations

  1. John Jay College of Criminal Justice, City University of New York, Tenth Ave. 899, New York, 10019, New York, USA

    Lior Gideon

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media New York

About this chapter

Cite this chapter

Vehovar, V., Slavec, A., Berzelak, N. (2012). Costs and Errors in Fixed and Mobile Phone Surveys. In: Gideon, L. (eds) Handbook of Survey Methodology for the Social Sciences. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3876-2_16

Download citation

Publish with us

Policies and ethics

Search

Navigation