Skip to main content

Advertisement

SpringerLink
Log in

Cognitive Interviewing Methods for Questionnaire Pre-Testing in Homeless Persons with Mental Disorders

  • Published:
Journal of Urban Health Aims and scope Submit manuscript

Abstract

In this study, cognitive interviewing methods were used to test targeted questionnaire items from a battery of quantitative instruments selected for a large multisite trial of supported housing interventions for homeless individuals with mental disorders. Most of the instruments had no published psychometrics in this population. Participants were 30 homeless adults with mental disorders (including substance use disorders) recruited from service agencies in Vancouver, Winnipeg, and Toronto, Canada. Six interviewers, trained in cognitive interviewing methods and using standard interview schedules, conducted the interviews. Questions and, in some cases, instructions, for testing were selected from existing instruments according to a priori criteria. Items on physical and mental health status, housing quality and living situation, substance use, health and justice system service use, and community integration were tested. The focus of testing was on relevance, comprehension, and recall, and on sensitivity/acceptability for this population. Findings were collated across items by site and conclusions validated by interviewers. There was both variation and similarity of responses for identified topics of interest. With respect to relevance, many items on the questionnaires were not applicable to homeless people. Comprehension varied considerably; thus, both checks on understanding and methods to assist comprehension and recall are recommended, particularly for participants with acute symptoms of mental illness and those with cognitive impairment. The acceptability of items ranged widely across the sample, but findings were consistent with previous literature, which indicates that “how you ask” is as important as “what you ask.” Cognitive interviewing methods worked well and elicited information crucial to effective measurement in this unique population. Pretesting study instruments, including standard instruments, for use in special populations such as homeless individuals with mental disorders is important for training interviewers and improving measurement, as well as interpreting findings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In the main study, homelessness is defined as being “absolute” (having no fixed place to stay for at least the previous seven nights and little likelihood of getting a place in the upcoming month) or “precariously housed.” No fixed place to stay includes living rough in a public or private place not ordinarily used as a regular sleeping accommodation for a human being (e.g., outside on the streets, in parks or on the beach, in doorways, in parked vehicles, squats, or parking garages), as well as those whose primary night-time residence is a supervised public or private emergency accommodation (e.g., a shelter or hostel). Those currently being discharged from an institution, prison, jail, or hospital with no accommodation also qualify as absolutely homeless. Precarious housing is defined as having a room in a single room occupancy facility, a rooming house, or hotel/motel as a primary residence, and two or more episodes of being absolutely homeless in the past year. The criteria for presence of a mental disorder includes two of five observed behaviors, one of five functional impairment items, written documentation of a diagnosed disorder or psychiatric inpatient admission, and/or an indication on the Mini International Neuropsychiatric Interview28 of the presence of current major depression, bipolar disorder, posttraumatic stress disorder, panic disorder, or psychotic disorder (more details available from the authors).

References

  1. Hwang SW. Homelessness and health. Can Med Assoc J. 2001; 164: 229–233.

    CAS  Google Scholar 

  2. Canadian Institute for Health Information. Improving the Health of Canadians: Mental Health and Homelessness. Ottawa, Canada: CIHI; 2007.

    Google Scholar 

  3. Goering P, Tolomiczenko G, Sheldon T, Boydell K, Wasylenki D. Characteristics of persons who are homeless for the first time. Psychiatr Serv. 2002; 53(11): 1472–1474.

    Article  PubMed  Google Scholar 

  4. Aubry T, Klodawsky F, Hay E. Panel Study on Persons Who Are Homeless in Ottawa: Phase 1 Results. Centre for Research on Community Services. Ottawa, Canada: University of Ottawa; 2003.

    Google Scholar 

  5. Patterson M, Somers JM, McIntosh K, Shiell A, Frankish CJ. Housing and Support for Adults with Severe and/or Mental Illness in British Columbia. British Columbia, Canada: Centre for Applied Research in Mental Health and Addiction, Simon Fraser University; 2008.

    Google Scholar 

  6. Pinkney S, Ewing S. The Costs and Pathways of Homelessness: Developing Policy-Relevant Economic Analyses for the Australian Homelessness Service System. Melbourne, Australia: Institute for Social Research, Swinburne University of Technology; 2006.

    Google Scholar 

  7. Hopper K, Baumohl J. Redefining the Cursed Word: a Historical Interpretation of American Homelessness. Westport, CT: Oryx; 1996.

    Google Scholar 

  8. Boydell K, Goering P, Morrell-Bellai T. Narratives of identity: re-presentation of self in people who are homeless. Qual Health Res. 2000; 10(1): 26–38.

    Article  PubMed  CAS  Google Scholar 

  9. Kessell ER, Bhatia R, Bamberger JD, Kushel MB. Public health care utilization in a cohort of homeless adult applicants to a supportive housing program. J Urban Health. 2006; 83(5): 860–873.

    Article  PubMed  Google Scholar 

  10. Hwang SW, Ueng JJM, Chui S, et al. Universal health insurance and health care access for homeless persons. Am J Public Health. 2010; 100: 1454–1461.

    Article  PubMed  Google Scholar 

  11. Wright JD. Address Unknown: the Homeless in America. New York, NY: Transaction Publishers; 2009.

    Google Scholar 

  12. Tsemberis S. From streets to homes: an innovative approach to supported housing for homeless adults with psychiatric disabilities. J Community Psychol. 1999; 27: 225–241.

    Article  Google Scholar 

  13. Tsemberis S, Eisenberg RF. Pathways to housing: supported housing for street-dwelling homeless individuals. Psychiatr Serv. 2000; 51: 487–493.

    Article  PubMed  CAS  Google Scholar 

  14. Pearson CL, Locke G, Montgomery AE, Buron L. The Applicability of Housing First Models to Homeless Persons with Serious Mental Illness. Washington, DC: US Department of Housing and Urban Development Office of Policy Development and Research; 2007.

    Google Scholar 

  15. Tsemberis S, Gulcur L, Nakae M. Housing first, consumer choice, and harm reduction for homeless individuals with a dual diagnosis. A J Public Health. 2004; 94(4): 651–656.

    Article  Google Scholar 

  16. Sadowski LS, Kee RA, VanderWeele TJ. Effect of a housing and case management program on emergency department visits and hospitalizations among critically ill homeless adults: a randomized trial. JAMA. 2009; 301(17): 1771–1778.

    Article  PubMed  CAS  Google Scholar 

  17. Larimer ME, Malone DK, Gardner MD. Health care and public service use and costs before and after provision of housing for chronically homeless persons with severe alcohol problems. JAMA. 2009; 301(13): 1349–1357.

    Article  PubMed  CAS  Google Scholar 

  18. Kertesz SG, Crouch K, Milby JB, Cusimano RE, Schumacher JE. Housing first for homeless persons with active addiction: are we overreaching? Milbank Q. 2009; 87(2): 495–534.

    Article  PubMed  Google Scholar 

  19. Yanos PT, Barrow SM, Tsemberis S. Community integration in the early phase of housing among homeless persons diagnosed with severe mental illness: successes and challenges. Community Ment Health J. 2004; 40(2): 133–150.

    Article  PubMed  Google Scholar 

  20. Gelberg L, Siecke N. Accuracy of homeless adults’ self-reports. Med Care. 1997; 35(3): 287–290.

    Article  PubMed  CAS  Google Scholar 

  21. Goldfinger SM, Schutt RK, Seidman LJ, et al. Self-report and observer measures of substance abuse among homeless mentally ill persons in the cross-section and over time. J Nerv Ment Dis. 1996; 184(11): 667–672.

    Article  PubMed  CAS  Google Scholar 

  22. Rosen MI, McMahon TJ, Rosenheck RA. Homeless people whose self-reported SSI/DI status is inconsistent with social security administration records. Soc Secur Bull. 2007; 67(1): 53–62.

    PubMed  Google Scholar 

  23. Mallinson S. Listening to respondents: a qualitative assessment of the Short-Form 36 Health Status Questionnaire. Soc Sci Med. 2002; 54: 11–21.

    Article  PubMed  Google Scholar 

  24. Collins D. Pre-testing survey instruments: an overview of cognitive methods. Qual Life Res. 2003; 12: 229–238.

    Article  PubMed  Google Scholar 

  25. Willis GB. Cognitive Interviewing: a Tool for Improving Questionnaire Design. New York, NY: SAGE; 2005.

    Google Scholar 

  26. Tourangeau R, Rasinski K. Psychol Bull. 1988; 103: 299–314.

    Article  Google Scholar 

  27. Matter R, Kline S, Cook KF, Amtmann D. Measuring pain in the context of homelessness. Qual Life Res. 2009; 18: 863–872.

    Article  PubMed  Google Scholar 

  28. Sheehan DV, Lecrubier Y, Harnett-Sheehan K, et al. The Mini International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview. J Clin Psychiatry. 1998; 59(suppl 20): S22–S33.

    Google Scholar 

  29. Shern DL, Wilson NZ, Saranga Coen A, et al. Client outcomes II: longitudinal client data from the Colorado treatment outcome study. Milbank Q. 1994; 72(1): 123–148.

    Article  PubMed  CAS  Google Scholar 

  30. Dennis ML, Chan Y, Funk RR. Development and validation of the GAIN short screener for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. Am J Addict. 2006; 15(suppl 1): S80–S91.

    Article  Google Scholar 

  31. Latimer EA, Lecomte T, Becker DR, et al. Generalisability of the individual placement and support model of supported employment: results of a Canadian randomised controlled trial. Br J Psychiatry. 2006; 189: 65–73.

    Article  PubMed  Google Scholar 

  32. Canadian Community Health Survey 4.1. http://www.statcan.gc.ca/cgi-bin/imdb/p2SV.pl?Function=getSurvey&SurvId=3226&SurvVer=1&InstaId=15282&InstaVer=4&SDDS=3226&lang=en&db=imdb&adm=8&dis=2. Accessed August 20, 2010.

  33. National Population Health Survey. http://www.statcan.gc.ca/concepts/nphs-ensp/index-eng.htm. Accessed August 20, 2010.

  34. Mares AS, Rosenheck RA. HUD/HHS/VA Collaborative Initiative to Help end Chronic Homelessness National Performance Outcomes Assessment: Preliminary Client Outcomes Report. West Haven, CT: Northeast Program Evaluation Center; 2007.

    Google Scholar 

  35. Segal SP, Aviram U. The Mentally Ill in Community Based Sheltered Care: a Study of Community Care and Social Integration. New York, NY: Wiley; 1978.

    Google Scholar 

  36. Aubry T, Myner J. Community integration and quality of life: a comparison of persons with psychiatric disabilities in housing programs and community residents who are neighbors. Can J Commun Ment Health. 1996; 15: 5–20.

    PubMed  CAS  Google Scholar 

  37. Chavis DM, Hogge JH, McMillan DW, Wandersman A. Sense of community through Brunswick’s lens: a first look. J Community Psychol. 1986; 14: 24–40.

    Article  Google Scholar 

  38. Boothroyd RA, Chen HJ. The psychometric properties of the Colorado Symptom Index. Adm Policy Ment Health. 2008; 35(5): 370–378.

    Article  PubMed  Google Scholar 

  39. Conrad KJ, Yagelka JR, Matters MD, et al. Reliability and validity of a modified Colorado Symptom Index in a national homeless sample. Ment Health Serv Res. 2001; 3(3): 141–153.

    Article  PubMed  CAS  Google Scholar 

  40. Corrigan PW, Salzer M, Ralph RO, Sangster Y, Keck L. Examining the factor structure of the recovery assessment scale. Schizophr Bull. 2004; 30(4): 1035–1041.

    Article  PubMed  Google Scholar 

  41. Lehman AF. Measures of quality of life among persons with severe and persistent mental disorders. Soc Psychiatry Psychiatr Epidemiol. 1996; 31: 78–88.

    Article  PubMed  CAS  Google Scholar 

  42. Uttaro T, Lehman A. Graded response modeling of the quality of life interview. Eval Program Plann Spring. 1999; 22: 41–52.

    Article  Google Scholar 

  43. Kavanaugh K, Moro TT, Savage T, Mehendale R. Enacting a theory of caring to recruit and retain vulnerable participants for sensitive research. Res Nurs Health. 2006; 29: 244–252.

    Article  PubMed  Google Scholar 

  44. Edwards M, Thomsen SC, Toroitich-Ruto C. Thinking aloud to create better condom use questions. Field Methods. 2005; 17: 183–199.

    Article  Google Scholar 

  45. Bradburn NM, Sudman S, Wansink B. Asking Questions: the Definitive Guide to Questionnaire Design for Market Research, Political Polls, and Social and Health Questionnaires. San Francisco, CA: Jossey-Bass; 2004.

    Google Scholar 

Download references

Acknowledgments

Appreciation is extended to Kimberley Lewis-Ng and Rebecca Godderis who assisted with the planning stages of this study. Susan Mulligan, Verena Strehlau, and Melinda Markey are also thanked for conducting some of the interviews. ACH was supported for this work through the University of Toronto “Comprehensive Research Experience for Medical Students.” The results of the study were presented at the University of Toronto Medical Student Research Day in February 2010. This, and the main study, was made possible through a financial contribution from Health Canada. The Mental Health Commission of Canada is the oversight organization for the study. The views expressed herein solely represent those of the authors and not of the afore-named organizations.

Author information

Authors and Affiliations

  1. Department of Community Health Sciences and Psychiatry, University of Calgary, Calgary, AB, Canada

    Carol E. Adair

  2. Centre for Research on Inner City Health, The Keenan Research Centre in the Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Toronto, ON, Canada

    Anna C. Holland, Kate S. Mason & Stephen W. Hwang

  3. Department of Medicine, Division of General Internal Medicine, University of Toronto, Toronto, ON, Canada

    Anna C. Holland & Stephen W. Hwang

  4. Faculty of Health Sciences, Simon Fraser University, Vancouver, BC, Canada

    Michelle L. Patterson

  5. Centre for Addiction and Mental Health, Toronto, ON, Canada

    Paula N. Goering

  6. Department of Psychiatry, University of Toronto, Toronto, ON, Canada

    Paula N. Goering

Authors
  1. Carol E. Adair
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna C. Holland
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michelle L. Patterson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kate S. Mason
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paula N. Goering
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stephen W. Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

At Home/Chez Soi Project Team

Corresponding author

Correspondence to Carol E. Adair.

Additional information

The At Home/Chez Soi Project Team includes Jayne Barker, Ph.D., VP Research Initiatives, Mental Health Commission of Canada; Cameron Keller, M.C., Director At Home/Chez Soi; and approximately 40 investigators from across Canada and the USA. In addition, there are five site coordinators (one for each city where the study is carried out) and numerous lead service and housing providers. CEA is lead for quantitative measurement and data collection at the national level; MLP and KSM are site research coordinators; SWH is lead investigator for the Toronto site; ACH was an undergraduate medical research student for the summer of 2009; and PNG is the national research lead.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Adair, C.E., Holland, A.C., Patterson, M.L. et al. Cognitive Interviewing Methods for Questionnaire Pre-Testing in Homeless Persons with Mental Disorders. J Urban Health 89, 36–52 (2012). https://doi.org/10.1007/s11524-011-9632-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11524-011-9632-z

Keywords

Search

Navigation