Skip to main content
SpringerLink
Log in

Applying cognitive design principles to formatting HRQOL instruments

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Researchers developing or using health-related quality of life (HRQOL) instruments can benefit from knowledge of state-of-the-art formatting methods for self-administered questionnaires. Three objectives in formatting design are: (1) to reduce errors in respondent navigation through the questionnaire that lead to item non-response and question misinterpretation; (2) to reduce respondent and administrative burden; and (3) to enhance respondent motivation in question answering and compliance with the request to participate. Based on an extensive literature review to identify techniques that have been shown to meet these objectives, we developed specific guidelines for HRQOL instruments concerning all aspects of questionnaire formatting. These guidelines represent well-motivated recommendations for improving HRQOL instruments, although their overall impact has not been empirically tested. We applied the guidelines to several HRQOL instruments that are widely used internationally, and obtained approval from the developers for all formatting changes to their instruments. Applying cognitive design principles and empirically substantiated formatting techniques produces an HRQOL instrument formatting with six critical attributes: simple, consistent, organized, natural, clear and attractive. The present paper contributes to the emerging research literature on the cognitive processes by which respondents answer HRQOL questions and demonstrates how `cognitive aspects of survey methodology' research can improve HRQOL data collection efforts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Jenkins CR, Dillman DA. Towards a theory of self-administered questionnaire design. In: Lyberg L, Biemer P, Collins M, et al. (eds), Survey Measurement and Process Quality. New York: John Wiley & Sons, 1997.

    Google Scholar 

  2. Wright P. Strategy and tactics in the design of forms. Vis Lang 1980; 14: 151–193.

    Google Scholar 

  3. Ware JE, Snow KK, Kosinski M, Gandek B. SF–36Health Survey Manual and Interpretation Guide. Boston, MA: New England Medical Center, The Health Institute, 1993.

    Google Scholar 

  4. Hays RD, Kallich J, Mapes DL, et al. Kidney Disease Quality of Life Short Form (KDQOL–SF), Version 1.3: A Manual for Use and Scoring. Santa Monica, CA: RAND, 1997.

    Google Scholar 

  5. Cella D. The Functional Assessment of Cancer Therapy-Anemia (FACT–An) scale: A new tool for the assessment of outcomes in cancer anemia and fatigue. Seminars in Hematol 1997; 34: 13–19.

    Google Scholar 

  6. McGrath C, Marquis P, Turner R. Development of a quality of life instrument for use in clinical trials of diabetic foot ulcer treatment. Drug Inform J 1996; 30: 856–857.

    Google Scholar 

  7. Wright P, Barnard P. Asking multiple questions about several items: the design of matrix structures on application forms. Appl Ergon 1978; 9: 7–14.

    Google Scholar 

  8. Krosnick JA. Response strategies for copying with the cognitive demands of attitude measures in surveys. Appl Cognitive Psychol 1991; 5: 213–236.

    Google Scholar 

  9. Lohr KN, Aaronson NK, Alonso J, et al. Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clin Therap 1996; 18: 979–992.

    Google Scholar 

  10. Dillman DA, Sinclair MD, Clark JR. Effects of questionnaire length, respondent-friendly design, and a dificult question on response rates for occupant-addressed Census mail surveys. Public Opin Quart 1993; 57: 289–304.

    Google Scholar 

  11. Broadbent DE. Perception and Communication. London: Pergamon, 1958.

    Google Scholar 

  12. Kahneman D. Attention and Effort. Englewood Cliffs, NJ: Prentice Hall, 1973.

    Google Scholar 

  13. Treisman A, Gelade G. A feature integration theory of attention. Cognitive Psychol 1980; 12: 97–136.

    Google Scholar 

  14. Eriksen B, Eriksen CW. Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophys 1974; 12: 201–204.

    Google Scholar 

  15. Baddeley AD, Hitch GJ. Working memory. In: Bower GA (ed), Recent Advances in Learning and Motivation (Vol. 8). New York: Academic Press, 1974; 47–90.

    Google Scholar 

  16. Kinsbourne M, Hicks R. Functional cerebral space. In: Reguin J (ed), Attention and Performance VII. Hillsdale, NJ: Erlbaum, 1978.

    Google Scholar 

  17. Barnard P, Wright P, Wilcox P. Effects of response instructions and question style on the ease of completing forms. J Occup Psychol 1979; 52: 209–226.

    Google Scholar 

  18. Glynn SM, Muth KD. Text learning capabilities of older adults. Educational Gerontol 1979; 4: 253–269.

    Google Scholar 

  19. Taub HA, Sturr JF, Monty RA. The effect of underlining cues upon memory of older adults. Exper Aging Res 1985; 11: 225–226.

    Google Scholar 

  20. Fowler FJ. Survey Research Methods. Newbury Park, CA: Sage Publications, 1993.

    Google Scholar 

  21. Gower AR, Dibbs R. Cognitive research: Designing a ‘respondent friendly’ questionnaire for the 1991 Census. Proceedings of the Bureau of the Census Fifth Annual Research Conference 1989: 257–266.

  22. Ware JE, Kosinski M. SF–36 Health Survey (Version 2.0) Technical Note. Boston, MA: The Health Institute, 1996.

    Google Scholar 

  23. Sudman S, Bradburn NM. Asking Questions: A Practical Guide to Questionnaire Design. San Francisco, CA: Jossey-Bass Publishers, 1982.

    Google Scholar 

  24. Berdie DR, Anderson JF, Niebuhr MA. Questionnaires: Design and Use. Netuchen, NJ: The Scarecrow Press, 1986.

    Google Scholar 

  25. Chandler P, Sweller J. Cognitive load theory and the format of instruction. Cognition and Instruct 1991; 8: 293–332.

    Google Scholar 

  26. Sweller J. Cognitive load theory, learning difficulty and instructional design. Learning and Instruct 1994; 4: 295–312.

    Google Scholar 

  27. Shiffrin RM, Schneider W. Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychol Rev 1977; 84: 127–190.

    Google Scholar 

  28. Schuman H, Presser S. Questions and answers in attitude surveys. New York: Academic Press, 1981.

    Google Scholar 

  29. Mirowsky J, Ross CE. Eliminating defense and agreement bias from measures of the sense of control: A 2 × 2 index. Social Psychol Quart 1991; 54: 127–145.

    Google Scholar 

  30. Bradford JD, Johnson MK. Contextual prerequisites for understanding: some investigations of comprehension and recall. J Verbal Learning and Verbal Behav 1972; 11: 717–726.

    Google Scholar 

  31. Wickens C, Carswell CM. The proximity compatibility principle: its psychological foundation and relevance to display design. Human Factors 1995; 37: 473–494.

    Google Scholar 

  32. Schriesheim CA, Denisi AS. Item presentations as an influence on questionnaire validity: A field experiment. Educational Psychol Measur 1980; 40: 175–182.

    Google Scholar 

  33. Mullin PA, Cashman ER, Straub HR. Answering question sequences: attention switching and memory organization. In: Herrmann D, Johnson M, McEvoy C, Hertzog C, Hertel P (eds), Basic and Applied Memory Research: Practical Aspects (Vol. 2). Hillsdale, NJ: Erlbaum, 1996: 179–192.

    Google Scholar 

  34. Hastie R, Park B. The relationship between memory and judgment depends on whether the judgment is memory-based or on-line. Psychol Rev 1986; 93: 258–68.

    Google Scholar 

  35. Dillman DA. Mail and Telephone Surveys: The Total Design Method. New York: John Wiley & Sons, 1978.

    Google Scholar 

  36. Wright P, Threlfall SM. Readers’ expectation about format influence the usability of an index. J Res Commun Studies 1980; 2: 99–106.

    Google Scholar 

  37. Holbrook AL, Krosnick JA, Carson RT, Mitchell RC. Violating conversational conventions disrupts cognitive processing of attitude questions. J Exper Social Psychol (in press).

  38. Hunt S, McKenna SP, McEwen J, et al. The Nottingham Health Profile: subjective health status and medical consultations. Soc Sci Med 1981; 15A: 221–229.

    Google Scholar 

  39. Gregory M, Poulton EC. Even versus uneven right-handed margins and the rate of comprehension in reading. Ergonomics 1970; 13: 427–434.

    Google Scholar 

  40. Posner M. Chronometric Explorations of Mind. Hillsdale, NJ: Erlbaum, 1978.

    Google Scholar 

  41. Poulton EC. Skimming lists of food ingredients printed in different brightness contrasts. J Appl Psychol 1969; 53: 498–500.

    Google Scholar 

  42. Rock I. The Logic of Perception. Cambridge, MA: MIT Press, 1983.

    Google Scholar 

  43. Hartley J, Burnhill P, Fraser S. Typographic problems of journal design. Appl Ergon 1974; 5: 15–20.

    Google Scholar 

  44. Whalley PC, Fleming RW. An experiment with a simple recorder of reading behavior. Programmed Learning Educational Technol 1975; 12: 120–123.

    Google Scholar 

  45. Albers J. Interaction of Color. New Haven, CT, 1975.

  46. Tinker MA. Bases for E€ective Reading. Minneapolis, MN: University of Minnesota Press, 1965.

    Google Scholar 

  47. Spenser H. The Visible Word. London: Lund Humphries, 1969.

    Google Scholar 

  48. Poulton EC. Searching for newspaper headlines printed in capitals or lower-case letters. J Appl Psychol 1967; 51: 417–425.

    Google Scholar 

  49. Poulton EC. Rate of comprehension of existing teleprinter output of possible alternatives. J Appl Psychol 1968; 52: 16–21.

    Google Scholar 

  50. Taub HA, Baker MT, Kline GE, Sturr JF. Comprehension of informed consent information by young-old and old-old volunteers. Exper Aging Res 1987; 13: 173–178.

    Google Scholar 

  51. Vanderplas JH, Vanderplas JM. Effects of legibility on verbal test performance of older adults. Perceptual and Motor Skills 1981; 53: 183–186.

    Google Scholar 

  52. Charness N, Bosman EA. Age-related changes in perceptual and psychomotor performance: Implications for engineering design. Exper Aging Res 1994; 20: 45–59.

    Google Scholar 

  53. Aberson DH, Bouwhuis L. Silent reading as determined by age and visual acuity. J Res Reading 1997; 20: 184–204.

    Google Scholar 

  54. Tim Smith. Research Triangle Institute, personal communication, 1997.

  55. Edwards B. Drawing on the Right Side of the Brain. Los Angeles, CA: Jeremy P. Tarcher, 1989.

    Google Scholar 

  56. Couper MP. Survey introductions and data quality. Public Opin Quart 1997; 61: 317–338.

    Google Scholar 

  57. Nederhof AJ. Effects of a final telephone reminder and questionnaire cover design in mail surveys. Social Sci Res 1988; 19: 353–361.

    Google Scholar 

  58. McBean AM, Turner CF, Fitterman LK, et al. Monitoring the health status and impact of treatment on older Americans: The Medicare Beneficiary Health Status Registry. Med Care 1999; 37: 189–203.

    Google Scholar 

  59. Velikova G, Wright EP, Smith AB, et al. Automated collection of quality-of-life data: A comparison of paper and computer touch–screen questionnaires. J Clin Oncol 1999; 17: 998–1007.

    Google Scholar 

  60. Willis GB, Royston P, Bercini D. The use of verbal report methods in the development and testing of survey questions. Appl Cognitive Psychol 1991; 5: 251–267.

    Google Scholar 

  61. Redline C, Smiley R, Lee M, DeMaio T, Dillman D. Beyond concurrent interviews: An evaluation of cognitive interviewing techniques for self-administered questionnaires. Proceedings of the American Statistical Association: Section on Survey Research Methods. Alexandria, VA: American Statistical Association, 1998, (in press).

    Google Scholar 

  62. Schecter S, Beatty P, Willis G. Asking survey respondents about health status: Judgement and response issues. In: Schwarz N, Park D, Knauper B, Sudman S (eds), Cognition, Aging, and Self-Reports. Philadelphia, PA: Psychology Press 1998: 265–284.

    Google Scholar 

  63. Jobe JB, Mingay DJ. Cognition and survey measurement: history and overview. Appl Cognitive Psychol 1991; 5: 175–192.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Triangle Institute, P.O. Box 12194, Research Triangle Park, NC, 27709-2194

    Paul A. Mullin & Kathleen N. Lohr

  2. ICOM Health Economics, Johnson & Johnson, USA

    Brian W. Bresnahan & Pauline McNulty

Authors
  1. Paul A. Mullin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kathleen N. Lohr
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brian W. Bresnahan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pauline McNulty
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mullin, P.A., Lohr, K.N., Bresnahan, B.W. et al. Applying cognitive design principles to formatting HRQOL instruments. Qual Life Res 9, 13–27 (2000). https://doi.org/10.1023/A:1008923301313

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008923301313

Search

Navigation