Abstract
Previous research on the survey measurement of sexual orientation and gender identity (SOGI) often focuses on the measurement of gender identity and sexual identity. Comparably little research exists that is focused on gender expression and sexual attraction, which are also each key features gender and sexuality.
The current study seeks to contribute to this line of research, examining what underlies participants’ answers to survey questions on gender expression and sexual attraction. We conducted cognitive interviews with 16 self-identified queer women, expecting participants to show variability and nuance in considering dimensions of gender expression and sexual attraction. We asked best practice versions of survey questions on gender expression and sexual attraction, then followed up with probes to ask what respondents think about when answering each question. We demonstrate how grounded theory methods can be applied to cognitive interview data by developing codes of emergent themes from participants’ responses in an inductive, iterative, and systematic process.
The results of this study indicate various considerations and refinements for measuring gender expression and sexual attraction in surveys, with implications for practitioners and researchers who are interested in measuring gender and sexual orientation holistically in survey research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Others have argued (The GenIUSS Group 2014; Lowry et al. 2018) that this concept may be particularly useful for adolescents and young adults who have not formed an explicit gender identity—although we might argue that young adults are the agents of emergent identity formation and understandings thereof.
- 2.
- 3.
Although some respondents indicated multiple sexual identity categories during the in-depth interview, we categorized them with one group for the purposes of analysis based on information they discussed during the in-depth interview and the cognitive interview. Participant 4 indicated an overarching alignment with the queer community and consistently noted academic considerations of queer politics, thus we placed them with “queer” in categorizing sexual identity. Participants 9, 10, and 16 all noted both queer and bisexual sexual identity. We characterize each as “bisexual” in terms of their sexual identity, as they each seem to use bisexual to specifically refer to their own identity and queer when speaking about community membership. Participant 4 indicated both cisgender and genderqueer identities.
- 4.
Participants ranged in age from 19 to 35 but were mainly in their twenties, with a median age of 24 and mean age of 24.75.
- 5.
We do not follow another approach common within grounded theory methodology and the constant comparative method: theoretical sampling, in which participants are purposively selected as indicated by the emerging conceptual codes in the data (Glaser and Strauss 1967).With theoretical sampling, data collection and analysis are confounded in that some analysis has occurred that indicates who should be interviewed next (as well as potential revisions to the interview script) (Willis 2015); this is not a problem with the method but something to be accounted for in analysis. Since our respondents were interviewed prior to data analysis, we did not follow the theoretical sampling that can be part of the iterative process of sampling and analysis in the grounded theory methodology. In this way, we are similar to other analyses of cognitive interviews that use the constant comparative method for coding but not sampling [see Willis (2015) for examples].
- 6.
Gender presentation varies for and is interpreted through the lens of particular audiences and contexts in previous research on sexual minorities. In some cases, certain presentations take on a particular gendered meaning based on the audience and context (Luzzatto and Gvion 2004; Moore 2006); in others, presentation is used to communicate or disclose one’s sexual identity (Bulgar-Medina 2018; Hebl et al. 2002). Indeed, distinctive looks to communicate queer identities have increasingly become both more diverse and mainstream.
- 7.
Sometimes probes were skipped if the interviewer thought the participant had answered the probe in another part of their answer.
- 8.
Wylie et al. (2010) propose using the phrase “on average” in gender expression questions to aid in respondents’ cognitive averaging and integration of their perceived gender expression across these different domains. We used the term “overall” so as not to predispose respondents to a particular estimation strategy; another option could be “in general” to be even less specific. Other sorts of response strategies include signaling a modal response, such as “most of the time,” or typicality, such as “on a typical day,” and each of these might lead to different considerations among respondents for whom their appearance varies depending on context, time, or audience.
- 9.
Although “mostly” and “very” are considered synonyms that indicate a higher values of what is being measured, participants also could have had trouble with the scale because the verbal quantifiers mix dimensions: “mostly” is a quantifier denoting the amount of a whole, whereas “very” is a qualifier denoting value.
- 10.
The English language has a decided lack of adverbial quantifiers between “somewhat” and “very.” “Pretty” and “quite” are two alternatives, although “pretty” may be complicated given that the question asks about appearance and “quite” sounds formal. Based on theory and research in survey methodology (Kronsick and Presser2010), we do not recommend rating scales with only endpoint labels and numbers for the intermediate response options (c.f. Magliozzi et al. 2016; Wilson et al. 2014).
- 11.
One common practice with the bipolar scale is to present the feminine side of the scale first if the respondent is presumed a woman and the masculine side if the respondent is presumed a man. Another practice is to present the same side of the scale to everyone regardless of gender in a paper-administered survey (e.g., Youth Risk Behavior Survey).
- 12.
Only one participant discussed consciously modifying their mannerisms: Participant 11, a cisgender woman, stated, “um when I was younger I tried to be very masculine and very like sort of aggressive way like masculine in the sense of being uh not just physically aggressive but like like aggressive in the way I talked and uh sort of taking up a lot of space that sort of thing. But at the same time I always sort of walked and moved in a way that a lot of women do and then like actively tried not to and then it’s just a mess I guess.”
- 13.
We coded for the presence of the following features of the interaction that may indicate various forms of processing and difficulties in answering the survey questions. These each may indicate grappling with the task at hand or the mismatch of the task with one’s lived experience: disfluencies tokens (“um,” “uh”) and markers of uncertainty: mitigating phrases (“probably,” “I think”) and “don’t know” as an answer or mitigating phrase.
- 14.
- 15.
The current practice with the bipolar scale is to present the “only attracted to males” side of the scale first if the respondent is presumed a woman and the reverse if the respondent is presumed a man.
References
Beckstead, J. W. (2014). On measurements and their quality. Paper 4: Verbal anchors and the number of response options in rating scales. International Journal of Nursing Studies, 51(5), 807–814.
Bem, S. L. (1993). The lenses of gender: Transforming the debate on sexual inequality. New Haven and London: Yale University Press.
Bem, S. L. (1981) Gender schema theory: A cognitive account of sex typing. Psychological Review, 88(4):354–364.
Bulgar-Medina, J. (2018). Surveying sexual orientation disclosure. Paper presented at the annual meeting of the American Association for Public Opinion Research (AAPOR), Denver, CO.
Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design—for market research, political polls, and social and health questionnaires. San Francisco: Wiley.
Brenner, P. S., & Bulgar-Medina, J. (2018). Testing mark-all-that-apply measures of sexual orientation and gender identity. Field Methods, 30(4), 357–370.
Brewster, K. L., & Tillman, K. H. (2012). Sexual orientation and substance use among adolescents and young adults. American Journal of Public Health, 102(6), 1168–1176.
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: Sage.
Clark, M. A., Armstrong, G., & Bonacore, L. (2005). Measuring sexual orientation and gender expression among middle-aged and older women in a Cancer screening study. Journal of Cancer Education, 20(2), 108–112.
Connell, R. W. (2005). Masculinities. Cambridge: Polity.
Constantinople, A. (1973). Masculinity-femininity: An exception to a famous dictum? Psychological Bulletin, 80(5), 389–407.
Corteen, K. (2002). Lesbian safety talk: Problematizing definitions and experiences of violence, sexuality and space. Sexualities, 5(3), 259–280.
Courtenay, W. H. (2000). Constructions of masculinity and their influence on Men’s Well-being: A theory of gender and health. Social Science & Medicine, 50(10), 1385–1401.
Dobson, K. S., & Mothersill, K. J. (1979). Equidistant categorical labels for construction of Likert-type scales. Perceptual and Motor Skills, 49(2), 575–580.
Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity (SOGI). (2016a). Current measures of sexual orientation and gender identity in federal surveys. Accessed December 22, 2018, from https://nces.ed.gov/FCSM/pdf/current_measures_20160812.pdf.
Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity (SOGI). (2016b). Evaluations of sexual orientation and gender identity survey measures: What have we learned? Accessed December 22, 2018, from https://nces.ed.gov/FCSM/pdf/Evaluations_of_SOGI_Questions_20160923.pdf.
Fowler, F. J. (1995). Improving survey questions. Thousand Oaks, CA: Sage.
Garbarski, D., Schaeffer, N. C., & Dykema, J. (2015). The effects of response option order and question order on self-rated health. Quality of Life Research, 24(6), 1443–1453. https://doi.org/10.1007/s11136-014-0861-y.
Garbarski, D., Schaeffer, N. C., & Dykema, J. (2016). Interviewing practices, conversational practices, and rapport: Responsiveness and engagement in the standardized survey interview. Sociological Methodology, 46, 1–38. https://doi.org/10.1177/0081175016637890.
Garbarski, D., Schaeffer, N. C., & Dykema, J. (2019). The effects of features of survey measurement on self-rated health: Response option order and scale orientation. Applied Research in Quality of Life, 14(2), 545–560. https://ecommons.luc.edu/soc_facpubs/18/.
Glaser, B. G. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Mill Valley, CA: Sociology Press.
Glaser, B. G., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine.
Glick, J. L., Theall, K., Andrinopoulos, K., & Kendall, C. (2018). For Data’s sake: Dilemmas in the measurement of gender minorities. Culture, Health & Sexuality, 20, 1–16. https://doi.org/10.1080/13691058.2018.1437220.
Gordon, L. E., & Silva, T. J. (2015). Inhabiting the sexual landscape: Toward an interpretive theory of the development of sexual orientation and identity. Journal of homosexuality., 62(4), 95–530.
Harrison-Quintana, J., Grant, J. M., & Rivera, I. G. (2015). Boxes of our own creation a trans data collection Wo/manifesto. TSQ: Transgender Studies Quarterly, 2(1), 166–174.
Hebl, M. R., Foster, J. B., Mannix, L. M., & Dovidio, J. F. (2002). Formal and interpersonal discrimination: A field study of Bias toward homosexual applicants. Personality and Social Psychology Bulletin, 28(6), 815–825.
Henley, N. M. (1977). Body politics. Power, sex, and nonverbal communication. Englewood Cliffs, NJ: Prentice Hall.
Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (pp. 263–314). Bingley, UK: Emerald Group Publishing.
Laumann, E. O., Gagnon, J. H., Michael, R. T., & Michaels, S. (1994). The social organization of sexuality: Sexual practices in the United States. Chicago: University of Chicago press.
Lorber, J. (1994). Paradoxes of gender. New Haven and London: Yale University Press.
Lowry, R., Johns, M. M., Gordon, A. R., Austin, S., Robin, L. E., & Kann, L. K. (2018). Nonconforming gender expression and associated mental distress and substance use among high school students. JAMA Pediatrics, 172, 1020–1028. https://doi.org/10.1001/jamapediatrics.2018.2140.
Luzzatto, D., & Gvion, L. (2004). Feminine but not femme: The dual lesbian body. Journal of Homosexuality, 48(1), 43–77.
Magliozzi, D., Saperstein, A., & Westbrook, L. (2016). Scaling up: Representing gender diversity in survey research. Socius, 2, 237802311666435. https://doi.org/10.1177/2378023116664352.
Martin, P. Y. (2003). “Said and done” versus “saying and doing” gendering practices, practicing gender at work. Gender & Society, 17(3), 342–366.
Moore, M. R. (2006). Lipstick or timberlands? Meanings of gender presentation in black lesbian communities. Signs: Journal of Women in Culture and Society, 32(1), 113–139.
National Survey on Drug Use and Health (NSDUH). (2018). Accessed December 19, 2018, from https://nsduhweb.rti.org/respweb/homepage.cfm.
Risman, B. J. (2018). Where the Millennials will take us: A new generation wrestles with the gender structure. Oxford: Oxford University Press.
Rust, P. C. (1992). The politics of sexual identity: Sexual attraction and behavior among lesbian and bisexual women. Social Problems, 39(4), 366–386.
Schaeffer, N. C. (1991). Hardly ever or constantly? Group comparisons using vague quantifiers. Public Opinion Quarterly, 55(3), 395–423.
Silva, T. (2017). Bud-sex: Constructing normative masculinity among rural straight men that have sex with men. Gender and Society, 31(1), 51–73.
SMART. (2009). Best practices for asking questions about sexual orientation on surveys. Created by the sexual minority assessment research team (SMART). L. Badgett and N. Goldberg (Eds.). Los Angeles, CA: The Williams Institute. Accessed December 19, 2018, from http://williamsinstitute.law.ucla.edu/wp-content/uploads/SMART-FINAL-Nov-2009.pdf.
Smyth, J. D., & Olson, K. (2020). Male/Female is not enough: Adding measures of masculinity and femininity to general population surveys. Understanding Survey Methodology: Sociological Theory and Applications, edited by P. Brenner. Springer.
Spence, J. T. (2011). Off with the old, on with the new. Psychology of Women Quarterly, 35(3), 504–509. https://doi.org/10.1177/0361684311414826.
Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.
The GenIUSS Group. (2014). Best practices for asking questions to identify transgender and other gender minority respondents on population-based surveys (Created by the Gender Identity in U.S. Surveillance (GenIUSS) Group). J. L. Herman (Ed.). Los Angeles, CA: The Williams Institute. Accessed December 22, 2018, from https://williamsinstitute.law.ucla.edu/wp-content/uploads/geniuss-report-sep-2014.pdf.
Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
Tourangeau, R., Couper, M. P., & Conrad, F. G. (2013). “Up means good”: The effect of screen position on evaluative ratings in web surveys. Public Opinion Quarterly, 77(S1), 69–88. https://doi.org/10.1093/poq/nfs063.
Van Anders, S. M. (2015). Beyond sexual orientation: Integrating gender/sex and diverse sexualities via sexual configurations theory. Archives of Sexual Behavior, 44(5), 1177–1213.
West, C., & Zimmerman, D. H. (1987). Doing Gender. Gender and Society, 1(2), 25–51.
Westbrook, L., & Saperstein, A. (2015). New categories are not enough: Rethinking the measurement of sex and gender in social surveys. Gender and Society, 29(4), 534–560. https://doi.org/10.1177/0891243215584758.
Willis, G. B. (2015). Analysis of the cognitive interview in questionnaire design. Oxford: Oxford University Press.
Wilson, B. D. M., Cooper, K., Kastanis, A., & Nezhad, S. (2014). Sexual and gender minority youth in Foster Care: Assessing disproportionality and disparities in Los Angeles. Los Angeles: The Williams Institute, UCLA School of Law. Accessed September 17, 2019, from https://escholarship.org/uc/item/6mg3n153.
Wolff, M., Wells, B., Ventura-DiPersia, C., Renson, A., & Grov, C. (2017). Measuring sexual orientation: A review and critique of U.S. data collection efforts and implications for health policy. The Journal of Sex Research, 54(4–5), 507–531. https://doi.org/10.1080/00224499.2016.1255872.
Wylie, S. A., Corliss, H. L., Boulanger, V., Prokop, L. A., & Austin, S. B. (2010). Socially assigned gender nonconformity: A brief measure for use in surveillance and investigation of health disparities. Sex Roles, 63(3–4), 264–276.
Zucker, K. J., Mitchell, J. N., Bradley, S. J., Tkachuk, J., Cantor, J. M., & Allin, S. M. (2006). The recalled childhood gender identity/gender role questionnaire: Psychometric properties. Sex Roles, 54(7), 469–483. https://doi.org/10.1007/s11199-006-9019-x.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Garbarski, D., LaVergne, D. (2020). The Measurement of Sexual Attraction and Gender Expression: Cognitive Interviews with Queer Women. In: Brenner, P.S. (eds) Understanding Survey Methodology. Frontiers in Sociology and Social Research, vol 4. Springer, Cham. https://doi.org/10.1007/978-3-030-47256-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-47256-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-47255-9
Online ISBN: 978-3-030-47256-6
eBook Packages: Social SciencesSocial Sciences (R0)