Skip to main content
SpringerLink
Log in

Mastering the Use of Control Variables: the Hierarchical Iterative Control (HIC) Approach

  • Original Paper
  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

There has been growing criticism of the established practice of automatically including control variables into analyses, especially with survey studies. Several authors have explained the pitfalls of improper use and have provided some best practice advice. I build upon this foundation in suggesting a programmatic approach to the use of control variables that can provide evidence to support or refute feasible explanations for why two or more variables are related. The hierarchical iterative control (HIC) approach begins by establishing a connection between two or more variables and then hierarchically adds control variables to rule in or out their possible influence. The HIC approach involves conducting a series of studies to iteratively test relationships among target variables, utilizing a variety of control variable strategies involving multiple methods. A 7-step programmatic approach is described beginning with development of the research question and background literature review and then conducting empirical tests in a hierarchical (within a study) and iterative (across studies) manner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Thanks to one of my reviewers for suggesting this.

References

  • Atinc, G. M., Simmering, M. J., & Kroll, M. J. (2012). Control variable use and reporting in macro and micro management research. Organizational Research Methods, 15(1), 57–74. https://doi.org/10.1177/1094428110397773.

    Article  Google Scholar 

  • Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8(3), 274–289.

    Article  Google Scholar 

  • Becker, T. E., Atinc, G., Breaugh, J. A., Carlson, K. D., Edwards, J. R., & Spector, P. E. (2016). Statistical control in correlational studies: 10 essential recommendations for organizational researchers. Journal of Organizational Behavior, 37(2), 157–167. https://doi.org/10.1002/job.2053.

    Article  Google Scholar 

  • Bernerth, J. B., & Aguinis, H. (2016). A critical review and best-practice recommendations for control variable usage. Personnel Psychology, 69(1), 229–283. https://doi.org/10.1111/peps.12103.

    Article  Google Scholar 

  • Boughzala, I., de Vreede, T., Nguyen, C., & de Vreede, G.-J. (2014, 6-9 Jan. 2014). Towards a maturity model for the assessment of ideation in crowdsourcing projects. Paper presented at the 2014 47th Hawaii International Conference on System Sciences.

  • Bowling, N. A., Alarcon, G. M., Bragg, C. B., & Hartman, M. J. (2015). A meta-analytic examination of the potential correlates and consequences of workload. Work and Stress, 29(2), 95–113. https://doi.org/10.1080/02678373.2015.1033037.

    Article  Google Scholar 

  • Breaugh, J. A. (2006). Rethinking the control of nuisance variables in theory testing. Journal of Business and Psychology, 20(3), 429–443. https://doi.org/10.1007/s10869-005-9009-y.

    Article  Google Scholar 

  • Breaugh, J. A. (2008). Important considerations in using statistical procedures to control for nuisance variables in non-experimental studies. Human Resource Management Review, 18(4), 282–293. https://doi.org/10.1016/j.hrmr.2008.03.001.

    Article  Google Scholar 

  • Burks, B. (1926). On the inadequacy of the partial and multiple correlation technique. Journal of Educational Psychology, 17(9), 625–630.

    Article  Google Scholar 

  • Carlson, K. D., & Wu, J. (2012). The illusion of statistical control: Control variable practice in management research. Organizational Research Methods, 15(3), 413–435. https://doi.org/10.1177/1094428111428817.

    Article  Google Scholar 

  • Clark, A., Oswald, A., & Warr, P. (1996). Is job satisfaction U-shaped in age? Journal of Occupational and Organizational Psychology, 69, 57–81.

    Article  Google Scholar 

  • Clogg, C. C., Petkova, E., & Shihadeh, E. S. (1992). Statistical methods for analyzing collapsibility in regression models. Journal of Educational Statistics, 17(1), 51–74. https://doi.org/10.3102/10769986017001051.

    Article  Google Scholar 

  • Crits-Christoph, P., Gallop, R., Gaines, A., Rieger, A., & Connolly Gibbons, M. B. (2018). Instrumental variable analyses for causal inference: Application to multilevel analyses of the alliance–outcome relation. Psychotherapy Research, 30, 53–67. https://doi.org/10.1080/10503307.2018.1544724.

    Article  PubMed  PubMed Central  Google Scholar 

  • Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: John Wiley.

    Google Scholar 

  • Judge, T. A. (1993). Does affective disposition moderate the relationship between job satisfaction and voluntary turnover? Journal of Applied Psychology, 78(3), 395–401.

    Article  Google Scholar 

  • Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. The American Journal of Psychiatry, 158(6), 848–856. https://doi.org/10.1176/appi.ajp.158.6.848.

    Article  PubMed  Google Scholar 

  • Lefkowitz, J. (2000). The role of interpersonal affective regard in supervisory performance ratings: A literature review and proposed causal model. Journal of Occupational and Organizational Psychology, 73(1), 67–85. https://doi.org/10.1348/096317900166886.

    Article  Google Scholar 

  • MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1(4), 173–181.

    Article  Google Scholar 

  • Malone, T., & Lusk, J. L. (2018). An instrumental variable approach to distinguishing perceptions from preferences for beer brands. Managerial and Decision Economics, 39(4), 403–417. https://doi.org/10.1002/mde.2913.

    Article  Google Scholar 

  • Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77(2), 143–148.

    Article  Google Scholar 

  • Moorman, R. H., & Podsakoff, P. M. (1992). A meta-analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behaviour research. Journal of Occupational and Organizational Psychology, 65(2), 131–149.

    Article  Google Scholar 

  • O’Connell, B. J. (1991). An examination of work related social support in a longitudinal study controlling for negative affectivity and transient mood. In Unpublished Doctoral Dissertation. Tampa: University of South Florida.

    Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.

    Article  Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452.

    Article  PubMed  Google Scholar 

  • Quinn, R. P., Staines, G. L., & McCullough, M. R. (1974). Job satisfaciton: Is there a trend? Manpower research monograph no. 30 U.S. Department of Labor. Washington: Government Printing Office.

    Google Scholar 

  • Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G.-J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., Schwabe, G., & Söllner, M. (2020). Machines as teammates: A research agenda on AI in team collaboration. Information & Management, 57(2), 103174. https://doi.org/10.1016/j.im.2019.103174.

    Article  Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  • Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632.

    Article  PubMed  Google Scholar 

  • Spector, P. E. (2017). The lost art of discovery: The case for inductive methods in occupational health science and the broader organizational sciences. Occupational Health Science, 1(1), 11–27. https://doi.org/10.1007/s41542-017-0001-5.

    Article  Google Scholar 

  • Spector, P. E. (2019). Do not cross me: Optimizing the use of cross-sectional designs. Journal of Business and Psychology, 34, 125–137. https://doi.org/10.1007/s10869-018-09613-8.

    Article  Google Scholar 

  • Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287–305. https://doi.org/10.1177/1094428110369842.

    Article  Google Scholar 

  • Spector, P. E., & Brannick, M. T. (2016). Methodological urban legends: The misuse of statistical control variables. In G. J. Boyle, J. G. O’Gorman, G. J. Fogarty, G. J. Boyle, J. G. O’Gorman, & G. J. Fogarty (Eds.), Work and organisational psychology: Research methodology; assessment and selection; Organisational change and development; human resource and performance management; emerging trends: Innovation/globalisation/technology (pp. 63–86). Thousand Oaks, CA, US: Sage Publications, Inc..

    Google Scholar 

  • Spector, P. E., Chen, P. Y., & O’Connell, B. J. (2000a). A longitudinal study of relations between job stressors and job strains while controlling for prior negative affectivity and strains. Journal of Applied Psychology, 85(2), 211–218. https://doi.org/10.1037/0021-9010.85.2.211.

    Article  Google Scholar 

  • Spector, P. E., & Meier, L. L. (2014). Methodologies for the study of organizational behavior processes: How to find your keys in the dark. Journal of Organizational Behavior, 35(8), 1109–1119. https://doi.org/10.1002/job.1966.

    Article  Google Scholar 

  • Spector, P. E., & Nixon, A. E. (2019). How often do I agree: An experimental test of item format method variance in stress measures. Occupational Health Science, 3, 125–143.

    Article  Google Scholar 

  • Spector, P. E., Rosen, C. C., Richardson, H. A., Williams, L. J., & Johnson, R. E. (2019). A new perspective on method variance: A measure-centric approach. Journal of Management, 45(3), 855–880. https://doi.org/10.1177/0149206316687295.

    Article  Google Scholar 

  • Spector, P. E., Zapf, D., Chen, P. Y., & Frese, M. (2000b). Why negative affectivity should not be controlled in job stress research: Don’t throw out the baby with the bath water. Journal of Organizational Behavior, 21(1), 79–95. https://doi.org/10.1002/%28SICI%291099-1379%28200002%2921:1%3C79::AID-JOB964%3E3.0.CO;2-G.

    Article  Google Scholar 

  • Watson, D., Pennebaker, J. W., & Folger, R. (1986). Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8(2), 141–157. https://doi.org/10.1300/J075v08n02_09.

    Article  Google Scholar 

  • Weitz, J. (1952). A neglected concept in the study of job satisfaction. Personnel Psychology, 5, 201–205. https://doi.org/10.1111/j.1744-6570.1952.tb01012.x.

    Article  Google Scholar 

  • Wright, J. D., & Hamilton, R. F. (1978). Work satisfaction and age: Some evidence for the ‘job change’ hypothesis. Social Forces, 56(4), 1140–1158. https://doi.org/10.2307/2577515.

    Article  Google Scholar 

Download references

Acknowledgments

Thanks to Steven Rogelberg for convincing me that there was something new to say about control variables and to Fred Oswald, Scott Tonidandel, and Jeremy Dawson for their incredibly helpful feedback on earlier versions of the paper.

Author information

Authors and Affiliations

  1. School of Information Systems and Management, Muma College of Business, University of South Florida, Tampa, FL, 33620, USA

    Paul E. Spector

Authors
  1. Paul E. Spector
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul E. Spector.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Spector, P.E. Mastering the Use of Control Variables: the Hierarchical Iterative Control (HIC) Approach. J Bus Psychol 36, 737–750 (2021). https://doi.org/10.1007/s10869-020-09709-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10869-020-09709-0

Keywords

Search

Navigation