Skip to main content

Advertisement

SpringerLink
Log in

To Aggregate or Not to Aggregate: Steps for Developing and Validating Higher-Order Multidimensional Constructs

  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

The use of higher-order multidimensional constructs (i.e., latent constructs comprised of standalone variables) in the organizational psychology and behavior literatures is becoming commonplace. Despite their advantages (e.g., greater parsimony and bandwidth), the development and validation of such constructs often unfolds in an indiscriminant fashion. It is not surprising, then, that much debate has arisen regarding the viability of many higher-order constructs. In this article, we outline ten recommendations for improving the construct- and criterion-related validity of higher-order constructs. Chief among these recommendations include the need for researchers to specify precise theoretical and empirical inclusion criteria, to rule out alternative explanations for the emergence of a higher-order factor and to assess incremental and relative importance. To illustrate how these recommendations play out, we apply them to core self-evaluation as an example. We believe that higher-order constructs may offer unique insight into work-relevant phenomena, provided they are established via rigorous means.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ambrose, M. L., & Schminke, M. (2009). The role of overall justice judgments in organizational justice research: A test of mediation. Journal of Applied Psychology, 94, 491–500.

    Article  PubMed  Google Scholar 

  • Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411–423.

    Article  Google Scholar 

  • Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110, 305–314.

    Article  Google Scholar 

  • Bollen, K. A., & Ting, K. -F. (2000). A tetrad test for causal indicators. Psychological Methods, 5, 3–22.

    Article  PubMed  Google Scholar 

  • Chang, C. -H., Ferris, D. L., Johnson, R. E., Rosen, C. C., & Tan, J. A. (in press). Core self-evaluations: A review and evaluation of the literature. Journal of Management (2012 Review Issue).

  • Chen, G. (in press). Evaluating the core: Critical assessment of core self-evaluations theory. Journal of Organizational Behavior.

  • Conway, J. M., & Lance, C. E. (2010). What reviewers should expect from authors regarding common method bias in organizational research. Journal of Business and Psychology, 25, 325–334.

    Article  Google Scholar 

  • Crampton, S. M., & Wagner, J. A., I. I. I. (1994). Percept-percept inflation in microorganizational research: An investigation of prevalence and effect. Journal of Applied Psychology, 79, 67–76.

    Article  Google Scholar 

  • Diamantopoulos, A., & Siguaw, J. A. (2006). Formative versus reflective indicators in organizational measure development: A comparison and empirical illustration. British Journal of Management, 17, 263–282.

    Article  Google Scholar 

  • Digman, J. M. (1997). Higher order factors of the Big Five. Journal of Personality and Social Psychology, 73, 1246–1256.

    Article  PubMed  Google Scholar 

  • Edwards, J. R. (2001). Multidimensional constructs in organizational behavior research: An integrative analytical framework. Organizational Research Methods, 4, 144–192.

    Article  Google Scholar 

  • Edwards, J. R. (2011). The fallacy of formative measurement. Organizational Research Methods, 14, 370–388.

    Article  Google Scholar 

  • Ferris, D. L., Rosen, C. C., Johnson, R. E., Brown, D. J., Risavy, S., & Heller, D. (2011). Approach or avoidance (or both?): Integrating core self-evaluations within an approach/avoidance framework. Personnel Psychology, 64, 137–161.

    Article  Google Scholar 

  • Franke, G. R., Preacher, K. J., & Rigdon, E. E. (2008). Proportional structural effects of formative indicators. Journal of Business Research, 61, 1229–1237.

    Article  Google Scholar 

  • Gerbing, D. W., Hamilton, J. G., & Freeman, E. B. (1994). A large-scale second-order structural equation model of the influence of management participation on organizational planning benefits. Journal of Management, 20, 859–885.

    Google Scholar 

  • Goldberg, L. R. (1992). The development of markers for the big-five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229.

    Article  Google Scholar 

  • Harrison, D. A., McLaughlin, M. E., & Coalter, T. M. (1996). Context, cognition, and common method variance: Psychometric and verbal protocol evidence. Organizational Behavior and Human Decision Processes, 68, 246–261.

    Article  Google Scholar 

  • Harrison, D. A., Newman, D. A., & Roth, P. L. (2006). How important are job attitudes? Meta-analytic comparisons of integrative behavioral outcomes and time sequences. Academy of Management Journal, 49, 305–325.

    Article  Google Scholar 

  • Jenkins, M., & Griffith, R. (2004). Using personality constructs to predict performance: Narrow or broad bandwidth. Journal of Business and Psychology, 19, 255–269.

    Article  Google Scholar 

  • Johnson, J. W. (2000). A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research, 35, 1–19.

    Article  Google Scholar 

  • Johnson, R. E., Rosen, C. C., & Levy, P. E. (2008). Getting to the core of core self-evaluations: A review and recommendations. Journal of Organizational Behavior, 29, 391–413.

    Article  Google Scholar 

  • Johnson, R. E., Rosen, C. C., Chang, C. -H., Djurdjevic, E., & Taing, M. U. (in press-a). Recommendations for improving the construct clarity of higher-order multidimensional constructs. Human Resource Management Review.

  • Johnson, R. E., Rosen, C. C., & Djurdjevic, E. (in press-b). Assessing the impact of common method variance on higher-order multidimensional constructs. Journal of Applied Psychology.

  • Judge, T. A., Locke, E. A., & Durham, C. C. (1997). The dispositional causes of job satisfaction: A core evaluations approach. Research in Organizational Behavior, 19, 151–188.

    Google Scholar 

  • Judge, T. A., Erez, A., Bono, J. E., & Thoresen, C. J. (2003). The core self-evaluations scale: Development of a measure. Personnel Psychology, 56, 303–331.

    Article  Google Scholar 

  • Kelley, T. L. (1927). Interpretation of educational measurements. Yonkers, New York: World Book Company.

    Google Scholar 

  • Kline, T. J. B., Sulsky, L. M., & Rever-Moriyama, S. D. (2002). Common method variance and specification errors: A practical approach to detection. Journal of Psychology, 134, 401–421.

    Article  Google Scholar 

  • Law, K. S., Wong, C. S., & Mobley, W. H. (1998). Toward a taxonomy of multidimensional constructs. Academy of Management Review, 23, 741–755.

    Google Scholar 

  • LeBreton, J. M., Hargis, M. B., Griepentrog, B., Oswald, F. L., & Ployhart, R. E. (2007). A multidimensional approach for evaluating variables in organizational research and practice. Personnel Psychology, 60, 475–498.

    Article  Google Scholar 

  • Macey, W. H., & Schneider, B. (2008). The meaning of employee engagement. Industrial and Organizational Psychology: Perspectives on Science and Practice, 1, 3–30.

    Article  Google Scholar 

  • MacKenzie, S. B., Podsakoff, P. M., & Jarvis, C. B. (2005). The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90, 710–730.

    Article  PubMed  Google Scholar 

  • McCrae, R. R., & Costa, P. T., Jr. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52, 81–90.

    Article  PubMed  Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., Lee, J. -Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903.

    Article  PubMed  Google Scholar 

  • Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009). A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance. Organizational Research Methods, 12, 762–800.

    Article  Google Scholar 

  • Rosen, C. C., Chang, C. H., Johnson, R. E., & Levy, P. E. (2009). Perceptions of the organizational context and psychological contract breach: Assessing competing perspectives. Organizational Behavior and Human Decision Processes, 108, 202–217.

    Article  Google Scholar 

  • Rushton, J. P., & Irwing, P. (2008). A general factor of personality (GFP) from two meta-analyses of the Big Five: Digman (1997) and Mount, Barrick, Scullen, and Rounds (2005). Personality and Individual Differences, 45, 679–683.

    Article  Google Scholar 

  • Schmitt, N. (2004). Beyond the Big Five: Increases in understanding and practical utility. Human Performance, 17, 349–359.

    Article  Google Scholar 

  • Spearman, C. (1904). “General intelligence” objectively determined and measured. American Journal of Psychology, 15, 201–293.

    Article  Google Scholar 

  • Spector, P. E. (1992). Summated rating scales. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Taing, M. U., Johnson, R. E., & Jackson, E. M. (2010, August). On the nature of core self-evaluation: A formative or reflective construct? Paper presented at the 70th Academy of Management Annual Meeting, Montreal, Quebec.

  • Tonidandel, S., & LeBreton, J. M. (2011). Relative importance analysis: A useful supplement to regression analysis. Journal of Business and Psychology, 26, 1–9.

    Article  Google Scholar 

  • Tonidandel, S., LeBreton, J. M., & Johnson, J. W. (2009). Statistical significance tests for relative weights. Psychological Methods, 14, 387–399.

    Article  PubMed  Google Scholar 

  • Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? Journal of Applied Psychology, 74, 462–468.

    Article  Google Scholar 

  • Williams, L. J., Hartman, N., & Cavazotte, F. (2010). Method variance and marker variables: A review and comprehensive CFA marker technique. Organizational Research Methods, 13, 477–514.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management, Broad College of Business, Michigan State University, East Lansing, MI, 48824, USA

    Russell E. Johnson

  2. Department of Management, Sam M. Walton College of Business, University of Arkansas, Fayetteville, USA

    Christopher C. Rosen

  3. Department of Psychology, Michigan State University, East Lansing, MI, 48824, USA

    Chu-Hsiang Chang

Authors
  1. Russell E. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher C. Rosen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chu-Hsiang Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Russell E. Johnson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Johnson, R.E., Rosen, C.C. & Chang, CH. To Aggregate or Not to Aggregate: Steps for Developing and Validating Higher-Order Multidimensional Constructs. J Bus Psychol 26, 241–248 (2011). https://doi.org/10.1007/s10869-011-9238-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10869-011-9238-1

Keywords

Search

Navigation