Incremental criterion prediction of personality facets over factors: Obtaining unbiased estimates and confidence intervals

Highlights

• We review methods for estimating prediction of personality facets over factors.

• Recommendations for comparing predictions by facets and factors is provided.

• A method for calculating confidence intervals on incremental prediction is shown.

• An R package for performing the recommended analyses is provided.

Abstract

Many researchers have argued that higher order models of personality such as the Five Factor Model are insufficient, and that facet-level analysis is required to better understand criteria such as well-being, job performance, and personality disorders. However, common methods in the extant literature used to estimate the incremental prediction of facets over factors have several shortcomings. This paper delineates these shortcomings by evaluating alternative methods using statistical theory, simulation, and an empirical example. We recommend using differences between Olkin–Pratt adjusted r-squared for factor versus facet regression models to estimate the incremental prediction of facets and present a method for obtaining confidence intervals for such estimates using double adjusted-r-squared bootstrapping. We also provide an R package that implements the proposed methods.

Introduction

Personality trait researchers have long been interested in how many personality traits are required to adequately capture individual differences. Hierarchical models of personality traits provide multiple levels of description, typified by the Five Factor Model in which the global factors of extraversion, neuroticism, conscientiousness, agreeableness and openness each consist of six facets representing a more detailed level of personality. Despite the popularity of the Big Five, there has been substantial debate about the relative merits of factor and facet assessments of personality (Ashton, 1998, Ashton et al., 1995, Ashton et al., 2014, Christiansen and Robie, 2011, O’Neill et al., 2013, Paunonen, 1998, Paunonen and Ashton, 2001, Paunonen et al., 2003, Paunonen et al., 1999, Salgado et al., 2013). Additionally, comparing the predictive value of a model with 30 facet predictors to one with only five factor predictors has presented a challenge for researchers concerned with issues of over fitting. Personality researchers seeking to predict outcomes such as well-being (Siegler & Brummett, 2000), job performance (Ashton, 1998, Christiansen and Robie, 2011, Ones and Viswesvaran, 1996, Salgado et al., 2013, Tett et al., 2003), and personality disorders (Bagby et al., 2005, Dyce and O’Connor, 1998) have then had to decide whether to include facets or only the Big Five factors as predictors.

Typically, incremental prediction of facets over factors has been estimated by subtracting the variance explained in a criterion by factors from that explained by facets. However, researchers have used many different estimators of variance explained, including unadjusted r-squared, adjusted r-squared, and cross-validated r-squared, combined with different regression procedures including direct entry (Mershon & Gorsuch, 1988) and stepwise regression (Baudin et al., 2011, Dyce and O’Connor, 1998, Ekehammar and Akrami, 2007, Quevedo and Abella, 2011, Schimmack et al., 2004); some studies have simply reported zero-order correlations (e.g., Rothmann and Coetzer, 2002, Siegler and Brummett, 2000). Thus, a principled selection of estimators is lacking (e.g., see critical review by O’Connor and Paunonen, 2007). Furthermore, the use of small sample sizes (e.g., Ashton et al., 1995, Mershon and Gorsuch, 1988, Schimmack et al., 2004) and incomplete facet–factor comparisons, based on the selection of subsets of either facets or factors (e.g., Ashton et al., 1995, Bagby et al., 2005, Dudley et al., 2006, Fruyt et al., 2006, Hastings and O’Neill, 2009, Paunonen and Ashton, 2001, Salgado et al., 2013, Stephan, 2009), has limited the available empirical evidence regarding the overall incremental value of facets over factors. Also, existing research has not explicitly specified a population parameter of interest. Furthermore, as will be shown many existing methods that have been used for estimating incremental prediction result in biased estimates. The lack of reporting of confidence intervals further compounds these issues. Further clarity is needed about these foundational issues in order to more clearly quantify the gains that can be achieved by the inclusion of facets in predictive models. Thus, existing approaches are insufficient for researchers seeking to make conclusions about the relative utility of facet- versus factor-level analysis in personality research.

The purpose of the current paper is (1) to identify the population parameter of interest for research on incremental prediction of facets over factors; (2) to compare methods for obtaining an estimate of this population parameter to demonstrate relative bias across methods through a series of simulations, and (3) to provide a method for reporting confidence intervals around this estimate. We also critically review the broader set of approaches that have been used to compare factor versus facet prediction of criterion variables. Based on our comparison of methods, we recommend the use of the Olkin–Pratt adjusted r-squared as an estimator, and the reporting of double-adjusted r-squared bootstrap (DAB) confidence intervals. We also review and make recommendations regarding methods for identifying which particular facets are of greatest incremental benefit. Finally, we present an R package that implements all the proposed methods.