Abstract
When multi-item questionnaires are included in psychological research, many factors can influence the response given. One such factor that has traditionally been overlooked is the potential impact of item-order effects. This paper extends upon the work of Kaplan et al. (J Happiness Stud 14:1443–1458, 2013. doi: 10.1007/s10902-012-9388-5) who explored item-order effects in measures of job satisfaction, and applies similar principles to the measurement of life satisfaction, or subjective wellbeing, by exploring item-order effects within the Personal Wellbeing Index (PWI; IWBG in Personal Wellbeing Index, 5th edn. Australian Centre on Quality of Life, Deakin University, Melbourne, 2013. http://www.deakin.edu.au/research/acqol/intruments/wellbeing-index/index.php). In a preliminary study, participants completed the PWI in its standard format (general-specific) and psychometric properties were compared to those who completed the PWI in an alternate format (specific-general). Analyses revealed that the PWI performed adequately for both groups, though there were subtle indications of item-order effects. In a second study, the order of the PWI domains was randomised (random-order) and compared to the standard format (fixed-order). Results revealed lower mean scores and more variation in scores when items were randomised. Overall, the PWI performed as expected for most interrogative analyses. It achieved a single factor solution, no matter the order of items, and the same domains emerged as significant unique predictors of general life satisfaction. The study highlights the importance of exploring item-order effects as part of the psychometric validation procedure, and it is recommended that all new scales be subject to this investigation to reduce measurement error and improve accuracy in psychological assessment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Australian Bureau of Statistics. (2016). Household use of information technology, Australia, 2014–2015, cat no. 8146.0. Canberra, Australia: ABS.
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1982). Political information processing: Question order and context effects. Political Behavior, 4, 177–200.
Carlson, J. E., Mason, R., Saltiel, J., & Sangster, R. (1995). Assimilation and contrast effects in general/specific questions. Rural Sociology, 60, 666–673.
Cummins, R. (2010). Subjective wellbeing, homeostatically protected mood and depression: A synthesis. Journal of Happiness Studies, 11, 1–17. doi:10.1007/s10902-009-9167-0.
Cummins, R. A., Eckersley, R., Pallant, J., Van Vugt, J., & Misajon, R. (2003). Developing a national index of subjective wellbeing: The Australian Unity Wellbeing Index. Social Indicators Research, 64, 159–190.
Cummins, R. A., & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case for subjective quality of life measurement. In Proceedings of the second international conference on quality of life in cities (pp. 74–93). National University of Singapore, Singapore.
Cummins, R. A., Woerner, J., Weinberg, M., Collard, J., Hartley-Clark, J., Horfiniak, K., et al. (2013). Australian Unity Wellbeing Index Survey 30.0: The wellbeing of Australians—social media, personal achievement and work. Melbourne, Australia: Centre on Quality of Life, School of Psychology, Deakin University. http://www.acqol.com.au/reports/auwbi.php.
Cummins, R. A., & Wooden, M. (2014). Personal resilience in times of crisis: The implications of SWB homeostasis and set-points. Journal of Happiness Studies, 15, 223–235.
Diener, E. (1984). Subjective well-being. Psychological Bulletin, 95, 542–575. doi:10.1037/0033-2909.95.3.542.
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71–76.
Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics, Vol. 3: Speech acts (Vol. 3, pp. 225–242). New York: Seminar Press.
Haviland, S. E., & Clark, H. H. (1974). What’s new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512–521.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
IBM. (2013). IBM SPSS statistics for windows, Version 22.0. Armonk, NY: IBM Corp.
International Wellbeing Group. (2013). Personal Wellbeing Index (5th ed.). Melbourne: Australian Centre on Quality of Life, Deakin University. Retrieved from: http://www.deakin.edu.au/research/acqol/intruments/wellbeing-index/index.php.
Kaplan, S., Luchman, J., & Mock, L. (2013). General and specific question sequence effects in satisfaction surveys: Integrating directional and correlational effects. Journal of Happiness Studies, 14, 1443–1458. doi:10.1007/s10902-012-9388-5.
Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of response order effects in survey measurement. Public Opinion Quarterly, 51, 201–219.
Lau, A. L. D., & Cummins, R. A. (2005). Test–retest reliability of the Personal Wellbeing Index. Unpublished research report, The Hong Kong Polytechnic University.
Nimon, K. F., & Oswald, F. L. (2013). Understanding the results of multiple linear regression: Beyond standardized regression coefficients. Organizational Research Methods, 16, 650–674.
OECD. (2013). Guidelines on measuring subjective wellbeing. Retrieved from: http://www.oecd.org/statistics/Guidelines%20on%20Measuring%20Subjective%20Well-being.pdf.
Schimmack, U. (2008). The structure of subjective well-being. In M. Eid & R. J. Larsen (Eds.), The science of subjective well-being (pp. 97–123). New York, NY: Guilford Press.
Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys: Experiments on question form, wording and context. Amherst, MA: Academic Press Inc.
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.
Schwarz, N., & Bless, H. (1992). Constructing reality and its alternatives: An inclusion/exclusion model of assimilation and contrast effects in social judgment. In L. L. Martin & A. Tesser (Eds.), The construction of social judgments (pp. 217–245). Hillsdale, NJ: Lawrence Erlbaum Associates Inc.
Schwarz, N., Strack, F., & Mai, H. P. (1991). Assimilation and contrast effects in part-whole question sequences: A conversational logic analysis. Public Opinion Quarterly, 55, 3–23.
Srull, T. K., & Wyer, R. S. (1980). Category accessibility and social perception: Some implications for the study of person memory and interpersonal judgments. Journal of Personality and Social Psychology, 38, 841–856. doi:10.1037/0022-3514.38.6.841.
Strack, F., & Martin, L. L. (1987). Thinking, judging, and communicating: A process account of context effects in attitude surveys. In F. Strack & L. L. Martin (Eds.), Social information processing and survey methodology (pp. 123–148). New York: Springer.
Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103, 299–314. doi:10.1037/0033-2909.103.3.299.
Tourangeau, R., Rasinski, K. A., & Bradburn, N. (1991). Measuring happiness in surveys: A test of the subtraction hypothesis. Public Opinion Quarterly, 55, 255–266.
World Health Organization. (2013). Joint meeting of experts on targets and indicators for health and well-being in Health 2020. Copenhagen: WHO Regional Office for Europe.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Weinberg, M.K., Seton, C. & Cameron, N. The Measurement of Subjective Wellbeing: Item-Order Effects in the Personal Wellbeing Index—Adult. J Happiness Stud 19, 315–332 (2018). https://doi.org/10.1007/s10902-016-9822-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10902-016-9822-1