Skip to main content
SpringerLink
Log in

Investigating the effects of personality traits on pair programming in a higher education setting through a family of experiments

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Evidence from our systematic literature review revealed numerous inconsistencies in findings from the Pair Programming (PP) literature regarding the effects of personality on PP’s effectiveness as a pedagogical tool. In particular: i) the effect of differing personality traits of pairs on the successful implementation of pair-programming (PP) within a higher education setting is still unclear, and ii) the personality instrument most often used had been Myers-Briggs Type Indicator (MBTI), despite being an indicator criticized by personality psychologists as unreliable in measuring an individual’s personality traits. These issues motivated the research described in this paper. We conducted a series of five formal experiments (one of which was a replicated experiment), between 2009 and 2010, at the University of Auckland, to investigate the effects of personality composition on PP’s effectiveness. Each experiment looked at a particular personality trait of the Five-Factor personality framework. This framework comprises five broad traits (Openness to experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism), and our experiments focused on three of these - Conscientiousness, Neuroticism, and Openness. A total of 594 undergraduate students participated as subjects. Overall, our findings for all five experiments, including the replication, showed that Conscientiousness and Neuroticism did not present a statistically significant effect upon paired students’ academic performance. However, Openness played a significant role in differentiating paired students’ academic performance. Participants’ survey results also indicated that PP not only caused an increase in satisfaction and confidence levels but also brought enjoyment to the tutorial classes and enhanced students’ motivation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Our focus was on between-pair differences.

  2. FFM is detailed in Section 3

  3. See detail about FFM in Section 3

  4. The personality test is available at the public domain http://www.personal.psu.edu//j5j/IPIP/

  5. The source code is available from the first author

References

  • Acuna ST, Gomez M, Juristo N (2009) How do personality, team process and task characteristics relate to job satisfaction and software quality? Inform Software Tech 51:627–639

    Article  Google Scholar 

  • Arisholm B, Gallis H, Dyba T, Sjoberg DIK (2007) Evaluating pair programming with respect to system complexity and programmer expertise. IEEE Trans Software Eng 33(2):65–86

    Article  Google Scholar 

  • Barrick MR, Mount MK (1991) The big five personality dimensions and job performance: a meta-analysis. Personality Psychology 44:1–26

    Article  Google Scholar 

  • Barrick MR, Stewart GL, Neubert MJ, Mount MK (1998) Relating member ability and personality to work-team processes and team effectiveness. J Appl Psychol 83(3):377–391

    Article  Google Scholar 

  • Barrick MR, Mount MK, Judge TA (2001) Personality and performance at the beginning of the new millennium: what do we know and where do we go next? Pers Perform 9(1–2):9–30

    Google Scholar 

  • Basili VR, Rombach HD (1988) The TAME Project: towards improvement-oriented software environments. IEEE Trans Software Eng 14(6)

  • Basili VR, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Software Eng 25(4):456–473

    Article  Google Scholar 

  • Beck K (1999) Extreme programming explained: embrace change, 2nd edn. Addison-Wesley, Boston

    Google Scholar 

  • Bell ST (2007) Deep-level composition variables as predictors of team performance: a meta-analysis. J Appl Psychol 92(3):595–615

    Article  Google Scholar 

  • Boekaerts M (1996) Personality and the psyhocology of learning. Eur J Pers 10:377–404

    Article  Google Scholar 

  • Bowers CA, Pharmer JA, Salas E (2000) When member homogeneity is needed in work teams: a meta-analysis. Small Group Res 31(3):305–327

    Article  Google Scholar 

  • Boyle GJ (1995) Myers-Briggs type indicator (MBTI): some psychometric limitations. Aust Psychol 30(1):71–74

    Article  MathSciNet  Google Scholar 

  • Bradley JH, Hebert FJ (1997) The effect of personality type on team performance. J Manag Dev 16(5):337–353

    Article  Google Scholar 

  • Buchanan T, Johnson JA, Goldberg LR (2005) Implementing a five-factor personality inventory for use on the Internet. J Psychol Assess 21(2):115–127

    Article  Google Scholar 

  • Burch GSJ, Anderson N (2008) Personality as a predictor of work-related behavior and performance: recent advances and directions for future research. In: Hodgkinson GP, Ford JK (eds) International review of industrial and organizational psychology. Wiley, Chichester, UK, pp. 261-305

  • Busato VV, Prins FJ, Elshout JJ, Hamaker C (2000) Intellectual ability, learning style, personality, achievement motivation and academic success of psychology students in higher education. Pers Indiv Differ 29(6):1057–1068

    Article  Google Scholar 

  • Cegielski CG, Hall DJ (2006) What makes a good programmer? Comm ACM 49(10):73–75

    Article  Google Scholar 

  • Chamorro-Premuzic T, Furnham A (2003a) Personality predicts academic performance: evidence from two longitudinal university samples. J Res Pers 37:319–338

    Article  Google Scholar 

  • Chamorro-Premuzic T, Furnham A (2003b) Personality traits and academic examination performance. Eur J Pers 17:237–250

    Article  Google Scholar 

  • Chamorro-Premuzic, T, Furnham, A (2008) Personality, intelligence and approaches to learning as predictors of academic performance. Personality and Individual Differences, 44(7):1596–1603

    Google Scholar 

  • Choi KS (2004) A discovery and analysis of influencing factors of pair programming. Unpublished Ph.D. Dissertation, New Jersey Institute of Technology, USA

  • Choi KS, Deek FP, Im I (2008) Exploring the underlying aspects of pair programming: the impact of personality. Inform Software Tech 50(11):1114–1126

    Article  Google Scholar 

  • Cockburn A (2002) Agile software development. Addison-Wesley Longman Publishing Co. Inc., Boston

    Google Scholar 

  • Cockburn A, Williams L (2001) The costs and benefits of pair programming. In: Extreme programming examined. Addison-Wesley Longman Publishing Co., Inc, Boston, MA, pp. 223-243

  • Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. L. Erlbaum Associates, Hillsdale

    MATH  Google Scholar 

  • Conard MA (2006) Aptitude is not enough: how personality and behavior predict academic performance. J Res Pers 40:339–346

    Article  Google Scholar 

  • Cook TD, Campbell DT (1979) Quasi-experimentation: design & analysis issues for field settings. Rand McNally College Publishing, Chicago

    Google Scholar 

  • Costa PT, McCrae RR (1992) NEO PI-R professional manual. Psychological Assessment Resources, Odessa

    Google Scholar 

  • Costa PT, McCrae RR (1995) Domain and facets: hierarchical personality assessment using the revised NEO personality inventory. J Pers Assess 64:21–50

    Article  Google Scholar 

  • Costa PT, Terracciano A, McCrae RR (2001) Gender differences in personality traits across culture: robust and surprising findings. J Pers Soc Psychol 81(2):322–331

    Article  Google Scholar 

  • Cunha ADD, Greathead D (2007) Does personality matter? An analysis of code-review ability. Comm ACM 50(5):109–112

    Article  Google Scholar 

  • Davito A (1985) A review of the Myers-Briggs type indicator. In: Mitchell J (ed) Ninth mental measurement yearbook. University of Nebraska Press, Lincoln

    Google Scholar 

  • De Raad B, Schouwenburg HC (1996) Personality in learning and education: a review. Eur J Pers 10:303–336

    Article  Google Scholar 

  • DeClue TH (2003) Pair programming and pair trading: effects on learning and motivation in a CS2 courses. J Comput Sci Coll 18(5):49–56

    Google Scholar 

  • Dollinger SJ, Orf LA (1991) Personality and performance in "personality": conscientiousness and openness. J Res Pers 25:276–284

    Article  Google Scholar 

  • Donaldson SI, Grant-Vallone EJ (2002) Understanding self-report bias in organizational behavior research. J Bus Psychol 17(2):245–260

    Article  Google Scholar 

  • Driskell JE, Salas E, Goodwin FF, O'Shea PG (2006) What makes a good team player? Personality and team effectiveness. Group Dyn-Theor Res Pract 10(4):249–271

    Article  Google Scholar 

  • Duff A, Boyle E, Dunleavy K, Ferguson J (2004) The relationship between personality, approach to learning and academic performance. Pers Indiv Differ 36(8):1907–1920

    Article  Google Scholar 

  • Dyba T, Kampenes VB, Sjoberg DIL (2006) A systematic review of statistical power in software engineering experiments. Inform Software Tech 48:745–755

    Article  Google Scholar 

  • Farsides T, Woodfield R (2003) Individual differences and undergraduate academic success: the roles of personality, intelligence, and application. Pers Indiv Differ 34(7):1225–1243

    Article  Google Scholar 

  • Faul F, Erdfelder E, Lang A-G, Buchner A (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39:175–191

    Article  Google Scholar 

  • Feingold A (1994) Gender differences in personality: a meta-analysis. Psychol Bull 116(3):429–456

    Article  Google Scholar 

  • Feldt R, Angelis L, Samuelsson M (2008) Towards individualized software engineering: Empirical studies should collect psychometrics. Paper presented at the CHASE'08

  • Francis L, Craig C, Robbins M (2008) The relationship between the Keirsey Temperament Sorter and the short-form Revised Eysenck Personality Questionnaire. J Indiv Differ 29(2):116–120

    Article  Google Scholar 

  • Furnham A (1996) The big five Vs the big four: the relationship between Myers-Briggs Type Indicator (MBTI) and NEO-PI five factor model of personality. Pers Indiv Differ 21(2):303–307

    Article  Google Scholar 

  • Gallis H, Arisholm E, Dyba T (2003) An initial framework for research on pair programming. Proc. of the International Symposium on Empirical Software Engineering (ISESE'03), pp. 132-142

  • Goldberg LR, Johnson JA, Eber HW, Hogan R, Ashton MC, Cloninger CR et al (2006) The international personality item pool and the future of public-domain personality measures. J Res Pers 40(1):84–96

    Article  Google Scholar 

  • Gorla N, Lam YW (2004) Who should work with whom? Building effective software project teams. Comm ACM 47(6):79–82

    Article  Google Scholar 

  • Gosling SD, Vazire S, Srivastava S, John OP (2004) Should we trust web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. Am Psychol 59(2):93–104

    Article  Google Scholar 

  • Hanks B (2006) Student attitudes toward pair programming. Proceedings of the 11th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE06), pp. 113-117

  • Hannay JE, Arisholm E, Engvik H, Sjoberg DIK (2010) Effects of personality on pair programming. IEEE Trans Software Eng 36(1):61–80

    Article  Google Scholar 

  • Hicks L (1984) Conceptual and empirical analysis of some assumptions of an explicitly technological theory. J Pers Soc Psychol 46:1118–1131

    Article  Google Scholar 

  • Ho C-w (2004) Examining impact of pair programming on female students (No. TR-2004-20). Raleigh, NC: North Carolina State University

  • Hsu B-F, Wu W-L, Yeh R-S (2007) Personality composition, affective tie, and knowledge sharing: a team level analysis. Proceedings of the Portland International Center for Management of Engineering and Technology (PICMET 2007), pp. 2583–2592

  • John OP, Srivastava S (1999) The big five trait taxonomy: History, measurement and theoretical perspectives. In: Pervin LA, John OP (eds) Handbook of personality: theory and research. The Guilford Press, New York/London, pp 102–138

    Google Scholar 

  • Johnson JA (2005) Ascertaining the validity of individual protocols from Web-based personality inventories. J Res Pers 39:103–128

    Article  Google Scholar 

  • Johnson JA (2008) The IPIP-NEO personality assessment tools. Retrieved from http://www.personal.psu.edu/j5j/IPIP/

  • Juristo N, Moreno AM (2001) Basics of software engineering experimentation. Kluwer Academic Publishers, Boston

    Book  MATH  Google Scholar 

  • Karn JS, Cowling T (2006) A follow up study of the effect of personality on the performance of software engineering teams. Proceedings of the ISESE'06, pp. 232-241

  • Katira N, Williams L, Wiebe E, Miller C, Balik S, Gehringer E (2004) On understanding compatibility of student pair programmers. SIGCSE Bull 36(1):7–11

    Article  Google Scholar 

  • Keirsey D (1998) Please understand me II. Prometheus Nemesis Book, Del Mar

    Google Scholar 

  • Keirsey D, Bates M (1984) Please understand me. Prometheus Nemesis Book, Del Mar

    Google Scholar 

  • Kichuk SL, Wiesner WH (1997) The big five personality factors and team performance: implications for selecting successful product design teams. J Eng Technol Manage 14:195–221

    Article  Google Scholar 

  • Lan L, Lian Z (2010) Application of statistical power analysis - How to determine the right sample size in human health, comfort, and productivity research. Build Environ 45:1202–1213

    Article  Google Scholar 

  • Layman L (2006) Changing students' perceptions: an analysis of the supplementary benefits of collaborative software development. Proceedings of the 19th Conference on Software Engineering Education & Training (CSEET'06), pp. 159-166

  • Leech NL, Barrett KC, Morgan GA (2005) SPSS for intermediate statistics: use and interpretation, 2nd edn. Lawrence Erlbaum Associates, Mahwah

    Google Scholar 

  • Leedy PD, Ormrod JE (2005) Practical research planning and design, 8th edn. Pearson Merril Prentice Hall

  • LePine JA (2003) Team adaptation and postchange performance: effects of team composition in terms of members' cognitive ability and personality. J Appl Psychol 88(1):27–39

    Article  Google Scholar 

  • Levine JM, Moreland RL (1990) Progress in small group research. Annu Rev Psychol 41:585–634

    Article  Google Scholar 

  • Lounsbury JW, Sundstrom E, Loveland JM, Gibson LW (2003) Intelligence, "Big Five" personality traits, and work drive as predictors of course grade. Pers Indiv Differ 35:1231–1239

    Article  Google Scholar 

  • Matzler K, Renzl B, Muller J, Herting S, Mooradian TA (2008) Personality traits and knowledge sharing. J Econ Psychol 29:301–313

    Article  Google Scholar 

  • McCrae RR, Costa PT (1989) Reinterpreting the Myers-Briggs Type Indicator from the perspective of the five-factor model of personality. J Pers 57(1):17–40

    Article  Google Scholar 

  • McCrae RR, Costa PT (1997) Personality trait structure as a human universal. Am Psychol 52:509–516

    Article  Google Scholar 

  • McCrae RR, John OP (1992) An introduction to the five-factor model and its application. J Pers 60(2):175–215

    Article  Google Scholar 

  • McDowell C, Werner L, Bullock HE, Fernald J (2003) The impact of pair programming on student performance, perception and persistence. Proceedings of the 25th International Conference on Software Engineering (ICSE'03), pp. 602-607

  • Morgan GA, Leech NL, Gloeckner GW, Barrett KC (2004) SPSS for Introductory Statistics. Use and Interpretation, 2nd edn. Lawrence Erlbaum Associates, Inc, New Jersey

    Google Scholar 

  • Murphy KR, Myors B (2003) Statistical power analysis: a simple and general model for traditional and modern hypothesis tests, 2nd edn. Lawrence Erlbaum Associates, New Jersey

    Google Scholar 

  • Myers JL, Well AD (2003) Research design and statistical analysis, 2nd edn. Lawrence Erlbaum Associates, Inc, New Jersey

    Google Scholar 

  • Myers IB, McCaulley MH, Quenk NL, Hammer A (1998) MBTI Manual (A guide to the development and use of the Myers Briggs type indicator), 3rd edn. Consulting Psychologists Press

  • Neuman GA, Wagner SH, Christiansen ND (1999) The relationship between work-team personality composition and the job performance of teams. Group & Organization Management 24(1):28–45

    Article  Google Scholar 

  • Nguyen NT, Allen LC, Fraccastoro K (2005) Personality predicts academic performance: exploring the moderating role of gender. J High Educ Pol Manag 27(1):105–116

    Google Scholar 

  • Norman G (2010) Likert scales, levels of measurement and the "laws" of statistics. Advances in Health Science Education (Online First)

  • O'Connor MC, Paunonen SV (2007) Big five personality predictors of post-secondary academic performance. Pers Indiv Differ 43:971–990

    Article  Google Scholar 

  • Pallant JF (2007) SPSS Survival Manual: a step by step guide to data analysis using SPSS for Windows (Version 15), 3rd edn. Allen & Unwin, Crows Nest

    Google Scholar 

  • Paunonen SV, Ashton MC (2001) Big five predictors of academic achievement. J Res Pers 35:78–90

    Article  Google Scholar 

  • Peeters MAG, Tuijil HFJMV, Rutte CG, Reymen IMMJ (2006) Personality and team performance: a meta-analysis. Eur J Pers 20:377–396

    Article  Google Scholar 

  • Peslak AR (2006) The impact of personality on information technology team projects. Proceedings of the SIGMIS-CPR'06, pp. 273-279

  • Pfleeger SL (1995) Experimental design and analysis in software engineering. Ann Software Eng 1(1):219–253

    Article  Google Scholar 

  • Philips P, Abraham C, Bond R (2003) Personality, cognition, and university students’ examination performance. Eur J Pers 17:435–448

    Article  Google Scholar 

  • Pieterse V, Kourie DG (2006) Software engineering team diversity and performance. Proceedings of the South African Institute for Computer Scientists and Information Technologists (SAICSIT), 2006, pp. 180-186

  • Poropat AE (2009) A meta-analysis of the five-factor model of personality and academic performance. Psychol Bull 135(2):322–338

    Article  Google Scholar 

  • Pulford BD, Sohal H (2006) The influence of personality on higher education students' confidence in their academic abilities. Pers Indiv Differ 41(8):1409–1419

    Article  Google Scholar 

  • Puus U, Seeba A, Salumaa P, Heiberg S (2004) Analyzing pair-programmer's satisfaction with the method, the result, and the partner. Proceedings of the 5th International Conference on Extreme Programming and Agile Processes in Software Engineering (XP 2004), LNCS 3092, pp. 246-249

  • Rutherfoord RH (2001) Using personality inventories to help form teams for software engineering class projects. Proc ITiCSE 2001:73–76

    Google Scholar 

  • Salleh N, Mendes E, Grundy J, Burch GSJ (2009) An empirical study of the effects of personality in pair programming using the five-factor model. Proceedings of the 3rd ACM-IEEE Int’l Symposium on Empirical Software Engineering & Measurement (ESEM 2009), pp. 214-225

  • Salleh N, Mendes E, Grundy J, Burch GSJ (2010a) An empirical study of the effects of conscientiousness in pair programming using the five-factor personality model. Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (ICSE 2010), 1, pp. 577-586

  • Salleh N, Mendes E, Grundy J, Burch GSJ (2010b) The effects of neuroticism on pair programming: An empirical study in the higher education context. Proceedings of the 4th ACM-IEEE Int’l Symposium on Empirical Software Engineering and Measurement (ESEM 2010)

  • Salleh N, Mendes E, Grundy J (2011a) Empirical studies of pair programming for CS/SE teaching in higher education: A systematic literature review. IEEE Trans Software Eng 37(4):509–525

    Article  Google Scholar 

  • Salleh N, Mendes E, Grundy J (2011b) The effects of openness to experience on pair programming in a higher education context. In: Proceedings of the 24th IEEE-CS Conference on Software Engineering Education and Training (CSEET2011). Honolulu, Hawaii, IEEE Computer Society

  • Saulsman LM, Page AC (2003) Can trait measures diagnose personality disorders? Curr Opin Psychiatr 16(1):83–88

    Article  Google Scholar 

  • Schmitt N (2008) The interaction of neuroticism and gender and its impact on self-efficacy and performance. Hum Perform 21:49–61

    Article  Google Scholar 

  • Schriesheim C, Hinkin T, Podsakoff P (1991) Can ipsative and single-item measures produce erroneous results in field studies of French & Raven's (1959) five bases of power? J Appl Psychol 76:106–144

    Article  Google Scholar 

  • Sfetsos P, Stamelos I, Angelis L, Deligiannis I (2006) Investigating the impact of personality types on communication and collaboration-viability in pair programming - an empirical study. Proceedings of the 7th International Conference on Extreme Programming and Agile Processes in Software Engineering (XP 2006), LNCS 4044, pp. 43-52

  • Sfetsos P, Stamelos I, Angelis L, Deligiannis I (2009) An experimental investigation of personality types impact on pair effectiveness in pair programming. Empir Software Eng 14(2):187–226

    Article  Google Scholar 

  • Tabachnick BG, Fidell LS (2001) Using multivariate statistics, 4th edn. Allyn and Bacon, Boston

    Google Scholar 

  • Thomas L, Ratcliffe M, Robertson A (2003) Code warriors and code-a-phobes: a study in attitude and pair programming. SIGCSE Bull 35(1):363–367

    Article  Google Scholar 

  • Trochim WMK (2006) Research methods knowledge base, 2nd edn. From http://www.socialresearchmethods.net/kb/considea.php (version current as of October 20, 2006)

  • Walle T, Hannay JE (2009) Personality and the nature of collaboration in pair programming. Proceedings of the 3rd Int'l Symp. Empirical Software Engineering & Measurement (ESEM 2009), pp. 203-213

  • Weinberg GM (1971) The psychology of computer programming. Van Nostrand Reinhold, New York

    Google Scholar 

  • Whyte JJ (2006) The use of surrogate outcome measures. A case study: home prothrombin monitors. In: Becker KM, Whyte JJ (eds) Clinical evaluation of medical devices: principles and case studies, 2nd edn. Humana Press, New Jersey

    Google Scholar 

  • Williams L, Kessler RR, Cunningham W, Jeffries R (2000) Strengthening the case for pair programming. IEEE Software 17(4):19–25

    Article  Google Scholar 

  • Williams L, McDowell C, Nagappan N, Fernald J, Werner L (2003) Building pair programming knowledge through a family of experiments. Proceedings 2003 International Symposium on Empirical Software Engineering (ISESE 2003), pp. 143-152

  • Williams L, Layman L, Osborne J, Katira N (2006) Examining the compatibility of student pair programmers. Proceedings of the Conference on AGILE 2006 (AGILE'06), IEEE Computer Society, pp. 411-420

  • Yuan K, Maxwell S (2005) On the post hoc power in testing mean differences. J Educ Behav Stat 30(2):141–167

    Article  Google Scholar 

Download references

Acknowledgments

This research is supported by the Ministry of Higher Education Malaysia. The authors would like to thank all tutors and demonstrators of CS101 and CS230 at the University of Auckland (2008-2010) for the help given to run the experiments. Thanks also to all students who have participated in the experiments.

Author information

Authors and Affiliations

  1. Department of Computer Science, International Islamic University Malaysia, P.O. Box 10, 50728, Kuala Lumpur, Malaysia

    Norsaremah Salleh

  2. School of Computing, Blekinge Istitute of Technology, 37179, Karlskrona, Sweden

    Emilia Mendes

  3. Faculty of Information and Communication Technology, Swinburne University of Technology, P.O. Box 218, Hawthorn, Victoria, Australia

    John Grundy

Authors
  1. Norsaremah Salleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emilia Mendes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John Grundy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Norsaremah Salleh.

Additional information

Communicated by: Filippo Lanubile

Appendix A PP questionnaire

Appendix A PP questionnaire

figure a

Rights and permissions

Reprints and permissions

About this article

Cite this article

Salleh, N., Mendes, E. & Grundy, J. Investigating the effects of personality traits on pair programming in a higher education setting through a family of experiments. Empir Software Eng 19, 714–752 (2014). https://doi.org/10.1007/s10664-012-9238-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-012-9238-4

Keywords

Search

Navigation