Performance appraisal of software testers
Introduction
The reliability of delivered software, to a large extent, depends on the performance of software testers. An accurate performance appraisal of software testers is thus very important for their recruitment, monitoring and development, and for testing team performance management. Furthermore, from a research perspective, to conduct studies of factors that potentially affect software testers’ performance, a validated, reliable instrument to assess software testers’ performance is an essential prerequisite. For example, in a research study [1] investigating the influence of personality on the effectiveness of software testing the authors needed to distinguish different levels of performance. To accomplish this, a method to assess high performing software testers was necessary.
However, from an extensive search of relevant literature we did not find any widely accepted and well established performance appraisal method for software testers. Therefore, as an operational need for our research, we have proposed a new Performance Appraisal Form (PAF) for software testers. However, any such instrument should be validated for use.
In this study, we sought a broader insight into industrial practice in the area of tester performance appraisal by surveying nearly 20 software development project managers to describe the practices in their own organizations, and their own views on tester performance appraisal. We then attempted to validate the approach taken in our PAF by collecting feedback from software development project managers on the proposed PAF, obtaining detailed feedback from 10 of them. With this two-pronged approach, we not only sought direct comment on our proposed PAF, we hoped to find out whether industrial practice could further inform our PAF design, and also whether the PAF proposed could be of industrial as well as research interest.
The rest of the article is arranged as follows – Section 2 summarizes our review of relevant literature, Section 3 details our research questions, Section 4 describes the proposed Performance Appraisal Form (PAF), Section 5 describes the method of this research study, Section 6 presents our results, Section 7 lists the threats to validity of the research, Section 8 presents our discussion on the findings and finally Section 9 concludes the article.
Section snippets
Performance evaluation of software testers
As reported in our earlier research [2], there is no widely accepted instrument or approach to evaluating the performance of software testers in the academic literature. However, we found some suggestions about criteria that may be important for evaluating software testers’ performance. Fenton and Pfleeger [3] suggest measuring efficiency of software testing using the number of bugs found per KLOC. Grady and Caswell [4] suggest looking for average reported bugs per working day. However, Kaner
Research questions
Current methods utilized to appraise software testers in the software industry have not been reported or evaluated in any detail in the open literature [2]. We therefore do not know what are the relative advantages and disadvantages of current performance appraisal practices for software testers and how we can improve on these.
Based on our review of the literature and analysis of different requirements listed in job advertisements for testers, we designed a new Performance Appraisal Form (PAF)
Proposed Performance Appraisal Form (PAF)
The objective of our proposed PAF is to provide a standard assessment instrument to assess overall performance of software testers from different performance dimensions. Some performance appraisal instruments use multiple forms to assess different aspects of employee performance. However, for simplicity we chose to design an integrated form. The performance dimensions of our proposed PAF were based on different approaches [13] to performance appraisal: Performer focused appraisal: This approach
Methodology
A personal opinion survey [18] was used to conduct our survey. Compared to other available research methods, a survey enabled us to collect opinions of higher number of participants in limited available time [19]. The design of the survey was by following the steps as suggested by Kitchenham and Pfleeger [18], and these are presented in the following subsections.
Results
Our survey was divided into two main sections. We noticed that we obtained a different rate of participation for the two sections. Unfortunately, 22.2% of participants dropped out after completing the first section. Thus we describe the results obtained from each section separately.
A total of 18 participants (8 in the initial survey and 10 in the new lightweight survey) participated in this section.
Threats to validity
One of the threats that can limit the external validity of our reported results is over-generalization of the findings. We believe that the nature of participation required for this study put off many potential participants and as such we obtained a limited number of participants. Due to the small number of participants we had to modify the survey to require less time and yet still the participation rate was not satisfactory. We found that many participants were less interested in the second
Discussion
From the responses to the survey questionnaire of the state of practice of performance appraisal, it is evident that a formal process of employee appraisal is practiced in the majority of organizations. A specialized appraisal process is also common for software testers. However, when there is no such specialized process, performance of software testers is most commonly appraised using a more general employee appraisal process or a manager’s evaluation. In only a small number of organizations
Conclusion
This study aimed to obtain information about the state of practice of performance appraisal of software testers and to make suggestions on how the appraisal process can best be conducted. We found that there are two trends: some organizations use the same performance appraisal process for all employees, whereas some use a specialized one for software testers. Our participants suggested a number of criteria that should be considered in appraising performance of software testers. Among those
References (21)
- T. Kanij, R. Merkel, J. Grundy, An empirical study of the effects of personality on software testing, in: Accepted to...
- et al.
Performance assessment metrics for software testers
- et al.
Software Metrics: A Rigorous and Practical Approach
(1997) - et al.
Software Metrics: Establishing a Company-Wide Program
(1987) Don’t use bug counts to measure testers
Softw. Test. Quality Eng.
(1999)- C. Kaner, Measuring the effectiveness of software testers, in: Software Testing Analysis & Review Conference (STAR)...
- et al.
A model for motivating and measuring quality performance in information systems staff
Inform. Syst. Manage.
(2001) - et al.
Selection and evaluation of computer personnel – the research history of sig/cpr
A programmer appraisal instrument
Research studies of programmers and programming
IBM Corporation
(1964)
Cited by (13)
Recruitment, engagement and feedback in empirical software engineering studies in industrial contexts
2018, Information and Software TechnologyCitation Excerpt :Piloting a study with a small number of practitioners has been demonstrated to be valuable in previous work in case study research with industry, where they recommend structuring case studies iteratively, taking early samples and pilot testing [1]. We failed to sufficiently pilot our early personality testing surveys, resulting in low engagement [14]. In contrast, we refined our worklog, tester appraisal and software AD usage studies, tasks and instruments, and GT interview questions from early feedback from such pilot studies [12,17,21].
Dealing With Data Challenges When Delivering Data-Intensive Software Solutions
2023, IEEE Transactions on Software EngineeringMining and Fusing Productivity Metrics with Code Quality Information at Scale
2023, Proceedings - 2023 IEEE International Conference on Software Maintenance and Evolution, ICSME 2023Human Performance Management: A Humanist and Supportive Model for IT Professionals
2023, Lecture Notes in Business Information ProcessingPerformance Management of IT Professionals: A Humanistic Model
2022, Proceedings of the 17th Conference on Computer Science and Intelligence Systems, FedCSIS 2022DHuTI: A Maturity Model for Human Performance in Information Technology
2022, Iberian Conference on Information Systems and Technologies, CISTI