Pathways to becoming an internal evaluator: Perspectives from the Australian non-government sector

https://doi.org/10.1016/j.evalprogplan.2019.01.007Get rights and content

Highlights

  • Career pathways into internal evaluation are not clear.

  • Clarity around career pathways and role identification would be valued.

  • Evaluative thinking can be initiated by different catalysts.

  • The professionalisation movement could be enhanced by increasing the focus on internal evaluation.

Abstract

There is a lack of clarity around intra-organisational evaluation roles and pathways into these roles in non-government organisations (NGOs). This article presents three auto-narratives from the authors who are working as internal evaluators in the NGO sector. We examine this phenomenon of role ambiguity by exploring our evaluation journeys and struggles to find identities in the formal evaluation community. Findings from the auto-narratives identify implications for the evaluation field regarding professionalisation. This article explores how aspects of professionalisation, such as clarification of roles and tasks of internal evaluators, could facilitate their recruitment, assess credibility and guide career trajectory. Elucidating internal evaluation career pathways contributes to the evaluation discipline by providing information relevant for evaluation capacity building, evaluator training, and the professionalisation movement.

Introduction

This paper presents the authors’ experiences as internal evaluators in non-government organisations (NGOs) to discuss the career pathways of people conducting and championing evaluation, and inform the professionalisation movement. This is in response to the call for more published experiences and practical solutions to address the limited literature on internal evaluation (Sonnichsen, 2000; Volkov & Baron, 2011). There is also a shortage of literature regarding the ways in which individuals find career pathways into internal evaluation roles in organisations and a lack of recognition of employees who value evaluation and promote use and understanding among their colleagues (Rogers & Gullickson, 2018; Silliman, Crinion, & Archibald, 2016). It is important to know more about this career journey because employees with interest and expertise in evaluation can help meet NGO accountability and improvement demands in increasingly competitive funding environments (Gill, 2010; McCoy, Rose, and Connolly, 2013; Moxham, 2014). Understanding the experiences of individuals who demonstrate an interest in evaluation can enable NGOs to effectively recruit and retain and have the necessary information to access appropriate professional development. As such, this paper contributes to the professionalisation movement by broadening the scope of understanding in relation to internal evaluators in the non-government sector. While the bulk of our experiences are situated in Australia, these are applicable to a global audience as evidenced through similar themes and concerns identified in the literature (Nakaima & Sridharan, 2017).

This is the second exploratory inquiry we have conducted to share perspectives from internal evaluation employees. The first inquiry was undertaken in 2017 to examine the role of evaluation literacy in internal evaluation in the non-government sector (Rogers, Kelly & McCoy, 2019). As there are significant gaps in the literature on internal evaluation (Baron, 2011; Mayne, 2014), these papers contribute to evaluation discourse by considering aspects of internal evaluation based on the life experiences of three internal evaluators. We initiated this inquiry to demonstrate and unpack some of the issues and challenges facing emerging and early career internal evaluators.

To restate, in this paper we seek to understand how employees in NGOs become internal evaluators. The purpose of this inquiry is to elucidate some of the career pathways to becoming an internal evaluator in the NGO sector, contributing to addressing the gap in the literature on internal evaluation (Volkov & Baron, 2011). Additionally, we discuss the need to clarify when and how employees should identify themselves as evaluators. This paper does not critique the advantages and disadvantages of internal or external evaluation, nor analyse the professionalisation debate. The research question we explore in this paper is: What are some of the career pathways to becoming an internal evaluator in the NGO sector? Firstly, we briefly consider how evaluation is being used in the non-government sector, define ‘internal evaluation’ and ‘internal evaluators’, and review the literature related to internal evaluators in these organisations. Secondly, we present our three narratives as internal evaluators working in the non-government sector; examining this phenomenon through exploration of our evaluation journeys and struggles to find identities in the formal evaluation community. Thirdly, we discuss findings from the narratives to identify implications for the evaluation field regarding professionalisation. Through this process, we seek to answer Nakaima and Sridharan (2017) call for “better stories of the dynamics of organizational capacity building from specific case studies” (p. 3).

NGOs use different approaches for sourcing evaluation expertise to implement processes of inquiry, feedback, reflection, change and value judgement-making (Baron, 2011; Beere, 2005; Bourgeois, Hart, Townsend, & Gagne, 2011). There are major differences in the variety of approaches undertaken by NGOs in regards to how evaluation is incorporated; including what questions are asked, how the information is collected and how it is used (Gill, 2010). Evaluation in NGOs can be challenging. Although NGOs may potentially have lower levels of bureaucracy, they often have limited resources and constraints on time as well as all the intricacies of for-profit organisations in terms of personnel management, infrastructure, and compliance (Mayne, 2017; Rogers et al., 2019). In addition, the problems the NGO sector seeks to resolve are referred to as ‘wicked’ because of the inherent degree of complexity (Gilchrist & Butcher, 2016, p. 4). There are also high community expectations in combination with a lack of coordination across multiple levels of service delivery and between organisations (Gilchrist & Butcher, 2016).

Gill (2010) illustrates the multiple demands being placed on these organisations: “Nonprofits are being held accountable for, at the same time, solving all the ills of our society, showing measurable results, and being financially solvent” (n.p.). The changes that NGOs seek are not easily measurable or quantifiable. For organisations working in human services and social programs, effectiveness is often subjective, impacts can take a long time to realise, and causation is difficult to attribute (Bamberger, Rugh, & Mabry, 2012). Additionally, there is a common disconnect between the reporting demands of the funders, who may be focused on value for money, and the internal model of practice whereby staff predominantly focus on service delivery, improving operational functioning and accountability to beneficiaries (Campbell, Lambright, & Bronstein, 2012; King & Volkov, 2005; Naccarella et al., 2007).

Aiming towards embedded self-evaluation where evaluation is part of the organisational culture can be a strategy to assist NGOs to manage these challenging circumstances. In the preface to Wadsworth (2011) ‘Everyday evaluation on the run’ she encourages everyone who is engaged in an organisation to participate in “small-scale but essential everyday evaluative activity” (p. xvii). Wadsworth (2011) highlights the need to increase confidence and build capacity for evaluative practice in employees who do not necessarily identify as evaluation professionals. When attempting this way of working, Mayne (2017) suggests, “In a small organization this likely means that almost everyone has to become a mini-evaluator, seeking out and using evidence on results to improve service delivery to their clients. They need to see results information as an essential part of their job” (p. 14). In some cases leaders assign individuals into roles dedicated to evaluation. In other scenarios, individuals may take on evaluative responsibilities in addition to their main role, volunteers may contribute their time and skills, or there may be employees that champion evaluation among their peers even though their role is not related to evaluation (Bourgeois & Cousins, 2013; Carman, 2007; Silliman et al., 2016; Stockdill, Baizerman, & Compton, 2002; Taylor-Powell & Boyd, 2008).

A useful definition of internal evaluation in organisational contexts states that internal evaluation is “a comprehensive and context-dependent system of intraorganizational processes and resources for implementing and promoting evaluation activities” (Volkov & Baron, 2011 p. 102). Further, Baron (2011) proposes that an internal evaluator is “an employee of the organization who performs evaluation functions to any degree—whether alone or in conjunction with other duties and responsibilities" (p. 88). There is an abundance of literature on the importance of internal evaluators for aligning evaluation with context, linking evaluation with organisational learning and capacity building and advocating for change among their colleagues (Leviton, 2014; Love, 1991; Mathison, 2011; Sonnichsen, 2000). This reveals the importance of internal evaluation positions: particularly because they are in an ideal location to use evaluation to influence decision-making and organisational learning, support improvements and make judgements about programs and policies (Volkov, 2011). Baron’s (2011) definition is broad enough to encompass staff who may not formally identify as internal evaluators but who undertake evaluation as part of another position, advocate for evaluation, promote evaluation among their colleagues and practise reflection and critical thinking.

There are currently no formal mechanisms, such as credentialing or accreditation processes, for internal evaluators to formally identify with the field of evaluation in Australia or in most other nations. However, the professionalisation of evaluation globally continues to be a strong topic of discussion and contention throughout the discipline (Altschuld & Engle, 2015). In the absence of any professional recruitment criteria, managers, human resource personnel and employees may use resources on evaluator competencies, such as the Australasian Evaluation Society Evaluation Competencies, for recruitment and role clarification purposes (AES, 2013). These, and similar guidelines, can be useful when developing position descriptions for internal evaluator roles or when incorporating evaluation duties as components of positions that have a broader remit. Resources such as the Program Evaluation Standards (Yarbrough, Shulha, Hopson, & Caruthers, 2011), American Evaluation Association Guiding Principles for Evaluators (AEA, 2018), European Evaluation Society’s Evaluation Standards of National and Regional Evaluation Societies (EES, 2012), United Kingdom Evaluation Society Evaluation Capabilities Frameworks (UKES, 2012), Aotearoa New Zealand Evaluation Association Evaluation Competencies (ANZEA, 2011) and the Canadian Evaluation Society Credentialed Evaluator Designation Program (CES, 2010) provide useful information to assist organisations in defining evaluation roles in their specific context.

Although not explicitly stated, the resources listed above are sufficiently broad to include both internal and external evaluator roles. Despite this, an evaluation of the CES Credentialed Evaluator Designation Program recommended that a version of the program be specifically tailored for internal evaluators (Fierro, Galport, Hunt, Codd, & Donaldson, 2016). Their explanation included that: “in the federal context there was a suggestion that the competencies seemed more tailored to external rather than internal evaluators” (Fierro et al., 2016, p. 5). This indicates that there are some questions about the applicability of the credentialing system for internal evaluators.

Recognising diverse pathways, the ANZEA evaluation competencies acknowledge that an academic pathway into evaluation is not necessarily the primary route (Wehipeihana, Bailey, Davidson, & McKegg, 2014). Wehipeihana et al. (2014) recognise that teachers, social workers, community development and health workers from government, non-government and tribal organisations as well as individuals working in their communities with life knowledge and experience, can identify as evaluators (p. 58). ANZEA purposively developed the competencies to be inclusive of individuals engaging with evaluation from a variety of backgrounds (Wehipeihana et al., 2014). Hence, even though adaptation to context may be required, the existing resources from international evaluation societies may provide useful guidance for decision makers within organisations when considering individuals for positions with an internal evaluation focus.

The extant literature clearly demonstrates that internal evaluators are well-positioned to promote evaluation use (Love, 1991; Rogers et al., 2019; Sonnichsen, 2000; Volkov, 2011). There are resources related to evaluation guidelines and competencies from the field of evaluation that can assist with defining the roles and responsibilities of an internal evaluator as highlighted above. As described, individuals in NGOs working to promote evaluation use may engage with evaluation for a short time or, alternatively, foster and support evaluation throughout their careers as part of other roles. Some employees may even transition from a role that has some evaluation responsibilities into an internal evaluator position. As established at the beginning of this paper, understanding employees’ interactions with evaluation and how they move into internal evaluation positions, particularly in the NGO sector, is a gap in the literature. This exploratory inquiry seeks to address this gap by revealing a small sample of pathways from program delivery practitioners to internal evaluators.

Section snippets

Methods

We undertook an inquiry in 2017 to examine the role of evaluation literacy in internal evaluation in the non-government sector (Rogers et al., 2019). The methodology involved a literature review and analysis of auto-narratives. It revealed the underlying issues affecting evaluation use in NGOs and the skills needed to motivate and enable others to access, understand and use evaluation information. Following the acceptance of these findings for publication we repeated the methodology in 2018 to

Discussion

With the purpose of understanding how employees in NGOs become internal evaluators, the narratives presented above were analysed to identify common patterns in our pathways and journeys. Identification of three overarching themes resulted: catalysts for engaging with evaluation are different, development of evaluation skills and competencies does not follow a regular pattern, and identifying as an internal evaluator can create confusion and anxiety.

Conclusion

While there are many possible pathways to becoming an internal evaluator, clarification of the role could assist with guiding the career trajectory of employees into internal evaluation. This article has found that evaluative thinking can be initiated by many different catalysts but that career pathways leading to self-identification as an internal evaluator are not clear. Whilst the diversity of entry points is to be celebrated for adding a variety of rich perspectives to the evaluation

Declarations of interest

None.

References (47)

  • I. Bourgeois et al.

    Using hybrid models to support the development of organizational evaluation capacity: A case narrative

    Evaluation and Program Planning

    (2011)
  • L. Naccarella et al.

    Building evaluation capacity: Definitional and practical implications from an Australian case study

    Evaluation and Program Planning

    (2007)
  • AEA American Evaluation Association

    American evaluation association guiding principles for evaluators

    (2018)
  • AES Australasian Evaluation Society

    Evaluators’ professional learning competency framework

    (2013)
  • J. Altschuld et al.

    The inexorable historical press of the developing evaluation profession

    New Directions for Evaluation

    (2015)
  • ANZEA Aotearoa New Zealand Evaluation Association

    Evaluator competencies. Aotearoa New Zealand: Aotearoa New Zealand evaluation association

    (2011)
  • M. Bamberger et al.

    RealWorld evaluation: Working under budget, time, data, and political constraints

    (2012)
  • M. Baron

    Designing internal evaluation for a small organization with limited resources

    New Directions for Evaluation

    (2011)
  • D. Beere

    Evaluation capacity-building: A tale of value-adding

    Evaluation Journal of Australasia

    (2005)
  • I. Bourgeois et al.

    Understanding dimensions of organizational evaluation capacity

    The American Journal of Evaluation

    (2013)
  • V. Braun et al.

    What can “thematic analysis” offer health and wellbeing researchers?

    International Journal of Qualitative Studies on Health and Well-being

    (2014)
  • J. Buckley et al.

    Defining and teaching evaluative thinking: Insights from research on critical thinking

    The American Journal of Evaluation

    (2015)
  • D.A. Campbell et al.

    In the eyes of the beholders

    Public Performance & Management Review

    (2012)
  • J.G. Carman

    Evaluation practice among community-based organizations: Research into the reality

    The American Journal of Evaluation

    (2007)
  • CES Canadian Evaluation Society

    Competencies for Canadian evaluation practice

    (2010)
  • D.J. Clandinin

    Engaging in narrative inquiry

    (2013)
  • D.J. Clandinin et al.

    Knowledge, narrative and self-study. Chap. 15

    International handbook of self-study of teaching and teacher education practices

    (2004)
  • A. Coffey et al.

    Narratives and stories. Chap. 5

    Making sense of qualitative data: Complementary research strategies

    (1996)
  • B. Doherty et al.

    Building evaluation capacity in Micro community organisations - more burden than benefit?

    Evaluation Journal of Australasia

    (2015)
  • EES European Evaluation Society

    Evaluation standards of national and regional evaluation societies

    (2012)
  • L.A. Fierro et al.

    Canadian evaluation society credentialed evaluator designation program - evaluation report

    (2016)
  • D. Gilchrist et al.

    Introduction. Chap.1

  • S.J. Gill

    Developing a learning culture in nonprofit organizations

    (2010)
  • Cited by (5)

    View full text