Cross-country evidence on teacher performance pay

Abstract

The general-equilibrium effects of performance-related teacher pay include long-term incentive and teacher-sorting mechanisms that usually elude experimental studies but are captured in cross-country comparisons. Combining country-level performance-pay measures with rich PISA-2003 international achievement micro data, this paper estimates student-level international education production functions. The use of teacher salary adjustments for outstanding performance is significantly associated with math, science, and reading achievement across countries. Scores in countries with performance-related pay are about one quarter standard deviations higher. Results avoid bias from within-country selection and are robust to continental fixed effects and to controlling for non-performance-based forms of teacher salary adjustments.

Introduction

Educational reform discussions around the globe regularly include the idea of paying teachers based on how they perform in advancing their students’ educational achievement. From a theoretical viewpoint, performance-related pay may elicit both incentive effects – raising motivation and effort of current teachers who strive to increase their pay – and sorting effects – attracting and retaining graduates in the teaching profession who expect to do well under performance-based compensation schemes.¹ Given recent evidence that teacher quality is very important for student achievement but unrelated to most observable teacher characteristics,² motivating teachers to perform well and attracting a pool of high-performing teacher applicants are promising policies to improve student outcomes. This paper provides international evidence whether variation in the use of performance-related pay for teachers is associated with student achievement across countries.

The best available evidence on the effects of teacher performance pay stems from a set of recent experimental studies performed in Israel, Kenya, and India.³ However, there are obvious limits to what we can learn from such experimental studies. Because of their short-term character, experimental studies are bound to miss any sorting effect into the teaching profession, which is “perhaps the most important aspect of compensation effects” (Lazear, 2003, p. 186). In addition, while experimental studies cleanly identify the short-term incentive effect for current teachers, long-term incentive effects may differ. On the one hand, time required to accustom oneself to incentives contracts may render the long-term incentive effect larger than the identified short-term incentive effect. On the other hand, adjustments to the detriment of non-incentivized teaching tasks⁴ as well as regular experience of failure may render it smaller. Furthermore, it is unclear to what extent the available experimental evidence generalizes to developed countries. The available non-experimental evidence for developed countries⁵ has the additional interpretation problem that better schools may choose to adopt teacher performance pay and that teachers and students may select themselves into schools on the basis of whether there is performance-related pay or not, possibly introducing substantial selection bias.

One way of capturing long-term effects of teacher performance pay, including general-equilibrium effects of attracting different pools of teachers, while at the same time getting around within-system selection bias, is to draw on cross-country variation to compare countries that do and do not use teacher performance pay. A number of countries have extensive experience with performance-related adjustments of teacher salaries, whereas other countries do not.⁶ We combine country-level indicators of the use of performance-related pay with the micro database of the 2003 Programme for International Student Assessment (PISA) to estimate international education production functions that account for a rich set of family, school, and institutional determinants of student achievement at the individual, school, and country levels. Based on data for 190,000 students who form a representative sample of 15-year-olds in 28 OECD countries, the analysis provides descriptive evidence on the association between the use of salary adjustments for teacher performance (at the national level) and student achievement on an internationally comparable test. While the list of topics analyzed using international education production functions is long – including the effects of family background, resources, teacher salary levels, accountability, school autonomy, choice, tracking, and more⁷ – to our knowledge this is the first cross-country econometric study to address the topic of teacher performance pay.

We find that students in countries that adjust teacher salaries for outstanding performance in teaching perform about 25% of a standard deviation higher on the international math test than students in countries without teacher performance pay, after controlling extensively for student, school, and country measures. Similar associations are found for reading achievement, and somewhat smaller, but still substantial and significant associations for science achievement. While there are only limited signs of effect heterogeneity overall, there is some indication that the association is stronger where parents have higher socioeconomic background, but also for students who speak a foreign language at home. Tertiary degrees in pedagogy of teachers are positively associated with student achievement in countries without teacher performance pay, but not in countries with teacher performance pay.

Using international variation for the identification of effects of teacher performance pay has several advantages over national studies, but also clear limitations.⁸ At the most basic level, cross-country evidence can exploit institutional variation that is usually not available, or only available at a much more limited scale, within single countries. A main additional advantage is that cross-country evidence is able to capture the long-run, general-equilibrium effects of teacher performance pay that often elude studies in a single country. Another additional advantage is that by using measures of teacher performance pay aggregated at the country level, cross-country evidence circumvents the selection issues that usually plague within-country identification.

Obvious limitations of cross-country identification relate to the limited number of independent observations available at the country level and the possibility of measurement error in national reported data on the use of teacher performance pay. More fundamentally, possible issues of omitted country variables and reverse causation make identification less clean than in randomized experiments. For example, countries with low student achievement may feel pressure to introduce teacher performance incentives, which would bias the estimated effect of teacher performance pay downward. More generally, the use of teacher performance pay may be related to other unobserved country factors, such as general cultural features or the use of pay adjustment mechanisms more generally, that are themselves related to student achievement.

We implement a series of specification tests to check whether some of these remaining concerns with cross-country identification are likely to be a major source of bias in our estimates. In particular, to address issues of divergent national cultures around the world, we show that our results are robust to – and even stronger in – specifications that include continental fixed effects or restrict the analysis to European countries only. To address issues of possible correlation with the general prevalence of more flexible teacher pay schemes, the international database allows us to control for a rich set of indicators of the availability of teacher salary adjustments for other, non-performance-related criteria, such as based on teaching conditions and responsibilities, teachers’ qualifications and training, and demographics. Hardly any of these other forms of teacher salary adjustments is significantly related to student achievement, and none of them affects the qualitative result on teacher performance pay. Apart from these falsification tests, all our models control for a rich set of about 50 family, input, and institutional factors measured at the student, school, and national level that are usually unavailable in national datasets. While these analyses rule out the most obvious sources of bias in the international identification, the limits to the interpretation of results based on an international cross-section of student achievement data mentioned above require a cautious interpretation of the reported results as descriptive associations indicative of areas of possible important effects in educational production.⁹

The remainder of the paper is structured as follows. The next section introduces the international data on teacher performance pay and the micro database of the PISA international student achievement test. Section 3 describes and discusses the empirical model. Section 4 presents the results of our estimations together with robustness and specification tests. Section 5 concludes.