What does certification tell us about teacher effectiveness? Evidence from New York City

https://doi.org/10.1016/j.econedurev.2007.05.005Get rights and content

Abstract

We use six years of panel data on students and teachers to evaluate the effectiveness of recently hired teachers in the New York City public schools. On average, the initial certification status of a teacher has small impacts on student test performance. However, among those with the same experience and certification status, there are large and persistent differences in teacher effectiveness. Such evidence suggests that classroom performance during the first two years is a more reliable indicator of a teacher's future effectiveness. We also evaluate turnover among teachers by initial certification status, and the implied impact on student achievement of hiring teachers with predictably high turnover. Given modest estimates of the payoff to experience, even high turnover groups (such as Teach for America participants) would have to be only slightly more effective in each year to offset the negative effects of their high exit rates (I2, J24).

Introduction

Traditionally, federal and state governments have regulated teacher quality with ex ante certification requirements. To gain legal permission to teach, prospective teachers have been required to study full-time for one or two years in an approved education program. However, recruiting difficulties have forced many districts to hire large numbers of uncertified or alternatively certified teachers. Despite the ubiquity of alternative teacher certification (AC) programs, there is little research on the impacts on student achievement. We examine the relationship between teachers’ certification status and student achievement in New York City (NYC), using students’ test scores in math and reading in grades four through eight.

Besides having the largest enrollment in the United States, NYC is a major employer of certified, uncertified, and alternatively certified teachers. During the school years 1999–2000 to 2004–2005, New York hired more than 50,000 teachers, of which 46% were certified, 34% uncertified, and 20% AC teachers. The vast majority of AC teachers in New York are recruited through the NYC Teaching Fellows program, while the remainder comes primarily from international recruitment and the Teach for America (TFA) program (a non-profit entity that recruits and sends AC teachers to districts throughout the nation).

When studying their impacts on math achievement, we find no difference between Teaching Fellows and certified teachers or between uncertified and certified teachers. Classrooms of students assigned to internationally recruited teachers scored 0.02 standard deviations lower in math than similar classrooms assigned to certified teachers, while classrooms of students assigned to TFA corps members scored 0.02 standard deviations higher relative to certified teachers. (We measure teacher effectiveness in terms of test scores among NYC students, where test scores have been normalized by year and grade level to have a mean of zero and standard deviation of one.) In reading, students assigned to Teaching Fellows underperformed students assigned to certified teachers by 0.01 standard deviations. All of the above reflect average differences in student impacts between groups of teachers, controlling for years of teaching experience. These are the only instances in which we find that a teacher's initial certification status has statistically significant implications for student achievement.

Consistent with other studies, we also find that both certified and AC teachers’ effectiveness improves with the first few years of experience. We examine teacher turnover and its implications for student achievement. Critics of alternative certification programs argue that such programs actually harm student achievement by bringing in candidates with less commitment to teaching as a career and, as a result, have higher turnover rates (Darling-Hammond, 2007). However, while turnover was indeed high among TFA corps members—reflecting their two-year commitment—our results suggest that Teaching Fellows and traditionally certified teachers had very similar retention rates. Moreover, even the higher prevalence of novice teachers among TFA participants has only a small negative effect on student achievement—about 0.02 standard deviations of achievement in math and reading. On net, the modest negative impact of higher turnover is roughly offset by the slightly higher initial effectiveness of TFA participants.

Although initial certification status provides little predictive power, there are large differences in teacher effectiveness within all of these groups. We estimate that the average value-added among the top quartile of elementary school math teachers is 0.33 standard deviations greater than that generated by the bottom 25%—almost 10 times the magnitude of any difference associated with initial certification status! Thus, although shifting the mix of teachers with different types of certification does not appear to be a useful tool for improving student achievement, selectively retaining only the most effective teachers appears to be a much more promising strategy (Gordon, Kane, & Staiger, 2006).

There exist only a few high-quality studies of AC teachers, most notably Decker, Mayer, and Glazerman (2004), who conducted a randomized evaluation of the TFA program. They find that teachers recruited through TFA are significantly more effective than both uncertified and certified teachers at math instruction and statistically indistinguishable in reading instruction.1 Although this is an extraordinarily important study for evaluating the impact of TFA corps members in the districts and schools where they are operating, it offers few conclusions that can be generalized to other AC programs. TFA is unique among AC programs in that it is highly selective, draws from a national pool of applicants, assigns teachers to schools nationwide, and recruits individuals with an explicit two year teaching commitment. Moreover, TFA corps members are typically placed in a small number of high needs schools.

Although our study does not have the benefit of being able to randomly assign teachers to classrooms, our results allow us to compare impacts for those entering teaching from a number of different routes and for teachers working in a large number of schools. Boyd, Grossman, Lankford, Loeb, & Wyckoff (2005a), Boyd, Grossman, Lankford, Loeb, & Wyckoff (2005b) also use data from NYC to evaluate differences in teacher effectiveness by initial certification status. Our work differs from theirs in several ways. We use an additional year of testing data; we incorporate application data from the Teaching Fellow program to study how the fellows are selected; and we estimate the signal variance in teacher effectiveness within each certification group. We view the last distinction as important, since we interpret between-group differences in effectiveness in light of these within-group differences. Although there are statistically significant differences between groups, such differences are generally dwarfed by the differences in effectiveness within groups of teachers.

Section snippets

Alternative teacher certification in NYC

In the school year 1999–2000, approximately 60% of all new teachers hired by the NYC Department of Education (DOE) were uncertified. Recruiting difficulties were more severe in schools with low average achievement levels; 73% of new hires in 1999–2000 were uncertified in both elementary and high schools in the lowest deciles of pass-rates on end of year math examinations. Motivated by changes in New York State law that made certification requirements more stringent, the DOE cut hiring of

Data on students and their teachers in NYC

For students, we assembled data on demographic background, attendance, suspensions from school, test performance, eligibility for free lunch, special education and bilingual education, and a student identification number. The dataset also contains teacher identification numbers for students’ math and reading teachers, which were often the same teacher for elementary school students. We capture data on teachers using information from the DOE payroll system. This gives us information on teachers’

Estimation of teacher effectiveness

To generate estimates of teachers’ effectiveness in raising student achievement, we estimate the following regression with student-level data:Ait=βgXit+γgX¯itc+ζgX¯its+δWit+πgt+εit,where Ait represents the math or reading test score of student i in year t, Xit represents student characteristics (prior-year math and reading scores, gender, ethnicity, receipt of free/reduced price lunch, special education and ELL services, and prior-year absences and suspensions), X¯itc and X¯its are the mean

Variation in value-added within groups of teachers

The above results suggest that there are little or no differences in average value-added by initial certification status. In this section, we derive estimates of the variation in teacher effectiveness among those with the same initial certification status. First, we calculate classroom average residuals from a regression of student test scores on student, classroom, and school characteristics, as well as grade-by-year fixed effects. If the residual for teacher j is the sum of a persistent

Conclusion

State and federal efforts to regulate teacher effectiveness focus almost entirely on ex ante qualifications of prospective teachers. For example, under the federal No Child Left Behind act, states and districts are required to hire certified teachers or those enrolled in an alternative certification program. However, our results suggest that the emphasis on certification status may be misplaced. We find little difference in the average academic achievement impacts of certified, uncertified and

Acknowledgments

We are grateful to Betsey Arons, Vicki Bernstein, Bruce Groneveldt, Julie Horowitz, Dominic Mancello, Lori Mei, Cathi Schoenberger, Henry Solomon, LaVerne Srinivasan, and Harvey Weinstein at the NYC DOE, who generously assembled data and patiently answered questions. A number of others in the NYC DOE, including Robert Gordon, Joel Klein and James Liebman, provided helpful comments. Timothy Daly, Michelle Rhee, and Andrew Sokatch of The New Teacher Project and Abigail Smith and Cynthia Skinner

References (29)

  • R. Ehrenberg et al.

    Do school and teacher characteristics matter? Evidence from high school and beyond

    Economics of Education Review

    (1994)
  • Aaronson, D., Barrow, L., & Sander, W. (2003). Teachers and student achievement in the Chicago public schools. Chicago:...
  • D. Ballou

    Do public schools hire the best applicants?

    Quarterly Journal of Economics

    (1996)
  • Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2005a). How reduced barriers to entry into teaching...
  • Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2005b). How reduced barriers to entry into teaching...
  • Cavalluzzo, L. C. (2004). Is national board certification an effective signal of teacher quality? CNA Corporation...
  • Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2006). Teacher–student matching and the assessment of teacher...
  • Clotfelter, C. T., Ladd H. F., & Vigdor, J. L. (2007). How and why do teacher credentials matter for student...
  • Darling-Hammond, L. (2007). A marshall plan for teaching: What it will really take to leave no child behind. Education...
  • L. Darling-Hammond et al.

    Does teacher preparation matter? Evidence about teacher certification, Teach for America, and teacher effectiveness

    Education Policy Analysis Archives

    (2005)
  • Decker, P. T., Mayer, D. P., & Glazerman, S. (2004). The effects of teach for America on students: Findings from a...
  • R. Ferguson et al.

    How and why money matters: An analysis of Alabama Schools

  • Figlio, D. N. (2005). Testing, crime and punishment. National Bureau of Economic Research working paper 11194,...
  • Figlio, D. N. & Winicki, J. (2002). Food for thought: The effects of school accountability plans on school nutrition....
  • Cited by (0)

    View full text