Blog

Danny Hayman

Nerd Alert: I’m the Charter Center’s Policy Data Analyst, and this post will assume some familiarity with New York City’s system of school Progress Reports. The official DOE documentation is here.

In an excellent new report commissioned by New Visions for Public Schools, Sean P. Corcoran and Grace Pai examine NYC’s high school progress reports and point out some distortions in these (generally valuable) tools for comparing school quality. (At the Charter Center we recently critiqued the reports’ transparency, and our CEO James Merriman has called for a redesign of their letter grades.)

Among other findings, the report highlights how a group of metrics called “weighted regents pass rates,” which collectively accounts for 25 of the 114 total points, corrects for students’ incoming proficiencies by limiting the points a school can gain for students who had high test scores in grade 8.* As a result, “schools with high-achieving students may be penalized for failing to achieve mathematically impossible growth targets.”

There are other special cases that Corcoran and Pai’s paper do not deal with where the unfairness is even worse: a high-performing school that serves middle as well as high school grades.

Consider Harlem Village Academy Charter School (HVA), which serves grades 5-12. Even though the proper year for looking at ‘incoming’ proficiency would be grade 4, HVA’s high school progress report metrics are still based on where students were in grade 8. Unfortunately for HVA, its students do well in grade 8: in 2011-12, 92% of HVA 8th graders were already passing Regents exams and receiving high school credit. Because of this success, HVA can only earn a low number of points on the “weighted regents” metric.

Even when 100% of HVA test takers pass a Regents exam – as they actually did in US History and Global History in 2011-12 – the school earns less than a third of the possible Progress Report points in this category.

And that’s not all. High grade 8 proficiency also influences a school’s peer group in every metric in the high school progress report, which makes perfect sense for traditional grade 9-12 high schools. For HVA – a charter school with lottery-based admissions in grade 5 – this means being in the same peer group as Brooklyn Latin School and NEST+m, two schools with highly-selective, test-based admissions processes. The shift in peer group hurts HVA’s scores in categories like graduation rate and credit accumulation, where they are at or above the citywide median, but in the lowest third of their peer group.

How did this affect the final progress report? HVA, with passing rates north of 85% in all five graduation-required-regents exams, 67% college readiness rate, but middling graduation rates and credit accumulation, along with poor “weighted regents pass rate” scores, received an overall progress report grade of C.

Harlem Village Academy is not alone. There are 67 district schools and 19 charter schools that serve both the middle school and high school grades in New York City. Pegging the peer index to either an earlier baseline, or other factors, would stop punishing these schools in high school for doing good work in their middle school grades.

*Technical explanation: The “weighted regents pass rate” attributes a different number of ‘points’ to a student passing a regents exam depending on how well they scored on their ELA and math tests in grade 8. (Those points are different than the overall Progress Report points.) According to the methodology chart below, a student who scored in the top 10% (decile 10) on their 8th grade social studies test would only be worth 1.01 points if she goes on to pass her US History Regents exam, whereas one in the bottom 10% (decile 1) would be worth 4.68 points if she passed the same test.

1. Weighted regents pass rate table, NYC DOE High School Progress Report Methodology, values are determined based on reciprocal of probability of given decile student passing given regents exam.