The NYC Department of Education’s Learning Environment Surveys are a rich source of data about New Yorkers’ experiences in their public schools. Upwards of 450,000 parents, 425,000 secondary students, and 55,000 teachers completed the surveys in 2010-11—in district and charter schools. Although survey scores only count for 10 of 115 possible points on the Progress Reports, they count for a lot more when they direct our attention to bright spots, areas to improve, and opportunities to learn and share.
In that spirit, we took a look at how survey responses varied for charter and district schools, and where those differences were statistically significant. Below are the questions where the charter-district differences were widest, grouped by topic. The responses are grouped by the school’s grade range.
(Note that most district schools in the K-3 category have grown to their planned size, while for charter schools K-3 schools are typically still adding grades to reach K-5 or K-8. More detailed methodology and sample size notes are at the end of this post.)
Looking at the statistically significant differences, there is some good news for charter schools:
- Charter parents tend to respond more positively. Compared to district parents, charter parents schools score their schools better across the board in educational quality (topic 1), communication (2), respect and discipline (3), and engagement (4).
- Charter middle schools outpace district middle schools in almost all survey areas. In addition to the areas where all charter schools outpace their district counterparts, charter middle schools do well in relationships between teachers and administration (topic 5), professional development opportunities (6), general school safety (7), and disciplinary environments as viewed by teachers (3).
- In secondary grade levels, charter students report doing more group discussions, projects, or essays than their district peers. At other grade levels, there are no statistical differences seen between charter and district responses (topic 9).
- There are few perceived differences in inclusion of students with disabilities. Some differences exist in the early childhood and the secondary grade levels (topic 11).
On the other hand:
- Secondary students in charter K-8s are much less satisfied with their teachers than are their district K-8 peers. Secondary charter students in schools with other grade configurations are comparatively much more satisfied (topic 8).
- Teachers at K-3 (i.e. recently opened) elementary charter schools feel that their school environments are much more unsafe than do teachers in K-3 district schools. Parent perceptions about bullying follow the same pattern (topic 7).
- Charter parents hold School Safety Agents in comparatively high regard, but charter teachers strongly disagree. The opinions of charter secondary students do not differ in any way from their peers (topic 10).
These patterns only invite more questions, of course, and there always may be other differences at work. (I have not controlled for school enrollment size or length of the school day, for example.) Still, I hope this view of the data is an interesting starting point. When the voices of almost half of the parents, teachers, and students in the city are available for us to hear, it is wise for us to listen.
To explore specific responses to questions, use the tool below. Note that it does not indicate which differences are statistically significant.
Methodological Notes
The survey data are available for batch download. Differences refer to “question scores” according to the NYC Dept. of Education’s weights: Strongly Disagree = 0, Disagree = 3.33, Agree = 6.67, Strongly Agree = 10, or some variation thereof as explained in the DOE Scoring Model for Community Schools. For those and other technical details, visit the DOE’s survey page.
Schools were divided into grade level classifications based on the responses to parent question 1 (“What grade is your child in?”) to determine whether the school served and appropriately surveyed a given grade. K-3 included schools serving K-1 or K-2. Elementary incudes schools serving up to grade 5. Middle school was defined as grades 5-8 or grades 6-8. High school was defined as serving some subset of grades 9-12. 6-12 was defined as a school overlapping between the middle school and high school grades.
Sample sizes were as follows:
| K-3 | Elem | K-8 | Mid | 6-12 | High | |
| Charter Schools | 43 | 19 | 27 | 21 | 7 | 8 |
| District Schools | 50 | 545 | 169 | 277 | 77 | 399 |
For each set of question responses, we created contingency tables with the four option categorical response data (Strongly Agree, Agree, Disagree, Strongly Disagree, or variation,) and flags for Charter schools or District schools, aggregated across all responses. Then for each question, we performed a chi-squared analysis with 3 degrees of freedom to determine if the response distributions were different at p<.05. While this can determine if distributions are different, in order to determine if one distribution was significantly “higher” based on the DOE’s methodology of converting responses into overall question scores, we had to also do a student’s t-test on difference in mean overall score, using question by question variances weighted by school size. When both of these tests proved to indicate statistical significance at p<.05, we were able to place “+” or “-“ signs indicating the direction of this difference.
