Reading school report cards for all they’re worth

If you want to know how Milwaukee schools are doing, your first stop might be to pull up the school report cards compiled and released by the Wisconsin Department of Public Instruction. 

The DPI’s report cards annually publish the performance of traditional public schools and districts, charter schools, and private schools participating in the parental choice program. As the bedrock of the state’s accountability system, the report cards offer a means of comparing schools across Milwaukee’s three school sectors. 

But for those outside the education realm, reading and interpreting the report cards can be confusing. In recent years, changes to the computational system underpinning them and pandemic-related interruptions to the data have further complicated matters. More than a plain reading is required to understand the value – and limitations – of the report cards. 

What do the report cards measure? 

The report cards are an aggregation of a variety of school performance measures largely drawn from students’ performance on state and federally mandated standardized tests. 

Each school and district is assigned an overall score on a scale of 0 to 100, corresponding to five rating categories that range from “Significantly Exceeds Expectations” to “Fails to Meet Expectations.” Those categories are also associated with a five-star rating system. 

The overall score is determined by the weighted average of four factors: a school’s achievement (how much students know, based on state testing); growth (how much students progressed over the course of a year, including the pace of improvement); target group outcomes (how the lowest-scoring students are performing on achievement, growth, absenteeism, attendance and graduation); and how well students are reaching certain milestones that predict whether they will graduate on time. 

Academic outcomes on the report cards are based on K-8 students’ performance on the annual Forward exam and 9-12 grade students’ performance on the ACT/ACT Aspire exams.

How far does the data go back? 

Public schools were first issued report cards in the 2011-‘12 school year and private schools with at least 20 parental choice program students were added in 2015-‘16, as required by state law. One benefit of the report cards is the creation of a consistent reporting system that has allowed for more meaningful comparison among choice, charter and traditional public schools. 

For a city with a fragmented education system, having data for all three sectors is essential for accountability, said Abby Andrietsch, president and chief executive officer of St. Augustine Preparatory Academy, a private voucher school in Milwaukee. 

“We have more clear information today than, for sure, we had 15 years ago across all three sectors,” said Andrietsch, who also founded and formerly led Milwaukee education nonprofit Schools That Can Milwaukee. “For voucher schools and charter schools to hold ourselves accountable to quality in the same way we’re asking public schools to hold themselves accountable is really important.”

Beginning in the 2019-‘20 school year, the pandemic brought significant disruptions, resulting in a pause in testing. No school report cards were produced that year. The following year saw historically low participation rates – with about 50% of students citywide taking the tests – which further obscured year-over-year comparisons. 

DPI also rolled out changes to the report cards in 2020-‘21, including alterations to the formula that determines overall scores and cosmetic changes to the layout of the cards. Notably, the overall score ranges that determine a school’s star rating were adjusted. Previously, an overall score of 63 or higher was needed to “meet expectations.” With the change, that threshold was lowered to 58. The net result: despite higher overall scores on the report cards, student proficiency rates have not actually increased over the past six years. 

All of this makes it difficult to hold up schools’ overall scores from 2021 forward against previous years’ scores. 

“Essentially, if you’re looking at report cards today, you can’t compare these report cards post-pandemic with report cards from before the pandemic,” said Spencer Schien, senior manager of data and analytics for City Forward Collective. “It’s really not a great comparison, especially if you’re looking at overall scores.” 

Why are report cards controversial?

One of the most controversial aspects of the report cards is methodological, particularly the formula that determines a school’s overall score. 

The weighting of the achievement and growth components vary, depending on the percentage of economically disadvantaged students in a school. For schools with 65% or more low-income students, growth is weighted at nine times the rate of achievement. In most Milwaukee schools, the achievement score counts for 5% of a school’s overall score, while growth counts for 45%. 

By comparison, in neighboring Whitefish Bay, where 1.4% of students are economically disadvantaged, that weighting is reversed: achievement is weighted at 45% of the overall score, while growth is weighted at 5%. 

Variable weighting attempts to recognize the more challenging work high-poverty districts – where students are much more likely to begin the school year behind grade level – have in making up large achievement gaps among students within a year’s time, said Colleston Morgan Jr., vice president of strategy and policy for City Forward Collective. 

“Both things matter. I’m a parent; I care about how much my children learn in a year, but I also care that my kids are on grade level,” Morgan said. “I think the intention was a good one, to make sure we were both recognizing that students are starting at different places and schools have different levels of challenge in, based on that starting point, in bringing students up to grade level.” 

Compared to neighboring states, Wisconsin is an outlier in how heavily growth is weighted versus proficiency among low-income schools and districts. 

Highlighting the disconnect, a 2022 City Forward Collective report found six out of 10 Milwaukee students are enrolled in schools that meet expectations, though fewer than one in five are able to read, write and do math on grade. 

“Because it’s so high, we ended up in a place where you can have schools that have done good work – and it’s well recognized in the growth measure – but the vast majority of students are not meeting grade level expectations in math or in ELA (English Language Arts). And yet those schools are receiving ratings of ‘meeting expectations,’ despite the fact that maybe only a handful of students in the school are reading and writing and doing math at grade level,” Morgan said. 

Some have called for a recalibration of the growth/proficiency weighting. The Metropolitan Milwaukee Association of Commerce and City Forward Collective have each put forth policy recommendations this year seeking a more accurate picture of student outcomes, advocating for a better balance of the report card’s achievement and growth components.

“We have, at this point, almost a decade’s worth of data from Wisconsin, and we have examples from other states to be able to better calibrate the weighting of those two measures in a way that can both recognize the importance of proficiency and growth,” said Morgan. 

What report cards don’t show

Neither purely quantitative nor purely qualitative assessments fully capture the picture of a school’s performance, education leaders argue. Test scores offer a snapshot of how much students are learning but can’t address questions of, for example, how safe they feel in the classroom. Subjective testimonials, on the other hand, can’t answer how many students are at grade level for reading and writing. 

As founding board chair of the National Alliance for Public Charter Schools in 2005, education reform advocate Howard Fuller was on the ground floor of conversations about tying test scores to school performance and accountability for the growing school choice movement. 

“What happened was … test scores became the dominant and consummate way to understand the value of a school. And I believe that was a mistake,” he said. “Because we know we are in a situation where people look at test scores, but they don’t necessarily look at value added.” 

As founder of Dr. Howard Fuller Collegiate Academy, a public charter school in Milwaukee in which 80% of students are economically disadvantaged, Fuller said report cards hold schools accountable for factors outside their purview. 

“For example, how do you hold schools responsible for attendance? There’s an argument that, if the school is really doing great things, kids will come. That’s not a real thing. Because if a parent wakes up and says, ‘I’m not sending my kid to school today because I want her to take care of her younger brothers,’ you have no control over that as a school, but yet you’re held responsible for attendance,” Fuller said.

With its students coming in multiple grades behind, DHFCA is focused on getting students on grade level and accepted into college. School report cards, Fuller said, aren’t able to fully capture this work. 

“If you look at our school, I don’t think our school is a high-performing school,” he said. “But I’m not trying to be a high-performing school. I’m trying to have a school that will change children’s lives, that will put them on a trajectory that gives them a chance to be socially and economically productive citizens.” 

Report cards can lend themselves to questions about a school’s outcomes, but others are also worth considering, Morgan said.

“Do I see positive interactions between teachers and families? Between teachers and students? Between principal and others?” he said. “Those types of qualitative data points matter immensely in any field, but they really matter when we’re talking about children and families and their educational journey.”