Deconstructing Mercedes

The one thing that is clear to anyone who’s attempted to read Mercedes Schneider’s blog is that she’s angry: angry at John Merrow, angry about Common Core, angry about evolution, and angry at Teach For America, along with a whole host of other things. However, she reserves her greatest fury for the Recovery School District’s effort to improve public education in New Orleans.

The reasons behind Schneider’s rage at the RSD are difficult to discern. She never taught in New Orleans schools and currently teaches high school English (paradoxically, for someone with a Ph.D in statistics) in St. Tammany Parish, one of the highest-performing school districts in Louisiana. Schneider isn’t a product of New Orleans schools, either; as she notes on her blog, she was educated in neighboring St. Bernard Parish.

Whatever her motives, there’s no question that Schneider has spent untold hours and tens of thousands of words to unfairly malign the RSD, and lately, her efforts have been gaining wider visibility in the world of social media. For example, Schneider has become a favorite of Diane Ravitch, who has frequently brought attention to her blog posts in recent months. It was no doubt Ravitch’s embrace that caught the attention of the Huffington Post, which has published several of her diatribes on their site.

It was on Huff Post that I came upon her most recent rant, 2013 Louisiana School Letter Grades: Recovery School District Gains Nothing, which, in keeping with her hallmark style, is a seething, widely meandering attack on the RSD. Declaring that it’s “time to once again deconstruct a lie,” Schneider attempts to debunk the latest School Performance Scores (SPS) released by the Louisiana Department of Education (LDOE) because they demonstrate that New Orleans schools continue to improve. However, as I show below, the only thing that Schneider manages to debunk is her own credibility.

Schneider’s post begins with a caustic attack on State Superintendent of Education John White, one of her favorite targets for abuse, before turning to the SPS data:

“In this 2013 spreadsheet of SPS scores, I found information useful to the public in understanding that the ‘new’ 2013 SPS and letter grades are not ‘rigorous.’ Indeed, use of the 2013 calculations makes the Recovery School District (RSD) appear to have made gains that are nothing more than artifacts of a new scale and new calculations.”

Yet in the very next paragraph, Schneider goes on to directly contradict this statement, without batting an eye:

“In the spreadsheet released on October 23, 2013, John White included two sets of SPS/letter grades: those calculated using the new 2013 formula and based on the new scale, and those calculated using last year’s formula and scale.”

So just to be clear: in one breath Schneider claims that the new grading scale is an attempt by LDOE to create the false impression that schools have improved, while in the other, she admits the department intentionally released SPS scores using both the old and new calculations so that direct comparisons could be made.

In truth, when Superintendent White announced LDOE would issue two sets of SPS scores back in March, he emphasized to members of his Superintendents’ Advisory Council, “This is a common sense proposal so that the public understands that we are changing the rules of the game.” Clearly, if LDOE wanted to deceive the public, they wouldn’t have made the effort to release two sets of SPS scores for each school, but such logic is lost on Schneider. [More information on the old formula here and a line-by-line breakdown of how the formula changed here.]

Schneider continues:

“In order to help readers easily make comparisons between last year’s calculations and the two sets of scores offered by White this year, I created this spreadsheet of SPS/ letter grades. It includes information on all RSD-NO schools included in both the 2012 and 2013 LDOE releases of SPS/letter grades.”

Actually, Schneider’s spreadsheet inadvertently exposes the slapdash nature of her so-called “research” since it includes Kenilworth Science and Technology Charter School, which is located in East Baton Rouge Parish, not New Orleans. One would think that Schneider would have caught the error seeing that the school is clearly labeled “Recovery School District – LA” (as opposed to “Recovery School District – NO”) in her own spreadsheet.

Still, Schneider fervently believes that her simple spreadsheet reveals all sorts of sinister double-dealing by the RSD. For example she says, “I was able to capture some of the ‘charter churn’ hidden pretty well from all but local public eye.” “Charter churn” is one of Schneider’s oh-so-clever euphemisms (and she has many) for LDOE/RSD’s policy of closing low-performing charter schools. From the very beginning, the LDOE and RSD’s message has been clear: charter operators that fail to improve low-performing schools will lose their charters and those schools will be handed over to a new charter management organization. Furthermore, this isn’t a message that’s been hidden from a national audience. Former State Superintendent Paul Pastorek couldn’t have been more blunt about this policy when he spoke with Paul Tough for a 2008 New York Times Magazine cover story on New Orleans’ public school transformation:

“Over the long haul, the R.S.D. becomes an instrument that evaluates existing schools, supports existing schools, recommends the closure of schools and recommends the best operator to come in and take over, or the best operator to come in in place of that school. We put people in business, and we take people out of business.”

Note to Schneider: when public officials have something to hide, they usually don’t go on record about it in the New York Times. This is just one of many instances when education officials have stated loud-and-clear that failing charters will be (and have been) closed.

Schneider then goes on to make the following claim:

“The churn is evident where data is available for only 2012 but not 2013 and vice-versa (true of 14 entries). It is also evident in cases where schools have the two-year-allowed “T” (transition) in place of a letter grade. (In the case of one school, Fannie Williams, one sees that there is a “D” for 2012 then a “T” for 2013 — evidence that Fannie Williams has been handed over from one charter operator to another just this year.)”

Again, in her desire to excoriate LDOE & RSD, Schneider gets her facts wrong – or in this case, just makes them up. To start, Fannie C. Williams has not changed charter operators; it is still managed by the same non-profit board – Community Leaders Advocating Student Success – that was granted a charter to run the school in December 2010. In addition, Fannie C. Williams should have received a “T” in 2011-12, not a “D” as she claims. Although it’s unclear why the school received a “T” also in 2013, it definitely wasn’t due to a change in operators; Schneider just pulls that explanation out of thin air and presents it as fact.

Yet even when Schneider has the facts in front of her, there’s no guarantee that her readers are going to get the truth, as she demonstrates in her year-to-year comparison of performance:

“On my spreadsheet, I first list the 2012 SPS/letter grades that established RSD-NO as a “D” district in 2012. (See the 2012 SPS here.) Then I list the 2013 scores calculated if the formula and scale had been kept the same in 2013 as it was in 2012. This is the only meaningful way to compare SPS and letter grades from one year to the next. To change formulas and scales is to obliterate any meaningful comparison — exactly what reformers like White desire. Finally, I include White’s “new” 2013 SPS and letter grades — which are often higher than those of the former, 2012, calculation.”

Again, Schneider didn’t have to list the 2013 scores calculated as if the formula and scale had been kept the same in 2013 as it was in 2012; the LDOE already did this for her when they released the scores. Second, LDOE intentionally released two sets of scores (based the old and new calculations) to allow people to make comparisons, not to “obliterate” them.

But hold on, it gets better (or worse, if you’re Mercedes). Schneider, who goes out of her way to remind everyone that she has a Ph.D in statistics, makes a huge error that further exposes the sloppiness of her work. A closer look at Schneider’s spreadsheet comparing SPS scores from 2012 and 2013 shows that instead of making a meaningful comparison, she’s actually comparing apples with oranges.

Bear with me for a moment while I explain: The old SPS formula actually produced two calculations: a Growth School Performance Score and a Baseline School Performance Score:

• Growth SPS represents a school’s actual performance during a single academic year;
• Baseline SPS is calculated based on a school’s performance over a two-year period.

Therefore, in order to accurately compare performance between the two years, one should look at the Growth SPS scores from 2012 and 2013. Instead, Schneider compares the Baseline SPS from 2012 and the Growth SPS from 2013 – in short, she bases her entire analysis on two totally different calculations, as shown below.

This is the kind of error that one would expect from a freshman in Stats 101, not from someone with a doctorate in the field. Furthermore, one would think that Schneider would triple-check her work given the fact that she is essentially accusing LDOE and RSD of lying on the Huffington Post, one of the most popular online news platforms in the country. The fact that she is willing to make such brazen accusations without ensuring that her supporting evidence is valid speaks volumes about her lack of credibility.

At one point in her piece, Schneider speaks of “competent psychometricians and statisticians” who “are allowed to calibrate formulas before ever considering applying them to high-stakes situations.” Well, let the record show that we can’t count Mercedes Schneider as one of those “competent statisticians” and we certainly can’t trust her judgment on the performance of the RSD in New Orleans.

At the end of the day, there is really no question that public schools in New Orleans have vastly improved since the RSD took over nearly the entire district in 2005. That’s not to say that our city’s schools still don’t have a long way to go and it doesn’t mean that the RSD’s efforts have been perfect. Nevertheless, Schneider refuses to concede that New Orleans schools have made any progress in the past eight years:

“I realize that John White, and Caroline Roemer Shirley, and Leslie Jacobs will broadcast the 2013 RSD-NO results as Proof That the Reforms Are Working. However, the truth is in the spreadsheet I composed as part of this post. The ‘reforms’ aren’t working. They require the choreographed release of a carefully guarded shaping of limited information in order to produce the veneer of success. This post is my bullseye aim at shattering that veneer.”

In short, Schneider believes that the RSD has managed to pull off the most successful, longest-running government conspiracy in American history. She believes the respected academics and research institutions who have documented the improvements in our public schools are either liars or dupes. She believes that the parents of New Orleans public school students who claim the system has improved since Hurricane Katrina are suffering from mass delusion. The only thing more warped than Mercedes Schneider’s view of the world is the fact that the Huffington Post would publish such rubbish. I guess Diane Ravitch’s endorsement gives Schneider’s lies a veneer of validity. Consider this my bullseye aim at shattering that veneer.