SNOOKERING THE MEDIA
By George Mitchell
Give DPI (the dep-artment of Public instruction0 credit. The state’s education bureaucrats fooled the Milwaukee Journal Sentinel and some other reporters. In the process, they got a big one-day jump on a story dealing with the Milwaukee Parental Choice Program.
The back-story is interesting. It says a lot about the integrity and competence of several involved players.
For more than a month it has been known that respected, independent researchers would announce important findings on March 30 regarding their ongoing evaluation of school choice in Milwaukee. These findings would address scientifically valid comparisons of how students in the school choice program are doing vis a vis similar students in the Milwaukee Public Schools. The results arise from a rigorous process the scholars have used to provide assurance that the comparisons are valid. “Apples to apples,” as one might say.
So it was noteworthy when DPI decided to release its own test score data on March 29, a day before the rollout of the serious research. And it was not surprising, though disappointing, that the agency’s actual announcement trumpeted an invalid comparison. Specifically, it compared low-income school choice students with the overall population of MPS students. As anyone familiar with basic research methodology would know — included key people at DPI — such a comparison is meaningless. The two groups are sufficiently different as to invalidate the claim that MPS students outperformed choice students.
A reporter with elementary knowledge of research methods would have challenged DPI’s comparison. Such a reporter would have used previous work from the independent scholars to point out that a valid comparison would involve a comparison between choice students and similar MPS students. Such a reporter would have pointed out that based on that comparison there was not significant difference between choice and MPS students. Such a reporter would have been in a particularly strong position to make that challenge because she had an embargoed copy of the studies scheduled for release March 30, studies that directly refute the DPI spin.*
That did not occur. Instead, the Journal Sentinel’s March 29 edition carried a page one story with the following lead paragraph, emphasis added: “Students in Milwaukee's school choice program performed worse than or about the same as students in Milwaukee Public Schools in math and reading on the latest statewide test, according to results released Tuesday that provided the first apples-to-apples achievement comparison between public and individual voucher schools.”
This is demonstrably wrong, on two fronts. The DPI report was not an “apples-to-apples” comparison. Indeed, it was the opposite of apples-to-apples. And, it was not the “first” such comparison (the independent scholars have issued a serious of such comparisons, using rigorous research methods). In an e-mail exchange with one of two reporters on the story, the apples-to-apples claim was justified because students took the same test. That of course in insufficient. It does not in any way address the fact that the two groups of students are not comparable.
A day after the March 29 story, another page one story appeared, under the headline: “Voucher testing data takes a new twist — Voucher, MPS kids on par, study finds.” (The study also found higher graduation and college attendance rates for choice students.)
A “new twist”? Really? The Journal Sentinel had embargoed copies of the “new twist” findings for several days before its publication of the March 29 story. It knew full well that the March 29 comparison was not one that would be supported by responsible scholarship. (It is highly likely, by the way, that DPI also had an early look at the independent findings released March 30.)
The obvious choice for the Journal Sentinel newsroom was to not play DPI’s game. The paper should have told DPI it was not going to publicize an invalid comparison when it knew that a more rigorous study was about to be released. It should have asked DPI to demonstrate why the agency’s comparison was valid. The paper should have gone to the independent scholars for comment on the DPI comparison.
It did none of those things.
A disappointing performance by DPI. Inexcusable journalism at the Journal Sentinel.
*I asked Journal Sentinel reporter Erin Richards when she received the embargoed reports from the independent scholars. She responded, “Why do you want to know?” I then asked the University of Arkansas when it provided the embargoed reports to the media and was told on or about March 24. I so informed Ms. Richards, who responded, “That’s not when the DPI released them to me.” I had not asked when DPI released the independent reports so I don’t know what she meant by that. I said I would simply explain that she declined to say when she received the reports. The key point: it is virtually certain she had them before writing the March 29 story. But for argument sake, assume she did not. Then why has she not challenged the validity of the DPI comparison in light of the independent reports?