On Bridging Differences, an educational policy blog, Diane Ravitch writes:
Numbers don't lie, do they?Well, yes, they do. A major front-page story in The New York Times on February 6 described a major study conducted by criminologists who found that the numbers do lie. More than 100 retired, high-ranking police officers in New York City told them that intense pressure to produce improved crime statistics had led to manipulation of the data. For the past 15 years or so, the city boasted that its data system, known as CompStat, had brought about a major reduction in crime. But the survey said that the data system had encouraged supervisors and precinct commanders to relabel crimes to less serious offenses. The data mattered more than truth. Some, for example, would scout eBay and other Web sites to find values for stolen items that would reduce the complaint from a grand larceny (over $1,000 in value) to a misdemeanor. There were reports of officers who persuaded crime victims not to file a complaint or to change their accounts so that a crime's seriousness could be downgraded.
This is not only a major scandal, it is a validation once again of Campbell's Law, which holds that: "The more any quantitative social indicator is used for social decisionmaking, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
Anyone who wants to learn more about Campbell's law and how it applies to education should read Richard Rothstein's Grading Education and Daniel Koretz's Measuring Up. Or Google Rothstein's "Holding Accountability to Account," if you want to see what happens when data becomes our most important goal.
Posts
i don't guess i *have* heard of it;
ReplyDeletegood to have a handle to grab onto
when we turn the conversation
to touchy topics like "just how
compromised *are* our sacred
institutions?". somebody having
"given it a name" can make it
a little less scary to talk about.
which *is* the problem: talking
about it. everybody *knows*
where the booze *is*...
ps. sherman dorn
ReplyDelete(&, e.g. accountability frankenstein).
I don't see what you're referring to in that link...
ReplyDeleteoh, he's just done a book
ReplyDeleteand a bunch of posts
on "accountability"... the doctrine
of data-uber-alles. i look in on
his blog from time to time & he's
usually well worth reading.
i didn't intend anything too
specific; just "here's another
good source on this topic".
I think that this is an interesting point. When we think of mathematics we often think of one correct answer or one correct method. Knowing that numbers are not always telling the "truth" is just another excellent reason as to why high-stakes testing should be reconsidered.
ReplyDelete