Monday, July 21, 2008

How About a Nice Magic Trick?   [Robert VerBruggen]

Eduwonkette tells the story of New York's disappearing test-score gap. See, all gone!

Basically, authorities are claiming that racial gaps are closing, but the only evidence they point to is proficiency levels. These numbers are not up to the task of comparing gaps — if the test's difficulty changes, the gap in "proficiency" might decline, even if the actual gap in ability stays the same. There's considerable evidence that the tests have gotten easier.

Think of it this way: If a test is so easy everyone is considered proficient, there's no gap. If the test is so hard that no one is, there's no gap. In between, there is a point at which the gap is at its greatest, so by moving the test's difficulty away from this point, one can close the gap without actually teaching students better.

The only way to compare gaps between years is to use "scale scores" — basically, a statistical procedure that converts one year's scores to match another's. The schools have not made this data available, but on the NAEP test there has been no change whatsoever in this statistic.

By the way, if you don't get the title reference, see The Dark Knight already. It's everything it's cracked up to be.

UPDATE: I should note that another major point of Eduwonkette's is that the proficiency level is basically a bar, and teachers are encouraged to find kids just under that bar and push them over it. If minorities are disproportionately just a little below the bar, this will have the effect of closing the gap without having much of an effect on overall achievement. 

07/21 11:44 AMShare