Responding (reluctantly) to the Think Tank Review Project

Guest Blogger



2)      Second, Mr. Lee took issue with particular aspects of our methodology, some of which appeared to stem from his own misunderstanding. For example, he remarked that our use of percentile rank metrics must create “winners” and “losers.” But it is common practice to group students by percentile ranking, and it's the nature of the study that when one refers to high achievers, there are also non-high achievers.  That said, the percentile ranks used in our study were based on external norms, meaning that the numbers of high achievers need not remain static—and there is no such thing as a “zero sum game”.  In fact, the numbers of high achievers increased over time for all four of our cohorts. Additionally, we addressed the issue of measurement by calculating the likely rates of attrition due to measurement error, finding that the numbers of Descenders were still larger than expected.  We did not consider regression to the mean as a separate issue from measurement error, since the high correlation over time between the MAP assessments rendered the issue relatively trivial (see here).

3)      Third, and finally, we were perplexed by Dr. Lee's proposed “alternative” analyses, which he offered as a validity check to our study.  He followed completely different samples of students longitudinally over a different age/time span than in our study, using different assessments.  Then he uses his findings as evidence that our findings are “biased” (p. 10), simply because they were not consistent with his own.  Why would one expect two different studies using entirely different methods and measures to produce the same findings?

We designed a study to describe achievement trends among high performers and concluded that the number of students “descending” was unacceptable.  We believe these findings have serious consequences for our society, and in particular, for children and their families.  We've made no attempt to use the evidence of this study as an argument against the need for improving the prospects of low achievers, especially students from disadvantaged backgrounds.  But we do believe that all students, high achieving or not, deserve equal consideration under state and federal policy.

--Fordham and the Kingsbury Center at NWEA

More By Author

Related Articles