A new article in Education Week asks whether or not it’s practical to include better open-ended questions in high-stakes standardized tests.

The answer, in short, is that there are many practical obstacles to doing so. In his article, Stephen Sawchuk mentions p3 author Todd Farley.

For some, the idea of expanding human-scored items raises issues of reliability: Performance-based items are typically less mathematically reliable than those based entirely on multiple choice.

Todd Farley, a 15-year veteran of the test-scoring business who detailed his experiences in a recent book, Making the Grades, is among the skeptics. In the book, Mr. Farley alleges that the scoring guidelines for open-ended items were frequently counterintuitive, and that as a “table leader”—an individual supervising other scorers’ work—he occasionally changed other reviewers’ scores.

Though test publishers interviewed for this story dismissed Mr. Farley’s account, independent sources do point to areas of concern. At least two reports issued by the Education Department’s office of inspector general last year, for instance, found lapses in Florida’s and Tennessee’s oversight of test contractors charged with scoring open-ended items.

If I were angling for a piece of the $4.35 billion Race to the Top Fund, I would dismiss Todd’s account, too.