You’d think mathematics assessment—thought of as “testing” by most people—would be simple. If you want to know what a student knows, why not write (or get an expert to write) some questions about the content you want examined, give those questions to students, see if the answers are right or wrong, and add up a total score? Depending on your predilections (and how much time you have available) you might give a multiple-choice test. You might give an “open answer” or “constructed response” test in which students show their work. You could give partial credit if you wish.

This version of assessment fits with most people’s experiences in school, and fits with descriptions of the National Assessment of Educational Progress (NAEP) as “the nation’s report card.” From this perspective, mathematics assessment—discovering what mathematics a person (typically, a student) knows— seems straightforward.

Would that things were so simple. As this essay and later contributions to this volume will indicate, different groups can have very different views of what “counts,” or should count, in mathematics. Assessing some aspects of mathematical thinking can be very difficult—especially if there are constraints of time or money involved, or if the tests have to have certain “psychometric” properties (discussed further in this essay) in order to make sure that the test-makers stand on legally safe ground. Different groups may want different information from tests. And, the tests themselves are not neutral instruments, in the ways that people think of thermometers as being neutral measures of the temperature of a system. In many ways, tests can have a strong impact on the very system they measure.

This essay introduces and illustrates such issues. I begin by identifying a range of “stakeholder” audiences (groups who have an interest in the quality or outcomes) for mathematics assessments, and identifying some of the conflicts among their interests. I proceed with a discussion of some of the side effects of certain kinds of large-scale testing. These include: test score inflation and the illusion of competence; curriculum deformation; the stifling of innovation; the disenfranchising of students due to linguistic or other issues; and a possible impact on drop-out rates.

My purpose is to lay out some of the landscape, so that the varied groups with a stake in assessing mathematical proficiency (nearly all of us!) can begin to understand the perspectives held by others, and the kinds of issues we need to confront in order to make mathematics assessments serve the many purposes they need to serve.