Testing… 1, 2, 3… testing. This is a test. This is only a test.

Nothing like your back going out to make you feel old. Happens to me every few years, and comes out of the blue. Last time it was when I was brushing my teeth. This time it was tying my shoes. Anyway, after a hospital visit I was told to take a couple of days off, which in turn gave me a chance to catch up on the blog discussions on mathalicious and dy/dan that I’d been too busy to read in their entirety.

I’ll go into those blogs in detail below, but first let me summarize one theme that emerged for me in my pain-killer induced haze. For some reason, a song by Michael Franti’s old band—Disposable Heroes of Hiphoprisy—came to mind…

Television, the drug of a nation.
Breeding ignorance and feeding radiation.

… except the lyrics morphed into…

Testing, the drug of a nation.
Breeding ignorance, spoon-feeding information.

No matter how much that song brings me back to my twenties, when I was a budding new teacher like Karim and Dan, my back spasms are painfully compelling counterexamples to my theory that I’m still young. Alas! So anyway, let me take you on my blog trek…

I’d recently met Karim Ani, the founder of Mathalicious, at the Why Algebra Matters and How Technology Can Help conference, when we bonded over our mutual frustration with all the excitement about Khan. In his blog post “Khan Academy: It’s Different This Time,” he argues that this approach is exactly the same as the ineffective approach of the past: “rote procedure; multiple choice and fill-in-the-blank questions; lack of conceptual understanding.” A completely modern version of the same ineffective pedagogy, which is fine as it was—a free homework help site—but not when Gates and others bestow it with money and prestige.

My favorite part of Karim’s blog is his response to the notion that “this time it’s different” because “technology has finally gotten to a point on the exponential growth curve where we can create ‘adaptive learning’ systems,” while at the same time “schools and districts are finally under such budgetary pressure.”

After all, industrial farmers have made incredible leaps in bioengineering, while thousands of families across the country struggle to make ends meet. Still, it doesn’t follow that McDonald’s is therefore the solution to the hunger crisis.

Dan Meyer’s blog post, What Silicon Valley Gets Wrong About Math Education Again And Again, makes the point that “the medium defines, changes, and distorts the message,” and that when you use various media to distribute mathematics “each of those delivery media changes the definition of mathematics.” Dan argues that ed-tech start-ups often change the definition of mathematics so that it becomes “a series of simple, machine-readable tasks you watch someone else explain and then perform yourself,” while others like the Khan Academy delegate to teachers those tasks “for which teachers are a good medium: conversation, dialogue, reasoning, and open questions.” Ultimately, Dan argues, “that delegation only works to the extent that teachers and computers convey complementary definitions of mathematics,” when in fact their definitions are often contradictory.

Both these blog posts received a large number of comments (91 for Karim, 83 for Dan). Interestingly, Dan linked to a research paper with the phrase that “computers are not a natural working medium for mathematics” which generated many comments that specifically referred to Sketchpad (see comments #37, #48, #57, #60, #71).

While not as obvious as with composition, I think you undersell computers. For example, many of my geometry students see Geometer’s Sketchpad as a natural working environment for working with graphs and compass constructions, creating graphics, and even writing short mathematical arguments. They even use it in other classes of their own accord.

Another example, which also references the functionality of Fathom and TinkerPlots:

Dynamic geometry, as has been mentioned by some already. This tool has done amazing things for helping teachers to create opportunitites for students to make discoveries on their own and thus enage with mathematics. It can go beyond the teaching of geometry as well.

Graphing software – largely by labour saving, but also through dynamic functionality – these tools as well have created new opportunities for exploring relationships.

Data Handing – This has come to life through computers with access to real, live data, the functionality to collect it and the ability to process it. All this means that the nature of data handling tasks can now vary in new ways.

The paper that Dan linked to, “Computer-based assessment: a platform for better tests?” by Hugh Burkhardt and Daniel Pead of the Mathematics Assessment Resource Service (MARS), is a dense read with many salient points. It references interactive assessments—potential prototypes that might measure how students perform on more complex, open-ended tasks. These applets are super interesting to experience yourself. In particular I recommend exploring Sunflower, as well as Speed Limit and Make 100. [Update: the website hosting these applets has been taken down.]

The authors start off with “formal assessment, whatever its goals, plays several unavoidable roles” (paraphrased by me):

  • Measuring performance against curriculum goals (the traditional goal of assessment).
  • Epitomizing the curriculum (tests communicate clearly the aspects of performance that will be rewarded, which may or may not cover the declared objectives of the intended curriculum in a balanced way).
  • Driving classroom activities (for ‘high stakes’ assessment, classroom learning activities—the implemented curriculum—will closely match the aspects of performance that appear in the test—the tested curriculum).

They argue that high stakes assessment “must be a balanced measure of what is important, not just what is easily measurable. Any balanced assessment of the goals of most intended curricula implies the assessment of performance on complex tasks involving higher-level strategic skills and substantial chains of reasoning.” The paper then describes their work with developing more complex assessments that measure the type of mathematical practices that are not assessed by multiple-choice and short-answer questions.

This is the type of thinking that we are also currently engaged in at Key. I was especially pleased by the overlap of the work we do and what the authors regard as the “main current opportunities for computer-based assessment to contribute to raising the quality of assessment,” which included simulated microworlds, data-based investigations, modeling, and other naturally computer-based tasks. Automated feedback can be great for student learning when applied in ways that are designed to make the student think and reason.

But when all is said and done, a test is only a test. Finland abandoned high stakes testing in favor of increased teacher pay and professional development, and it seems to work. Instead, the US has invested heavily in machine-scored testing. I think we can all agree that tests have a role in the classroom, for teachers to evaluate student “performance against curricular goals.” But high-stakes testing outside of the classroom is the medium that distorts the message, and it makes us flock to the sources of the same bite-sized pieces of mathematics, only now they are streaming…

Testing, the drug of a nation.
Breeding ignorance, spoon-feeding information.

One thought on “Testing… 1, 2, 3… testing. This is a test. This is only a test.”

Leave a Reply

Your email address will not be published. Required fields are marked *