What the Test Can't Tell You

Article

About this time last month, my family and I found refuge from holiday shopping at a quiet bar and grill. It was nice to get out of the busy stores. Discussing our personal tolerances for crowds during this break from the fray, my wife shared that she was surprised I was an introvert, considering I was in education. (She is also an educator.)

"Yep, I am an ISTP," I shared.

"I don't believe that," she responded.

Not to be proven wrong, I broke one of my own rules: I pulled out my smartphone at the table. A quick search led me to a brief online Myers-Briggs test. After answering almost 100 questions, the results came back: ISTP.

"Introverted, Sensing, Thinking, Perceiving...there you go," I held the displayed results on my smartphone screen for everyone to see. But which assessment - a personality test or my wife’s observations - was more accurate?

If you look at the recommended jobs for someone described as an ISTP, you will find:

• Airline pilot

• Mechanic

• Carpenter

• Engineer

None of these careers interested me a lot growing up. Conversely, my wife's evaluation of my role as a school leader is an accurate read of my abilities and interests. If I’d taken the Myers-Brigg in high school instead of late in my college career, would I have put too much stock into the results and allowed them to influence my future?

Tests have their place in our world. They can provide some useful information for making a decision. But they rarely tell us much about who a person really is or how their interests might relate to their future work. And at an impressionable age and without thoughtful educators to put assessments in perspective for students, test results can also communicate to students inaccurate information about themselves.

A True Test

In the Phi Delta Kappan article “A True Test”, Grant Wiggins wrote about the embedded inaccuracies in the quantified results of standardized tests, especially in comparsion with what matters more in life.

We typically learn too much about a student's short-term recall and too little about what is most important: a student's habits of mind.

If all a student sees are numbers and levels as representative of their capacities as learners, how will they view themselves? Wiggins recommended designing tests that recreate the conditions and realistic constraints that allow for students to apply new knowledge, skills and dispositions. Students should experience authentic identities that exist in society, and not be defined by lesser measures.

All tests should involve students in the actual challenges, standards, and habits needed for success in the academic disciplines or in the workplace: conducting original research, analyzing the research of others in the service of one's research, arguing critically, and synthesizing divergent viewpoints. Within reasonable and reachable limits, a real test replicates the authentic intellectual challenges facing a person in the field. (Such tests are usually also the most engaging.)

To be fair, standardized tests cropped up when school districts failed to invest in effective professional learning or time for teachers to develop authentic assessments. They could have created better tests and learned how to analyze the results to determine if students learned.

Three Steps for a Better Test

Where to go from here? How can we prepare instruction in which we are teaching to the test because the test is the best way to assess if students have truly learned? Consider the following three steps for an overview of an effective process.

• First, look at what current professions and careers are asking for and expecting of individuals day-to-day. Consider starting here, an Educational Leadership article by Jay McTighe. Yes, I know some jobs available to students in the future are not invented yet. But many that exist today will exist tomorrow. Once qualities and skills are identified, develop rich tasks that embody the tenets of these professions along with the standards that we are expected to address.

As an example, when I taught curriculum design to prospective administrators, one technology education teacher created an authentic task around building a home. "As a carpenter..." began this authentic assessment and went on with a full description of the task.

• Second, build in opportunities for student reflection within a unit of study. Have the students come back to the big ideas and essential questions in context of the deep work they are engaged in. They can write responses weekly, and then analyze how their thinking has change over the course of their learning.

In the previous example, this technology education teacher had developed a seemingly simple essential question: “What skills do you need to be a successful carpenter?” But as they progress through the unit, likely the students will list character traits in addition to being able to measure precisely and to use certain tools effectively.

• Third, ensure criteria for success are clear for students. This goes beyond a rubric; examples along with discussion around exemplary work are helpful. Wiggins notes that students need to know how their performance will be judged in order for the task to be a true test. “Evaluation is most accurate and equitable when it entails human judgment and dialogue, so that the person tested can ask for clarification of questions and explain his or her answers.”

This gets to the heart of our work: developing real understanding that will transfer beyond the school walls. The construction unit described previously has the advantage of a clear outcome for the physical work. But what about the habits of mind that are also necessary? A possibility: The technology teacher could capture cooperative work on video, ask students to watch themselves in action, and then self-assess their skills.

The primary purpose of a test is not to measure whether students learned what you covered. Rather, it is to determine what knowledge, skills, and dispositions were developed and uncovered as a result of a unit of study.

For students to be engaged in our instruction, we have to provide voice and choice in our classrooms. They need to develop a personal understanding of themselves as well as of the bigger ideas from the curriculum. Giving them ownership in the learning process is one of the best ways to personalize these experiences.

I do recall one such experience: running a business in a high school technology education course. I was nominated as construction manager. It was a memorable experience; we built squirrel feeders and sold them to community members. One of the most important things I learned was that while I enjoyed leading, I was more interested in the process of learning than construction. The room our teacher gave us to create and collaborate taught us a lot, about ourselves as much as anything.