Read by Example
Read by Example
Crossing the Bridge: A conversation with Terry Underwood about portfolio assessment
0:00
-35:29

Crossing the Bridge: A conversation with Terry Underwood about portfolio assessment

Podcast
Image: Veterans Memorial Bridge, Ottawa, IL (Source: Wikipedia)

In this month’s podcast, I spoke with Terry Underwood about portfolio assessment in the context of literacy instruction.

Terry is a former classroom teacher and a university professor. He is the author of Portfolio Practices: Lessons from Schools, District, and States with Sandra Murphy, and The Portfolio Project: A Study of Assessment, Instruction, and Middle School Reform. Terry writes frequently at Learning to Read, Reading to Learn, an online journal where he shares his current thinking on literacy instruction today.

The full transcript of our conversation below is made possible due to the generosity of our full subscribers. Consider becoming one today. Full subscribers have access to additional community benefits, including live conversations with literacy experts, leadership guides, and discussion threads.

Full Transcript

Matt Renwick (00:03): 

This is Matt, and I'm joined by Terry Underwood, a literacy professor and author on a couple of books on portfolio assessment. And I asked Terry to come on. He's got a great newsletter, Learning to Read, Reading to Learn. Did I say that right? 

Terry Underwood (00:23): 

That's exactly right. 

Matt Renwick (00:25): 

He's got a great newsletter. I think everyone should subscribe to it. He had an interesting comment about failure and how the culture very much can drive that, either way for better or worse. And I just said, we got to talk about this if you get some time. And also talk about your work with portfolios, which I think I get a sense are starting to come back a little bit. I wrote a book on portfolios in 2017, and it didn't sell that well, but it seems I'm getting more questions about it now than I did in 2017. I'm curious as to maybe why. Terry, do you want to share just about your books some of that context? 

Terry Underwood (01:09): 

Yeah, absolutely. Matt. Hey, thanks for asking me to talk about this. This is really up my alley, more or less. The whole notion of fear, of sharing and fear of disclosing, I think is not just a problem for professional development of teachers. It's a problem for shining a light on pedagogy generally. And the fear is real. I mean, there there's a lot of teacher bashing out in the wild, you know, in the communities. And then when you add to that the fact that we have a model of schooling that is really grounded in a factory model where you have managers and workers, it's pretty ugly. I think that that's the bottom line. 

(02:09): 

And in the worst case, you get a kind of a surveillance of teachers, which will tend to make people fearful and drive them in into the shadows a little bit. I want to pivot a little bit and go back to where I started, my interest in portfolios. I didn't know that this was happening, but I've thought about this a lot of times. The last day of school when I was in high school. You know, everybody would leave the school building. It was a big school. We had about 2200 students all together. It was a township school in Illinois, and we had a lot of kids from rural areas that were bused in together into this high school. It sat right across from the Fox River in Illinois, and there was a bridge across the river. 

(03:08): 

And we had to, when we left school, almost everybody would walk across that bridge and go down Main Street to the west side of Chicago, Ottawa, where a lot of people lived. And what would happen, people would line up on the bridge and they would open their notebooks, and they would throw all of their papers into the river. And the river looked like it was just absolutely white with paper totally covered from shore to shore. And I always had a little bit of a problem every year with throwing my stuff away. A lot of it I didn't really care. But ever since I was in fourth grade, I started saving. I started writing stories in fourth grade, and I started saving some of my stories. And it just became kind of a habit. 

(04:01): 

And even to this day, I have rubber tubs full of papers from way back. I don't even know what's back there. My garage is just stuffed with this. But really, over the years I started to appreciate the value of collecting things that were of interest to me, and that I thought might show up. I found that I could never really predict when something would be useful to me. And so if I didn't have it, I didn't have it. I couldn't go back to it, which is part of the problem, you have a tendency to hoard there. 

Matt Renwick (04:43): 

You're collecting your knowledge. You had a value on what you created and sounds like you valued it more than some of your peers. 

Terry Underwood (04:56): 

Yeah, I never really advertised it or thought much about it until a lot later when I started looking back at this stuff. And I'm really glad I kept stuff from fourth grade. I still have some of my really early writings. Maybe 10, 12 years ago, I came upon, found out that the state of Minnesota...do you know anything about eFolio Minnesota? 

Matt Renwick (05:26): 

No. 

Terry Underwood (05:27): 

The state of Minnesota decided that every child born in the state would be given a portfolio site. And they developed this, they call it eFolio, Minnesota. I think that's the name of it. But anyway, if you were born in Minnesota, when you were born, you'd get a portfolio and they would give you a place to collect your stuff, and it's free, and any anybody else could buy a site from them, I think $19.99 a year. I started using that site in my teaching at Sacramento State. And prior to that, years before that, probably the early eighties, I was teaching composition at community colleges in California. 

(06:23): 

And I started to get wind of this portfolio movement that had been catching fire at least since the 1960s. And it essentially was rooted in this idea that if we don't attend to what we've done, regardless - you could call it a failure, you could call it a success, whatever it is - but we have done it. And if we don't look at it and examine it in relation to where we are now and where we would like to be, then we're sort of adrift. And it becomes very difficult to kind of make adjustments that may not be comfortable, but you can see you're convinced to take action. And I think there's a cognitive value that you have this intention that as you move forward, you're going to do things and you're going to pay attention to what you're doing because you know there's going to come a time when you're going to look back at it. 

(07:32): 

And so that got me really interested in portfolios. And when I taught fourth grade for three years in the late 1980s, I used portfolios pretty much exclusively. I mean, those kids were saving everything, and I actually would give them prompts and have them write in response to reflective prompts. When I entered a doctoral program in 1991, I knew that I wanted to do something along these lines. I was able to get positioned to study a formal portfolio system in English language arts in a middle school where I was teaching at the time. And that became the basis for my dissertation. I think the bottom line, Matt, is for me, portfolios are a way of thinking, of connecting the surface of our behavior with the deep structures that we are building in long-term memory. 

(08:50): 

And just as writing on paper as a mnemonic, when we're reading or writing, it allows us to manipulate furniture and long-term memory and build expertise through that manipulation. Portfolios are a sort of an extension of writing. And the one difference, when you write a text and you're going to send it out into the world, there's a kind of a boundary around it, and you hope that it will hold up all by itself with portfolios. You often don't know what you don't know, what the reader is going to make of what you put in. And so there isn't that kind of unity and coherence and organization that you find in a text, but it is a form of a text, especially when it's aimed at an audience. 

(09:45): 

At some point somebody's going to look at it. I think there's a lot of sort of romanticism around portfolios where it's all like joy and celebration and that is absolutely a part of it. I've thought about the idea of portfolio carnivals and portfolio reviews, portfolio showcases where we really are celebrating, and we're looking at accomplishments and taking stock of what we've done, that’s what it's about. We want, we have to have those as human beings. But there's also portfolio ethnography. I think of portfolio ethnography as opposed to portfolio assessment, where the purpose of the ethnography is to uncover values and patterns of behavior and understand, to be able to explain what the cultural factors are that are creating the motivating sort of behavior. 

(10:55): 

So the word portfolio is nothing more than a collection of artifacts. And it takes on meaning and significance when you have that other word portfolio: carnival portfolio, review portfolio. When you add the term assessment, you really throw a monkey wrench into the whole process because at at bottom there's an antagonism between assessment and portfolio. If I'm just curating my artifacts and presenting them, what's in it for me to present them to you? So you can assess them? And a lot of times what happens when assessment gets tacked on, you really sort of disenfranchise the person who's creating the portfolio. And so that's the part that really has to be sorted out even before you begin to think about designing a portfolio. 

(12:07): 

It's one thing to do it in your own classroom because you, as the teacher, you have pretty much control over how you're going to design it, and you can really lessen the impact of assessment and use it more as a celebratory kind of a tool. But when you begin to talk about assessment, which is serious business, you teach without assessment. There's no way that you can organize a school without assessment. It's absolutely necessary. And so it has to be dealt with, and I have a couple of stories about the beginnings of large portfolio systems that I want to talk about. 

Matt Renwick (12:59): 

Yeah I just wanted to note this it's more than a collection of artifacts. It can be more than an assessment. You talk about ethnography; it certainly communicates what a child values. But you're starting to talk about school-wide or district-wide. I have to think it starts as you look at portfolios across different, several kids. It's going to start to convey the values of a school or a district as well. 

Terry Underwood (13:36): 

Yes, it absolutely does. In 1989 in the state of Kentucky, the Kentucky Supreme Court ruled that the entire state school system was unconstitutional and had to be redone. And they didn't specify a remedy, but turned it back to the legislature, and the legislature came up with a law. The legislation decided that they were going to use a portfolio as a reform tool. By the way, the reason it was unconstitutional: there were 66 rural counties in Kentucky that filed a suit. Those schools were just so seriously under-resourced that, I mean, there were stories that were sort of legendary. 

(14:46): 

But the school finance cases had been coming up since the sixties. There was a 30 year backlog of cases where they were decided. Usually it was just a matter of the state legislature coming up with some kind of law about how we're going to finance the schools so that we don't have such unequal funding. But in this case in Kentucky, they decided that they were going to use a portfolio assessment as a reform tool. And this was in the legislation. So they came up with an advisory committee for developing the portfolio system. And essentially it, you know the requirement in the Common Core that LEAs have formative assessments where you pool the data and you're able to look at trends and that sort of thing? 

(15:45): 

It was on steroids in Kentucky. The teachers had to make assignments pretty much across the state that were at least commendable in intent. And so the long arm of the state was reaching into the classroom, and teachers were giving assignments to kids based on specifications from afar, and kids were writing and putting those papers in their portfolios, and they were going out for moderation to make sure that they were being reliably scored and all that. And they had to be reliably scored because school funding was dependent on this index. It was sort of like in California: we have the A.P.I: the average performance index or something per school. Well, in Kentucky, 14 of their score on that index was determined by portfolios. 

Matt Renwick (16:49): 

A standardized assessment, essentially. 

Terry Underwood (16:52): 

Extremely standard. The difference is it didn't have any standardized tests. Just imagine if every day when you went in to teach, you had to give a standardized, multiple choice test. It would be equivalent to that, except it wasn't multiple choice. I think there was flexibility in the prompts, but essentially the kids hated it. The teachers were in many cases resentful. It's interesting that there's evidence that it actually improved. 

Matt Renwick (17:33): 

I was gonna say, yeah. I mean, just having the kids write every day. There's got to be some growth there. 

Terry Underwood (17:40): 

Yeah. They did. When I was preparing for this, I remembered that the Kentucky Writing Advisory Committee created a writing portfolio assessment system. And that system had to embrace writing with a purpose, which means that it's more than spelling, it's more than generating paragraphs. There has to be a purpose to the writing. Writing is a process that we don't just think about and create a text, that there are stages that you go through to create a text writing for an audience: who you're writing for (and) who you can conceive of as the reader impacts both your purpose and your process. And then use that writing for a direct assessment to figure out what is going on in the classroom. 

Matt Renwick (18:52): 

Well, when you first mentioned this project, it sounded pretty good. I mean, there was good purpose and audience. That's great stuff. But you were mentioning that the teachers and the students were not appreciative of it, because it was kind of top down mandate. 

Terry Underwood (19:16): 

Well, it took all agency from them. I think part of the problem is in the rural schools, it may have been seen as a blessing compared to what they were getting before. But in the more affluent areas where kids were probably getting fairly good instruction anyway, it was an add-on. It was a chore, it was an intrusion. And so when you stop to think about, "How do you create a portfolio assessment system?", the first thing you have to ask is, "Why do I want a portfolio system?" And who is going to be involved in it? And if you forget that the primary participants are the students... 

Matt Renwick (20:04): 

Mm-hmm. 

Terry Underwood (20:05): 

and their teachers, then you're gonna fail. There's no sense even beginning. And that's what happened, I think, in Kentucky. They forgot that they're dealing with teachers and learners. David Pearson makes a case for what he calls "teacher prerogative". Professionals in other areas like medicine and law have what's called prerogative. And what that means is that research is research, but it is the practitioner's prerogative to use that research as they see it useful in their own practice. And it is an individual prerogative. Unless you're an MD and I get you from malpractice or whatever, I'm not going to be able to put you on remote control and tell you what to do. 

(21:05): 

Now, in an ideal world, we would have that in teacher practice. Now that might generate fear, but it's going to be a different kind of fear. Rather than fear, that I'm gonna be found out that I'm not really as good as I appear or whatever. It's fear that I might be wrong, that my kids aren't going to do very well. Even though there might be an activity that I'm thinking about doing in a classroom, even though I don't like it, I don't think it's going be very much fun, I'm going to have to work at it myself. I'm going to do it because I have a fear that if I don't do it, things aren't going to work as well. An, I'd be definitely willing to talk about that with other teachers in public. 

(22:02): 

The contrasting case here in 1990, 1991, the state of Vermont, I think it may have been the first state to voluntarily decide to do a portfolio assessment system. And how they went about it - I also wrote a chapter on Vermont in that book - was trying to get ahead of the fear that schools would get destroyed in the coming onslaught of standardized tests. So who is creating the portfolio system? The state of Vermont. And why are they doing it? In Kentucky, they were doing it to reform. So it had a political purpose in Vermont. They were actually acting out of fear that what was going on in the schools was going to be hyper standardized and pretty much controlled. 

(23:00): 

And they wanted to get ahead of that. This book, by the way, is designed for anybody who wants to create a portfolio system. You need this book, because what we do is we examine nine different portfolio systems from the level of the classroom to the school, to the district, to the state, to the nation. And we talk about their strategies for collection. How do they collect? What do they recommend people do to collect material in the portfolio, their strategies for selection? At some point in the portfolio process, learners have to select. The whole question of who selects the material, does the teacher, or does the kid? Well, that has to be negotiated as part of the design process. And then, reflection...some will argue me, among them, that you can't call it a portfolio if there is not a structured way. 

(24:09): 

Kids are gonna reflect on their work periodically in a formal public thoughtful way with their teachers and with their peers. So you have the collection part of it, the selection part of it, the reflection part of it, how are they going to do that? And then the publication part of it. At that point, you really begin to get it. You have to be very careful about who owns this work? What work is going to be made public and why is it gonna be made public? And that's where really I think if the public purpose is for an assessment, that it has to be really carefully controlled because you run the risk of disclosing all kinds of stuff. People will destroy the portfolio system. 

(25:05): 

If that's not the purpose, then it probably should be celebratory or showcase, or some sort of a portfolio review. Then it provides the same incentive that kids need to write, that somebody's going to read this stuff, and we're going to look at it for positive purposes. In this book, after each chapter we have a section called Opportunities for Discussion. Inquiry #1: the Kentucky system asks students to write for real world audiences during their classroom instruction, while simultaneously asking students to write for an examiner audience. That is for the individuals who review and score portfolios. What influence might this complex relationship have on students' writing processes? One time I was working with a local school, an elementary school, and they were interested in school-wide writing assessment. 

(26:07): 

And we designed some assessments. A couple years later, I went back to the school and I talked to a teacher that I had talked to before. I found out that some of the 3rd grade teachers were beginning to score higher on the writing assessment then the teachers who had been sort of writing gurus After I checked into it, I found out that the formula was if you wanted to score higher in third grade, for every noun, get your kids to write three adjectives so that you get this kind of sparkly, popping kind of writing. Now a good writing teacher will say, pay more attention to your nouns because the nouns carry the substance. And sure enough, when I looked at these papers, these kids were three adjectives per noun. I remember this one paper, "My best friend Kyle has short, dark, light hair." 

Matt Renwick (27:23): 

<laugh> 

Terry Underwood (27:24): 

You know, the attention is on the adjectives. So what's going to happen if you have the kids writing for real purposes at the same time, we're going to take that for an examiner audience. And it kind of creates a schizoid writing environment. "Kentucky's system was developed under the assumption that instruction would not change unless teachers were given financial incentives. Do you believe that this assumption is true?" So this portfolio system...you can see there's a lot of kind of strings attached to it that will distort what you really want. You want to get kids who, when they crossed that bridge after graduating from high school, they're not going to throw their stuff in a river. In my case, it was just accidental. I mean, I happen to have a fourth grade teacher who's like, my stories. 

Matt Renwick (28:27): 

There you go. 

Terry Underwood (28:29): 

Yeah. And how do we get that in some sort of systematic way? 

Matt Renwick (28:39): 

I'm thinking about today, why any district or school or even teacher would want to start portfolios. One thing that's starting to come up, I'm seeing more of is related to the test or to assessment like you're talking about here, Terry, not to circumvent them or to avoid over-standardization or to reform, but to compliment. So for example, English learners, some of these students are not doing well on ACCESS tests when I've worked with another district or two. And so they see the portfolio as another way to kind of create a richer picture of this kid. And maybe they're not doing well on the standardized test, but this portfolio can serve as assessment and, you know, it can also be gamed as you were saying with the three adjectives. It sounds like kind of a theme though, is they have very good intentions, and then when we put into practice, things get complex and complicated and messy. 

Terry Underwood (29:58): 

Yeah. And I really think that's the heart of it: the degree to which the student is legitimately involved and seen as a real active participant in it. And not just someone that we're trying to help or we're trying to make sure that that this person can grow up and function in this society, but actually an agent who we hope at some point is going to be self-actualized and would be able to self-assess. In an ideal world, what I would like to see is when we assess kids, let's see how well they self-assess, how well they know what they're up to and how clearly do they have a sense of where they're going. 

(30:59): 

I don't know if you are familiar with John Downing? John Downing was a researcher from the seventies, and he died young, but he had this theory of cognitive confusion that the problem with kids learning, especially in in literacy, is that they don't understand why they're doing this. They don't understand what the point is. I wrote a story where Mr. Bubbles goes out to talk to a first grade kid who's been this second grade kid. His teacher thinks he has dyslexia, and Bubbles talks to him and asks him, 'Well, what do you think is going on in people's minds when they're reading? Have you ever ever watched anybody read?" 

(31:51): 

"Yeah, watched my dad. My dad sits there. "Well, what's your dad doing?" "Well, he's pretty much just like real quiet, and he has the newspaper in front of his face and he's doing something, and I know he's doing something, but I know that he gets really mad if I bug him. But, you know, he wants to, he wants to read this paper." But he doesn't know what's going on. And then he goes to school and the first thing he finds out is, when you sit down at your desk, you see a letter and the letter goes B, but the kid knows that's not what my dad is doing, you know, and for four or five months, this is what he gets. And so eventually he just says, "You know, I'm confused and I'm never...this is not, this is not for me." 

(32:41): 

"I don't understand what's going on here, and I'm just going to tell me what to do because I have to." And so that early on, if we get comfortable with confusion and we start to think that we have to look elsewhere, or we have to capitulate and stop looking and just do what we're supposed to do. My own daughter, she's in a doctoral program, and she was telling me just a couple weeks ago that she had no idea how deeply she had internalized that. And it's only now that she's thinking about, "What am I going to do my dissertation on?" And she has to come up with it. It's got to be something she really deeply cares about. 

(33:32): 

All of that external guidance is by the wayside. Could you imagine if we start that in kindergarten? Yeah. And portfolios will allow you to do that. If you think about a writing prompt...if I give you the writing prompt, if you are supposed to write every day, and I'm telling you what to write about, right at that point, you are walled off from the activity. When I taught fourth grade, I taught my kids to write prompts. And what goes into a prompt? And when we did writing workshop, they had to tell me who you're writing this, what is the content, what's the substance going to be? How do you think that starting this with a narrative is gonna be a good idea? What makes you think that? That became their job. Then I had something when I conference with them. I would be able to go back and reference that. 

Matt Renwick (34:50): 

Whenever you hear, "Teach the reader and the reading," you know, or "Teach the writer and the writing," I think this is what you're talking about. You're teaching readers and writers while you're teaching reading and writing. And it's so much about shifting the responsibility and feeling the trust and the confidence in kids to not just teach them writing today, but to be a writer and a reader tomorrow. This has been great. I wanna keep talking with you, Terry, but I'm gonna pause the recording here and just want to say thanks for joining me. 

Terry Underwood (35:25): 

 Alright, Matt. Hey, take care of yourself.