Ben Daley:
I would recommend forming a committee to study the problem and maybe over a several-year period, you could map out a plan for how to approach this… OR—

Alec Patton:
This is High Tech High Unboxed. I’m Alec Patton, and that was the voice of Ben Daley, president of the High Tech High Graduate School of Education. In this episode, Ben is going to share a strategy for getting data for improvement, whether you’re a teacher, a superintendent, or you do something else entirely. If you work in or with schools, this technique will help you out. Seriously, it will. We start the conversation by talking about the excellent book Upstream by Dan Heath. In it, Heath tells a story about how the city of Rockford, Illinois used what Heath calls a by-name list to tackle homelessness. I’ll let Ben take it from here.

Ben Daley:
There’s one story he tells in particular about going into a city where they’re working on homelessness and they’ve gathered all these different nonprofit providers and social service agencies, and they’re talking about what to do about unhoused people in their community. And they’ve been getting together for years and they talk about the problem, “What are we going to do? Oh, this a challenge?” And then they did a by-name list and they just wrote down the names of all the unhoused people in their community. What they use the meeting time is to go through them. What about Bob? Actually, I saw Bob at our homeless shelter last week. He’s doing fine.
What about Mary? Oh, last Tuesday, she came in for a meal. I know what’s going on with her. Actually, she needs some medical appointments, et cetera. And they describe the transformation of that meeting from sitting around talking about things, not really clear what to do to this very obvious action orientation of like, “Oh, we’re talking about Alec, we’re talking about Ben, we’re talking about Mary. We know what to do now.” And I just found that even though it sounds simple, it’s not really. I am often in meetings where I’m sitting around talking about what we’re trying to do and not being as clear about that kind of action orientation.

Alec Patton:
Yeah, it may be simple, but it’s not happening that much.

Ben Daley:
Exactly. We did a project working on the Next Generation Science Standards. So there’s these new standards, they represent a different content to teach in a different order in a different way. There’s a lot and you have to understand what it even is, what it’s about. How does this align for each person’s teaching philosophy? How does it align with our vision at High Tech High? Is this a thing we’re trying to embrace or not? So we’d understand what it was. So there’s a lot of complexity. And also, we have not had a tradition of saying from the central office, “And now thou shalt implement Next Generation Science Standards.” So we’ve been working on it for some years, and then I read about this by-name list in Dan’s book, and I thought, “Well, that’s interesting.” So a colleague and I figured out, “Okay, what’s the name of all the science teachers in High Tech High elementary schools and middle schools?
And it was about 120 teachers across every elementary school teacher, all the science teachers in the middle schools. And then we just thought, “Okay, well, what are we after here?” There’s this NGSS thing. Well, how NGSS-ified is this teacher’s classroom? Which, of course, is an intentionally preposterous way to frame the question, but we thought, “Well, if they don’t even know about it as far as we can tell or are actively resistant to it, then okay, that’s like a one. And a three is they’re embracing it, they’ve been to a bunch of things, they’re really trying to make these changes in their classroom. Okay, that’s a three. And now it’s somewhere in the middle. Okay, that’s a two.” And we just went through 120 teachers’ names to say, “Are they one, two, or three on this ridiculous classification?” And of course, we knew many of the teachers, but not all of them, and we had to do a little follow-up, so it wasn’t just one meeting.
But it was so striking to me how obvious it was what to do next because we had one school where there were six science teachers and four were totally embracing the standards and two were working on it. And we had one school where all six teachers were not doing this at all. And it was like, “Oh, right, because the person who was supposed to be the lead on that, actually, they decided to not work on that project. We never managed to connect with anyone and get a lead on that. Okay, what are we going to do with this school next versus what are we going to do in that school next? There’s completely different actions we need to take.” So it was just like what Dan described. It took it from this general conversation about what should we do to really clear next steps.

Alec Patton:
Yeah. Why do you think that wasn’t the very first thing you did?

Ben Daley:
I think there’s several things that are going on there. One, it’s not normal practice that I can tell. So it’s not like we’re all doing it all the time and then we think, “Oh, that’s obviously a tool that we should pull out of our…” Two, you’re assigning a number to a person, or it could be a color. It makes you feel kind of weird. And I’ve experienced a lot of resistance to doing this. One of the things in that project, there was no sense that, “Well, if you’re doing NGSS, that’s good. Oh, great teachers are doing it and terrible teachers are not doing it.”
It was just trying to be clear on where people are in their work on this specific practice. So I think there’s a hesitancy. You might feel like, “Oh, well, Hi Tech High, that makes us nervous.” I’ve experienced it in quite a few different contexts where just even doing something like that feels evaluative. It feels like part of some weird accountability structure when actually it’s not. The point is as someone who’s trying to get something happening in the system, you could actually see, “Oh, well, here’s a real picture of what’s happening. I wonder what I should do next.”

Alec Patton:
Yeah. I think in some ways you’re understating the problem because I don’t think it’s that it doesn’t happen. I think it’s ubiquitous, but it’s always tied to accountability and possibly material rewards and punishments. We know that certainly since 2000, plenty of school districts have made lists of their teachers where they’ve given their teachers scores and then they’ve maybe tied that to bonus pay or it’s become a punishment and reward system.
We certainly, as teachers, have lists of kids and then we say, “Which of those kids turned in their homework and which one didn’t, or which one got a 10 out of 10 and which ones got a four out of 10?” But usually, if we’re making a list, it is, to quote Johnny Cash, “There’s a man going around taking names. He’s determining who to praise and who to blame.” I think I probably butchered that quote, but-

Ben Daley:
It’s a terrible quote as delivered, but keep going.

Alec Patton:
We’re cutting that.

Ben Daley:
Keep it in.

Alec Patton:
The point is, I think generally when you make a list of names and you put some kind of number next to it, the usual idea of follow-up is either we’re going to reward them or punish them, or we’re going to provide appropriate support in a way that we actually mean, “Scare them a little bit.” You might be listening to that and being like, “No, I do this all the time.” And if you’re thinking, “No, I do this all the time without attaching it to accountability measures,” then congratulations and get in touch with us. But I think for me, I associate putting down everybody’s name and putting a number next to that as then being tied to some kind of accountability measure. And I mean the fully loaded use of the word accountability.

Ben Daley:
And to me, the by-name list is, A, obvious, and B, a radically different way of thinking about, “We’re trying to do something, but how’s it being taken up? By whom? How are they finding it useful?” And the judgment is on yourself of what I need to do better. So there’s such a thing as looking at a gradebook for kids, and the point is for the teacher to realize, “Oh, all my students are struggling with blank, so I should change my behavior.” But you’re right, the main purpose of a gradebook is to assign a grade in a report card at the end of the semester. So even the same tool can be used in two radically different ways.

Alec Patton:
As a teacher, I found that one of the most helpful things I did was to make a spreadsheet with a list of all the names of my students and then teach them about making a claim based on evidence, and then have them all on an exit card, make a claim based on evidence that was relevant and then just code, “Could they do that? Could they sort of do it? Could they not do it at all?”
And then I just suddenly went, “Well, that’s basically the essence of my job is am I able to help kids learn the skills of humanities and do I know who’s learning it and who isn’t?” And I’m not even sure if I can articulate why doing that felt so different from the gradebook, but it felt like a completely different way of looking at it

Ben Daley:
Yeah, because the gradebook tends to be a collection of, “Did they turn in the work or not? Did they get the problems right? What percentage of questions on the test did they get correct?” Which is kind of what you’re saying, but it’s kind of not because you’re being much clearer, I think, about a very specific practice.
And even if we say how NGSS-ified is this teacher, which of course is an absurd, very broad, what does that even mean? And yet it’s specific in a certain kind of way about, “We’re making an effort towards this particular thing. Where is each individual at in that set of practices?” So I think there’s a distinction of what you’re even after.

Alec Patton:
Yeah. I also think there’s a real importance to being nimble and flexible about your measurements. If you did that and you said, “Hey, you know what? Turns out that how NGSS-ified are they is gibberish and it doesn’t work as a rating system.” Then you throw that out. You don’t just say, “No, that’s the system we came up with, so we’re going to stick with it.”
There’s a kind of non-attachment element to improvement science and continuous improvement. That’s one of the things that really attracts me to it, is that we’re going to try making this document using this clearly absurd idea of NGSS-ified. We’re going to do it once. We may never do it again. It will be definitely incomplete. It will be possibly incorrect. The only claim we’re making is this might be interesting, and then we see what we get from it.

Ben Daley:
Exactly right.

Alec Patton:
That’s really powerful.

Ben Daley:
I was at a meeting last week where someone described the question of whether kids feel engaged at school, and they described an activity that you can do with a grade-level team where, again, you have the list of the students, and I’m not going to get this quite right, but it was something like put up a red sticker if you just know who that kid is, “I recognize that name, I know who that is.” And you put up a yellow sticker if you know something about that student. And you put up a green if you have a productive relationship with that kid. And just have a grade-level team of teachers do that for a list of students and then say, “Ah, what are we noticing? Here’s some kids who none of us are connected to. We should do something about that.” And this person was saying, “Even in 10 minutes with this very rough measurement, we can get something that’s useful to us.” And again, it’s for us to change our behavior.

Alec Patton:
Yeah. Okay. So let’s say somebody’s convinced by this and they’re like, “All right, I really need now the making a list of names protocol.” Now, you just shared one, that was really cool, the sticker one, and I think that everyone should do that. But let’s say that you’re just more generally like, “Your improvement situation isn’t exactly like mine. I don’t need to know how NGSS-ified my teachers are, but this does sound kind of useful.” How should somebody actually take action?

Ben Daley:
I would recommend forming a committee to study the problem and maybe over a several-year period, you could map out a plan for how to approach this, or… The thing that’s tricky is you actually have to get a list of names. In many cases, that sounds easy, but depending on the context you’re in, that actually might not be that simple. So you need to get a list of names literally and put it in a spreadsheet and then sit down with one colleague and do the activity and just see what you learn from it. That’s what I think.

Alec Patton:
Yeah, that’s absolutely right. And the next step reveals itself through the task. If you go, “Oh, it turns out we could only code four out of the 20 names we put down-”

Ben Daley:
That’s telling.

Alec Patton:
… that’s telling, your next step reveals itself. If you’re like, “Man, I thought I knew everybody who was doing this. I don’t even recognize some of these names,” that gives you something. If you’re able to go, “Oh, you know what? I think I can code this here and see this,” then you’ve got something to build from. Or if you both do it and you find that you don’t agree about it, yeah, whatever you do, is going to-

Ben Daley:
It’s going to reveal, it’s actually somewhat obvious the next thing that should happen.

Alec Patton:
Yeah, exactly. I think we nailed it. Done. All right, I’m going to try that again. All right, Ben, I think that covers it. Thank you so much.

Ben Daley:
Thanks, Alec.

Alec Patton:
High Tech High Unboxed is hosted and edited by me, Alec Patton. Our theme music is by Brother Hershel. Huge thanks to Ben Daly for this conversation. We’ve got a link in the show notes where you can find lots more podcast episodes, articles, and videos about continuous improvement. Check it out. Thanks for listening.