Jon Orr: Hey there Math Moment Makers. In this episode, we are sharing two assessment strategies that are widely used every single day in math classes everywhere. And we're recommending these two strategies we got to stop using, or at least limit the use of these two strategies. So these two strategies that hinder our goal of building those math classrooms that allow students to think, and also allows all students to access mathematics.

Kyle Pearce: Yeah, that's right, John. Friends, stick with us and you're going to learn how two assessment practices, we're going to learn how you can stop using them and what you can do instead. John, are you ready to get started?

Jon Orr: Let's do this.

Kyle Pearce: Welcome to The Making Math Moments That Matter Podcast. I'm Kyle Pearce.

Jon Orr: And I'm John Orr. We're from makemathmoments.com.

Kyle Pearce: We are two math teachers who together with you, the community of math moment makers worldwide, who want to build and deliver math lessons that spark curiosity, fuel sense-making and ignite your teacher moves. John, it is early on a Sunday morning. I'm sure people can see it. Those who are watching here on YouTube, they can see it in our faces and our eyes. But I don't know about you anytime we're ready to chat about math and in particular assessment, I get all geeked out. How about you?

Jon Orr: Yes, Kyle, I am super excited to get into this episode. We're going to share those two strategies in just a moment. Before we dive in, we want to thank our listeners wherever they are, and actually listeners we'd love for you to stop what you're doing right now. Snap a pic of whatever you're doing. This always interests us. We are always looking to see what's going on? Where do you listen to us? Are you listening to us while you're washing the dishes or on a walk or on a job? Wherever you're doing, stop, take a pic and then tag us over on our Facebook page or @MakeMathMoments on Twitter. We'd love to see what you're up to.

Kyle Pearce: Yeah. We actually saw a couple come in from last week's podcast episode on Twitter, it's great to see what people are up to, what they do for the routine. And you know what, after you do that, why not pause for a second and leave us... Actually, you don't even have to pause. You can just grab your phone, go ahead, get into that podcasting app that you're in and leave us a five star rating and review. Again, don't leave a five-star if you don't think it's really a five star, go ahead and leave us whatever that rating and review is because we want to hear from you, especially on Apple podcasts, that's where most people download our show from. So head on over there, we read them all and we actually look at them and try to think on how we can improve the show for you.
So give us that feedback. You don't know how helpful that is. Just like this feedback we received from Katriane Liz and I don't know about you, John. I'm pretty sure it's the Katriane we know who actually has been on the podcast for a mentoring moment.

Jon Orr: Yeah. Katriane says, "It's the music for me. I'm pretty sure I've already rated you guys five stars, shame on me if I have not. But just wanted to say again that I've always learned something or get reminded of something important that I might have let slide in my math teaching. Today it was online wait time, acknowledging how hard that is and how important. Message received. And truly you guys have the best theme music. I walk along with my dog getting my PD and I end up dancing at the end of each episode."

Kyle Pearce: Fantastic. Isn't that awesome? I'm telling you nothing energizes us more than when we read those reviews. We see those ratings come in and I'm telling you, it makes us feel so great. So take 10 seconds. Pause, head into your podcasting app. Go ahead, leave a rating and review, be honest and accurate about it and let us know so that we can get that quick feedback.

Jon Orr: Awesome stuff. That would be the world to us if you could do that. All right, Kyle, let's get into this episode, we've got two strategies to share, two assessment strategies that we are going to encourage you to stop using or at least limit using because sometimes they are tough to stop using these. And so we're going to outline the two strategies, but we're going to do them one at a time. So the first strategy that we're going to start with is the idea of a learning goal. And this one comes from it's rooted in research for sure. And the research that we're talking about is this idea that when you walk into the classroom and you're doing your lesson or your activity, many of us think that we have to write the learning goal, like the topic, the idea, the thing we're setting out to do on the board right away.
For example, "Today, we will learn about solving equations. Or today we will learn about measurement of 2D shapes." We have been ingrained that we need to write these on the board and actually it's rooted in good research. We definitely want to be communicating learning goals. Now this is actually the strategy we're going to actually ask you to stop using. And we want to give you that heads up. Because you're like, "Wait a minute, John, this is rooted in research." And the research comes from a couple of different places. And the first one is this book that we've been reading for a number of years, it's actually probably the guidebook for as many district review or districts policies on assessment, which is from Dylan William, his book Embedded Formative Assessment.
He has five assessment strategies, five assessment techniques you need to be using in your classroom. And the first one, Kyle, the first one is clarifying and sharing and understanding learning intentions and success criteria. So no wonder everyone is like, we got to write these on the board. Dylan Williams says we got to share that information. It's number one so we think we should put it right up there, up front.

Kyle Pearce: Yeah. Absolutely. And also the other thing too, John, that a lot of people tell and kind of comment about when they're reaching out to us at makemathmoments.com is, "My district or my administrator or my department or whoever it is says, I have to clearly have that learning goal stated up on the board." So we know that there's this pressure coming, maybe downward pressure from your math consultant or whatever it might be. And now I want to be clear here because what you didn't say, John, what you didn't say was that the teacher should know or shouldn't know the learning goal, because that is absolutely imperative. You as the educator must actually know and understand the learning objective. I've got up for those on YouTube right now. You can check this out. Here's the book. And actually it's got kind of a new cover, the new version of it.
John's got the old version when he shares it. But yeah, this particular book it's based on research, but I'm going to argue John that sometimes when we build ideas off of research, sometimes we blanket them very quickly and we almost proceduralize them. So for those who know us here at Make Math Moments, we always talk about avoiding the rush to the algorithm. Because sometimes algorithms, they can be helpful in certain scenarios, but maybe they're not very helpful in others. An example of that would be subtraction. When I'm subtracting 2001 and 1,999 using the subtraction algorithm, is a horrible idea. They are two apart, the difference is two. And I see that in my mind using a number line, but we see students all the time borrowing, borrowing, borrowing, borrowing, making mistakes, not understanding what it is that they're doing. Well, the same happens here when we have something that's pushed down from a district and there isn't a clear understanding, sometimes not a clear understanding from the decision-maker themselves.
So they hear this, they hear it in a workshop or they read a small snippet online, or maybe they listen to a podcast like this one and they don't get the full picture. And then we've made a policy about it. John, there's a specific reason why you and I think, especially at the beginning of a unit, if we think about a problem based unit, there's a reason why we think it's actually, it could be detrimental to actually post a very explicit learning goal that actually articulates the concept right up front. So let's be real clear here, help people at home listening, understand exactly what we're referring to here.

Jon Orr: Yeah. And so before we think of why we get into why this particular one is not awesome for our learners is that actually, if you dig a little deeper, so going back to the policy aspect that you were just talking about, Kyle. If you dig a little bit deeper in this book, if you continue to read all about his first practice. He actually has a quote it's in his book, but I'm pulling this quote out. Dylan William actually argues that it's so important that we share this learning goal, but it doesn't have to be and it cannot be, this is the words that he uses. It cannot be in a formulaic way. Not at all. It cannot be in this formulaic way that we are making it. Like you just said, it's a policy. We got to write it up on the board.
However, he's arguing that you cannot do that in that formulaic way. And actually it's also in Principles to Actions. So Principles to Actions also says we've got to communicate success criteria and learning goals of one of their principles. But also in that book, if you dig a little bit deeper, it says, although daily goals need not be posted, it is important that students understand the mathematical purpose of a lesson and how the activities contribute to and support their mathematical learning. So these two research-based books that were building these policies off, actually don't say, "Thou shalt write this on the board at the start of that lesson." Because there's more to mathematics than just being formulaic. And this is actually goes into hand-to-hand with what we've been preaching on the podcast the whole time about-

Kyle Pearce: And teaching, right? And teaching in general.

Jon Orr: Right. You can't just say, this is the way math is done every single time. Like we've got to do more with it. And to go back to your lead in there, Kyle, about why. And the idea is that for us, it's been about problem solving. There's reasons why Dylan William and Principles to Actions is saying, you don't want to do this in a formulaic way. You don't want to necessarily post them. It's all about connection and making kids clear that what they're learning is connected to prior learning and future learning, that's the important part. And when you post them on the wall or at the beginning, what you're doing is you're saying today, whatever we're doing, it's going to be about measurement of 2D shapes or it's going to be about solving equations.
And so if you're trying to teach through a problem-based lesson or you're trying to teach through some productive struggle where you're posting a question and kids have to kind of struggle the way through it. When you tell them upfront that they're going to be solving an equation, or you're going to be working with measurement of 2D shape, or you're telling them what the learning goal is. Kids will try to like, they've played the math game for a long time. They're going to try to go like, well, how do I solve this using an equation? Or how do I solve this using this?

Kyle Pearce: Or I must I already know the formula, right?

Jon Orr: Exactly.

Kyle Pearce: crosstalk obviously told me the formula and he's just trying to pull a quick one on me or something.

Jon Orr: Right. Obviously we have to use this learning goal today to do that instead of actually problem solving and thinking about different strategies. Because if you are truly trying to show kids how mathematics connects from one topic to another, or you want to develop strategies and develop problem solvers, we need to hold back those learning goals so that when the strategies come forward on this problem, you can use like Peg Smith's and Mary Kay Stein's five practices to start to connect these strategies together. And then at the end, once you connect these strategies together, that's when you can state what the learning goal is. And even better, Kyle, I think this is one of our things to do instead of stating at the beginning is have kids state what the learning goal is afterwards.

Kyle Pearce: Yeah, for sure. For sure. And so this is just it. It's like the development with students on what that learning could be. Now I also want to mention too, you said at the beginning of the episode, and I think it's worth reiterating is we sort of said it like two things that we're going to ask you to stop doing. Your hands might be tied. Your district might say, you've got to do this. So what we're going to recommend for you, if that's your case, or if that's the case for you is how can I build in... In the US a common core language would be math practices into this learning goal. So rather than saying, "I'm going to go and today the goal is we're going to be counting change and we're going to be adding and subtracting using different denominations of coins."
You don't have to say that as your learning goal, that might be something that emerges from the beginning of this unit that you're going to use this problem-based lesson. So rather what you can say is you can pull out one of the math practices, which is more of an idea around maybe it's about perseverance or about productive struggle, or maybe it's something else. Here in Ontario we have math process expectations. So I could be saying reasoning improving, right? That's part of my curriculum. That could be the learning goal that I stayed on the board and think of how open that is. Now, first of all, we always talk about how these process expectations or these math practices are actually what we're trying to use the content in order to serve. We're trying to serve using content to build productive problem solvers. We're trying to do that so that students can use strategies and tools.
We're trying to do all of this stuff in service of these practices or process expectations, whatever you want to call them. So if you feel like your hands are tied, they really aren't. You just have to frame what that learning goal is going to be. And then if you do get challenged by a department chair, a consultant, a principal, a whatever. You can explain very clearly why you've selected the learning goal that you've selected for that day's lesson. And then you can talk about how the other content-based learning goals are going to emerge over that specific unit. For me, when I'm looking at this, for our YouTube friends on my screen, I'm sharing one of our latest units here. One of our problem-based units, if you go to makemathmoments.com/tasks, you'll see all our problem-based units. And one of our most recently published is the piggy bank math unit.
And when I click in there, you'll notice that under the spark tab where we want to spark curiosity with our students, we are essentially showing a scenario that's very curious, it's withholding information. It's leveraging the curiosity path. Think of how all of that curiosity is sucked out if I tell you ahead of time that today we're going to be adding and subtracting quantities using different denominations of coins. So those who are watching the video behind me on YouTube, they can see that this is like, you're not sure exactly what's happening. We can have a notice and wonder here, but if you come to the guide tab, which every single problem-based lesson has, you'll see that we have intentionality here. And we're very clear on the big ideas that we want to emerge. So going back to our point, the educator has to have a clear understanding of what these big ideas are in terms of what we're hoping to emerge in this unit of study.
And these are going to evolve over time as well, but that might look or sound different at the beginning of a unit for students than for what you see in your teacher guide. So to me, that is a huge shift in how I use learning goals in my classroom, because I still want to post something on the board so students don't feel like we're just hanging out, but I don't want to be clear on the content specific expectations or learning goals.

Jon Orr: Good tips there, Kyle. Teaching, it should not be black and white. Like this whole, thous shalt do this every single day. Some days it might be appropriate to do that if it's like, we're definitely want to make sure that we did maybe an investigation yesterday, like Kyle just showed up on the screen. We did a problem-based tasks, but today we're now more focused to using that to do this other thing or to do this practice. There are days where it makes sense and days that it doesn't. So in summary here, strategy number one being you have to post learning goals or the assessment strategy is clarifying your learning goals. That's true. However, you don't have to post them up for students to see every single day. It can't be this formulaic way. We've got to have more of an art than say this science approach to teaching mathematics.
And I think that goes with so many things, not just to say assessment or strategies to start class. So in recap here, strategy number one was we got to stop posting learning goals at the start of every lesson specifically to say, this is what this is going to happen. This is what we're going to learn. We want it to be more of art than science in that respect. Kyle gave you a couple of good suggestions on that with changing the language of the learning goal that's intended. Also, we talked about establishing a learning goal at the end. So those are two different ways you can tackle that same kind of issue with that assessment strategy. All right, Kyle, are we ready to roll into the second strategy we wanted to share here in this episode?

Kyle Pearce: I think so. And this is one that is going to be really hard, especially right at the onset, because I feel like it is the strategy that every educator has experienced themselves as a student. And it's sort of like the go-to, right? You do this all the time in your classroom. And if you're sitting there, you might be wondering, you're like, "Whoa, what is this strategy?" It's not the learning goal because we already talked about that.

Jon Orr: What's a strategy that you probably use every single day when you're working with your students. Most likely, and this is not everybody because this is again, it's more of an art than a science.

Kyle Pearce: Yeah, absolutely. And the other thing is too, it's like, you almost do it and you might not even realize you're doing it. It's so embedded in terms of how a typical lesson is led, whether it be math class or any other subject area. And for us, it's this idea of, and we're going to say having an over-reliance. So we were saying, stop doing these things. We're going to say stop being overly reliant on this particular strategy. And it's the strategy of asking students to raise hands and volunteer, to share. And I know I'm picturing in my classroom, it used to feel like crickets, especially when I was doing a pretty traditional lecture based approach to teaching, you would try to keep students engaged by asking them to essentially like you throw them these little Love Balls. And you'd be like, "All right, let's do this quick calculation over here. What's 15 plus three. Anyone?"

Jon Orr: Trying to get engagement somehow be like, "Oh, well let them do the easy kind of calculation non-thinking questions just so I can get more voices into my room."

Kyle Pearce: And think about this too, John, let's say I'm doing a problem-based lesson. So I just talked about in a lecture-based lesson, we keep the floor super low by asking questions that are almost obvious. So it's like fill in the blank stuff. "And the distance across the circle from one side through the center to the other is called? To, anyone?"

Jon Orr: "It starts with a D."

Kyle Pearce: "It starts with a D." And then think of how that makes us feel as students. First of all, you have that small group of students that they're going to raise their hand no matter what, but then other students are looking at them like, "Why do they keep on wanting to share?" And it almost creates this culture of, I'm not going to be that person who just wants to hear themselves speak. And then in a problem-based lesson, it's like, "Does anyone want to share their strategy?"
Think of how many students are in the room going? "Well, I think my strategy is pretty good, but I don't want everybody to think it's all about me." There's a lot of people out there that you're like, "Well, I don't want my voice to overpower the rest of the room. So I'm just going to kind of sit back." So I'm wondering John, what are some ideas that we might think about again, it's going to take time if you are like us and you were overly reliant on this raising hand strategy. What are some things we can think about that might get us maybe more aware of how often this is happening in our classrooms without us realizing it? And then what could we do instead in order to promote less of that strategy?

Jon Orr: Right. And I just want to touch on some things that we may not be thinking about when we use this strategy and then we can get into how we can change that around and kind of minimize that strategy. Because if you are using that hand raising strategy, like I did for many, many, many years as a teacher and the Socrative method was built on that. All of my teaching education at teacher's college or my pre-service education was all about teaching you exactly how to do that. And they would give you strategies like, well, make sure you get both sides of the room and you're getting a variety of kids. We were definitely asking kids to raise hand because that's the way we thought we would elicit evidence of learning from students. And this is also back to Dylan Williams' book as his second strategy for assessment is, we got it with elicit evidence of learning from kids. How do we do that? Well, we're going to ask them to raise their hands. That's what we did. For me that is what we were taught. Also it's again, it's back in Principles of Action.
We got to make sure we're eliciting kids evidence of learning. It's good for learning. Definitely. You got to know as a teacher, what students are learning. And the only way we did that was like, well, let's pull or let's ask them, do they know this concept? But what is happening when you do that? The background of what's going to happen in your room when you ask kids to raise their hand. And like you just said, Kyle, only certain kids are going to be brave enough to put their hands up. And this kind of goes back to something that we've shared, I think here on the podcast before, but definitely in our courses, the Matthew Effect. And the Matthew Effect, it was, I learned from Malcolm Gladwell's book Outliers. And he had that story at the very beginning. He looked at OHL hockey players and said, okay, for some reason, when you look at the roster of an OHL. Now, OHL... You want to jump in there Kyle?

Kyle Pearce: I'm in Australia right now. What the heck is OHL?

Jon Orr: Sorry.

Kyle Pearce: I'm in Ontario. So I know, but how about our friends outside of Ontario?

Jon Orr: Yeah. Malcolm Gladwell is a Canadian too. So he used OHL, but OHL is the Ontario Hockey League.

Kyle Pearce: That's where Wayne Gretzky came from. That was his league he grew up playing in as a junior. Right?

Jon Orr: Exactly. Many Canadian hockey players before they get to the NHL have played in the OHL. It's kind of like the junior league. So anyways, he said, when you look at a roster from the OHL, generally the birth dates of almost all players are between January and March. It's like both, it's like January, March, most players have that birthday. He's like rarely, rarely would you find a hockey player on the roster who was born in August, December, October. It just doesn't happen and it's very rare.

Kyle Pearce: And John, what about the MLB? Do you remember that example? I don't know if you recall. It just popped into my mind.

Jon Orr: I think it's because it has a different start date. Right, Kyle?

Kyle Pearce: Yeah.

Jon Orr: So for example, the start date of hockey here in Ontario and Canada is cutoff as January 1st. It's like, school-aged. So it's like January 1st to December 31st, if you're born in that year, you play in the same league. And so if you think young players here in Ontario, kids start hockey very, very young. And so when you're very, very young, there's a huge difference in height, weight for kids who are born on January between January and say March versus November to December. It's like all of a sudden it's like, because when you're talking five-year-olds, four-year-olds-

Kyle Pearce: That's like 25% more time on the earth for a student who is born 12 months later. Which is incredible to think about.

Jon Orr: It's exactly. So all of a sudden what happens in the Matthew Effect is that those kids between January and March, are a little bigger, a little stronger. And then therefore in hockey, the coaches who are selecting teams. So we're talking like travel, competitive teams, are selecting kids to be on a team. They choose the kids who are a little bigger, a little stronger. And then the next year they get more coaching and then they're a little bit more stronger and they're a little bit more bigger. And then they get more coaching. And so on it, all of a sudden effects, this huge amount of kids who are like this group of kids because of where they were and when they were and when they were born, that group of kids, all of a sudden gets stronger over time.
So it's like another way to summarize the Matthew Effect from Malcolm Gladwell was like the strong get stronger because you're giving them more feedback, more coaching. And so if you turn this back to math class, when you ask kids who are raising their hands, they're already probably the ones who are raising their hand, like Kyle said is they're already a little braver. They're already a little stronger. Those kids are going to get even stronger because their thinking is getting confirmed. It's like, they are wondering this question, they're answering this question. And then they get that feedback loop confirmation. And then the kids who are not raising their hand are not maybe confident, not sure about the answers. They're not getting that feedback. They're hearing that feedback, inaudible that other student, but they're not getting it themselves.
And so what also happens, Kyle, other than braveness is you've got the spectrum of introverts versus extroverts. I'm an introvert by nature I'm sure it might get people like, "Well, John, you're talking on a podcast right now. You can't be too much." But you know me, Kyle. I'm more of a, in the background kind of person. And I was never a person who would volunteer my answers in math class. But if you think about the students who were in your room, if you're going to serve all students in your room, when you ask kids to raise their hand, you're actually only catering to the brave. But also the extroverted kids who are fine with voicing concerns, flicking foolish in their class. Whereas the introverted students are reserved. And I learned about this, even though I felt like I'm an introvert now, it wasn't until I read the book Quiet, which kind of changed the way I thought about my classroom dynamic. I didn't actually think about how I was say, favoring extroverted students versus introverted students. It was a huge eye-opener for me.
In this book is called Quiet by Susan Cain. It's, The Power of Introverts in a World That Can't Stop Talking. It's not a math specific book, it's not a teaching specific book. It's about lots of different ideas about where introverts are in our world, which caters to extroverts. And often you kind of reward extroverts in your class and be like, "Oh, they're outgoing. They're making sure that you're participating." We've got this group of kids who are introverts that we might be underserving in our class, especially when we ask kids to raise their hands to engage. So I wanted to make sure that we brought reasons why we're saying not raising your hands, back to why we want to serve all of our kids and not just one say, part of our kids. That's the thing we want to do.
So Kyle, one thing that we've been doing is changing the way instead of asking questions that wait for hands to go up, is that we've been trying to engage all kids all at the same time using whiteboards or just paper. But I'll let you kind of chime in here and say some of the reasons or some of the ways we've been doing them.

Kyle Pearce: Again, those who are regular listeners. And we get emails all the time from people who are like, "I've made it through all 100 and now what are we at 115, 16?" I think this is 18 episode, 118 episodes. So for those who have gone through, you already know this, but for those who haven't, going back to episode 21, and I believe it's 98 where we talked to Peter Liljedahl about his research in vertical non-permanent surfaces, really I think is a starting point. Now I know a lot of people might still be in a virtual environment or even now who knows how long this is going to go for where there's hybrid environments going on, who knows? But by starting there, what we gained by looking at Peter's research and starting to apply this into our own practice was just this quick glance around the room. And when we looked around the room, you can see on everyone's whiteboard, who's where. You get a quick assessment.
Now you still have to go and talk to students, observations, conversations. We talk about that all the time. That's going to help you with where they're at, but it gives you a sense of where they are. And John, something that I know you and I both relied on for this understanding. So again, you mentioned Dylan William, his second strategy was eliciting evidence of learning. We did that traditionally through hand-raising, but again, it was like, we only elicited it from those kids who are willing to share. And I don't even know how helpful it was because usually the questions we were asking weren't very helpful. And then even in Principles to Actions, they talk about eliciting and using evidence of student thinking. Well, if you think about that, if I'm looking around this room and I can see where students are at, and if I'm able to notice a name, the progression.
So if I'm thinking about the math class that I'm in, and I'm thinking about that learning goal that we already referenced earlier. And I have already anticipated students solutions, that will help me determine where along this trajectory are students and who might I ask what? So now it brings back this spectrum you're talking about. And on the screen right now, people can see the book Quiet by Susan Cain that you had referenced. If you're on the spectrum of introverted extroverted, some people might be like, are you just going to cold call on a student and say, "Hey, you go ahead and explain this." Depends on the culture of your classroom, I'm going to argue. For you and me, we try to build that in our classroom. And we try to learn which students are comfortable with that scenario and which ones are obviously not feeling good about that. If I have a student who's maybe on the introvert side of the spectrum, and maybe they're not comfortable speaking to the entire group, I can then maybe team them up with a couple of groups of students.
So let's say there's a group over here who's got a similar strategy, but maybe has gone a slightly different direction. Their group over here is here. I could bring them together and I can call on one student to at least start the discussion around a certain portion of their thinking. So again, it's like we're eliciting and we're using this evidence of student thinking in order to advance the thinking of all groups. And it might include us facilitating that to the entire room. It might be us facilitating that with a small group. And I'm going to argue that we can do this in a virtual environment. I know John, you're doing that. For me when I go into classrooms right now, I'm usually there for a day. So I don't know the students as well. So if I pop into a breakout room, it's a little less helpful because it's like the first day of school for me. But for you, tell us a bit more of how this could work even in a virtual or online environment.

Jon Orr: Yeah. And so I had a lot of success with breakout rooms in my virtual remote classrooms when I was teaching remotely. And the idea of the small group that allows the discussion to happen. I know that if you are teaching remotely, especially in high school, when you're asking questions in the large group, Google Meet or whatever software you're using to kind of synchronously present from. You know that your class is going to be silent and cameras off for the most part of that say lecture. Because really what you're doing is you're doing a lecture style lesson if you're kind of doing that from your screen, but when you move to breakout rooms, now you're putting kids in small group settings. All of a sudden the magic happens. The voices will turn on. Kids will start to communicate at a much richer discussion level because they're in a small group setting. The comfort level is a little bit easier.
And all of a sudden you've got that group of kids who were say intimidated from the large group. They're a little bit less brave than some. They're now opening up and discussing. I had way more discussions with kids who are in small group settings versus large group settings, which is normal. That happens when you're in the classroom too. All of a sudden you can hear discussions happening around the room if you are in your, in class set up, instead of out a remote set up, it's natural for kids to kind of talk in a small group setting. And again, when you're popping from group to group, you can hear those conversations. You're eliciting evidence of learning from those kids because you're learning about what they know, what they don't know. And then you're at a great spot to give feedback when needed. It's action-based. Or they're getting to work faster. If you're giving them a virtual whiteboard to work with or on paper, they're going to get to work faster.
Just like if you're in class and asking them to write on the whiteboard in class, the work gets done faster and everyone's actioned. They are doing the work. So instead of like, even if you had to do this lecture-based and you said, "Hey, what's the next step in solving this equation?" And then waiting for hands to go up. If you're like, "Guys, I'm not even at the point where I can have my kids at the walls yet, or I'm just learning. I'm just starting this idea of new learning." The way I morphed it was I had all whiteboards at their desks or even on paper. It's like, you could just ask them to do the step at that time. And then they are doing the action and not you. Kyle went back to Peter Liljedahl. We want them to think in less you thinking, more them thinking.
So when you ask the hands to go up, you're waiting for one kid to answer. Don't even ask for hands to just go up. Instead of saying like, what's the next step? Just say, do the next step and then walk around. And you're going to see so much more. You get more information from that than you do from the one student who is raising their hand or the three students raising their hand. Or when you call upon a kid to say, "You didn't put your hand up, but I'm calling you out." And the kid's like, "Oh, I'm going to guess." At least when you ask them to do it at their desks and you walk around to see, you're seeing who gets it and who doesn't from that. So you get way more information from asking them to do an action than waiting for hands. So these are the strategies that we do instead of asking kids to raise hands.

Kyle Pearce: It's all based on so again, by eliciting student understanding, by eliciting this student learning, this evidence of student learning that we reference, it gives you the information you need formatively in order to ask the next purposeful question. So I want to make sure people don't misinterpret when you say like the next step. It's like, well, what you're saying is based on the purposeful question that I've asked, I want to see, what are you doing? And sometimes this might be, we've talked about focusing and funneling questions as well. And sometimes there's a place for both. So for example, if I'm going around the room and I'm like, okay, I feel good about students doing this idea. Now I might ask a very funneled question around, I want to see, who heard what? Did you get what I think you're getting right now? Because again, if we make assumptions, we don't know whether that's actually true or not.
So you might ask a very focused question to ask and see, okay, what are you going to do with this idea now? And behind me here, I've got that piggy bank unit again, by being able to go and preview and understand the math. So first of all, we know the learning goal, but then also anticipating what students might do. And we try to be helpful with our teacher guides to give you some student approaches that are possible. And we try to do it in a developmental way. Here, you can see a student who used paper folding. Is everybody in the room using paper folding? If, so my next purposeful question might be this. If maybe some students are paper folding, but other students are using a fraction tower. Maybe my prompt is how do I connect these two representations?
So all of these ideas are inside of these problem-based units. And these are the parts that I know that you and I didn't do early in this journey. We didn't realize that this was important. We all thought all students were going to solve it in the same way, but if we're doing this developmentally, if we're doing it through problem solving, that's actually not what should happen. Most likely it's not going to happen. All students are going to kind of come at it from where they are and based on their prior knowledge.
So I just wanted to share that piece because this could be really helpful. If I don't anticipate what students might do, it's very difficult for me to elicit that understanding and elicit who learned what? And then to give them this next purposeful question, that's going to kind of nudge us along and ensure that all students have a voice as well, because that's the other piece that John mentioned. We don't want the Matthew Effect to take over where it's only the three students who solved it the way I thought they were going to solve it. And then we just kind of keep going with that. But all these other students' strategies haven't actually been connected so that they can see that actually you're thinking is a lot like these other three students thinking. It's just coming at it from a different perspective. So there's so much here to unpack John.
I'm hoping as we start wrapping this thing up and we start thinking about kind of our big takeaways here, there were two things that we really wanted to make sure people took away. The first one was this idea of the learning goal. And we want to stop being too explicit about the learning goal early in the introduction or the exploration of a new idea. Now, as we work through that unit, again, we want to make sure that students are very clear on what is going on. We don't want that to remain a mystery. But when we introduce it, that curiosity piece, we want to keep that. So focus on your learning goals around math practices or process expectations. John, what was the second thing that we were hoping to chat about here today?

Jon Orr: Which we just ended talking about, which was the idea of most of us are still probably asking students to raise hands. We want to limit that technique. I know that it might be impossible always to get rid of it, but we want to limit it and reduce it as much as we can, because we can ask to elicit information from students in a better way, by getting them in action, getting them to do thinking. We gave you some suggestions about whiteboards. And if you're remotely breakout rooms, lots of good suggestions, hopefully because that's what we've morphed into doing in our class. We get so much more information from our end, but also have our students think at the exact same time. So those are two techniques that we have used, but we want to say limit those.

Kyle Pearce: Yeah, for sure. And it sounds like two small things, but as you know from this episode, it's two really big things and they will take time and reflection. Now, if you're sitting there and you're going, "Okay, I'm going to really focus in on this." And then you're starting to think about, "Okay, well, if I do these two things, that's going to influence and impact some of my other teaching practices, right? My pedagogical approaches and of course my assessment and evaluation practices." John help people understand, if they're ready to dive deep with these two ideas, but then maybe even start considering other aspects of their assessment and evaluation practices, policies, and essentially just approach and perspective to how they do those things. Where can they go if they want to set this on fire and fuel this learning a little bit further?

Jon Orr: We have created a course on the Make Math Moments Academy that kind of goes deeper into these two. This is actually the two techniques that we're referring to here was one lesson out of a five module course called Assessment for Growth inside the academy. We built this course because we had a lot of questions around, "I've changed my practice. I've been doing using estimation 180, I've been doing number talks. I've been doing open middle problems. I'm getting my students through productive inaudible guys where I've been using your problem-based tasks in my class. But now all of a sudden, the way I've been assessing students, it doesn't flow. It doesn't jive."
So were like, "What are you guys doing for assessment?" And so we built this course to kind of look at all the things that we are doing around assessment and specifically formative assessment and say, moving that towards more formative assessment techniques versus summative assessment techniques. Even though we have ideas on summative assessment techniques in the course, we look at this whole course from a perspective of the whole point of assessment is to help our learners move forward. That's it. Everything is about that. And so we built a course on that, where we talked about modifying your course so that assessment is built into your course instead of trying to fit it in after. And so we talk about setting the stage and saying, I've got to change some practices from the get-go and how do I change mindsets? How do I help my students with that? What are the things I need to set up?
And then we're going to go into some nitty gritty stuff about assessment practices in your classroom. What are the techniques you're using? What are the techniques you should be using? But then also we have like, how do I start to move from modular assessment practices like grading into standards-based grading? Because we morphed that after we started to change the other things in our classroom around assessment, we were like, okay, well a standards-based grading system, when I start to assign grades is way more intuitive. It's way more in line with the way I've been teaching. So we show you how we've changed from modular grading to standards-based grading. And then we show you how we are doing that with portfolios and spreadsheets. We kind of show you our journey and how to do that in your classroom and how to set that up for you.
So there ae our fourth module is like exactly what our growth days look like. Kyle and I have talked about setting up a full day towards assessment and growth on assessment and the policies, what we've put in our classrooms around that. So we show how we've done that, how to set that up in your classroom. And then our final module is about helping our stakeholders understand these changes. Like how do I help the parents understand this? How do I help students understand this? And how do I help administrators and colleagues understand what we're doing here for assessment? Because it's all about growth. So this course is called Assessment for Growth and you can get over there right now by... And you can take module one. Full module One is wide open. You can go to makemathmoments.com/AFG inaudible AFG. That will take you to module one.
And you can enroll in that course for free. Model one is wide open. If you are looking to go further, it is a course inside of our academy. So makemathmoments.com/academy is how you can enroll in the full course and get access to all five modules. And actually Kyle, you can always get 30 days for free inside our academy, which has loads of time to take this full course. It's definitely a doable achievement to get your full course inside those 30 days. So you could get in, do the course and cancel and boom, you just got that full course for free. This is the way we've set up. The academy is 30 days. And after that, you want to keep going with us. We've got lots of learning in there, but if you want to end your learning with us at that point, no problem, no problem at all.
But if you want more learning after that, we've got many other courses in the academy and online support inside the academy on a regular basis. So that's where you can dive deeper in assessment because we've been on a journey and we love talking about this. We love helping people with this because I think it's so important. It's basically how we've built our courses is really around assessment instead of trying to fit it in afterwards.

Kyle Pearce: I love it. John, you did a great job summarizing, and I love that strategy. So we know so many districts are, "Budgets are tight right now." So many districts have spent money on tech for students to ensure equity and access. So we totally get it. So we hope that people will at least take advantage of those 30 days, hop in there, learn what you can learn and then if it just doesn't fit for your district, for the budget. Totally cool. We have tons of people though, who do decide to stay on and continue learning. I think we're now over what, 825 members or so inside the academy. And we do awesome, awesome stuff in there.
So we are super, super excited that we got a chance to dive into this learning with you, friends. Right now, we're going to start signing off and tell you, thanks for listening, hit that subscribe button on your favorite podcast platform. And also keep in mind we've been posting and we're hoping to continue posting video podcast episodes like this one. This is our first attempt. So who knows what it's going to look like on YouTube, but make sure that you head over to YouTube. You smash that subscribe button and hit the notification button because actually John and I have been going live on Facebook a lot and on YouTube a lot. And we've been trying to put little nuggets, like 10-minute nuggets takeaways that you can take and apply right into your classroom. So make sure you go over there, make sure you're subscribing and yeah, we can't wait to see you in one of those learning opportunities.

Jon Orr: Yes, for sure. Show notes and links to resources from this episode and full transcripts from the episode can be found at makemathmoments.com/episode118. Again, that's makemathmoments.com/episode118. We release a new episode every Monday morning, as many of our ongoing listeners know to wake up and Monday morning on the way to work or run their jive or wherever they're doing. They know Monday morning, they can count on us to fuel their sense-making. So looking forward to releasing this one. Kyle.

Kyle Pearce: My friends, I think that's it. So until next time, I'm Kyle Pearce.

Jon Orr: And I'm John Orr.

Kyle Pearce: High fives for us.

Jon Orr: And a big high five for you.