After a recent Global Physics Department meeting about Standards-Based Grading (SBG), I got to thinking about how I would adapt my grading scheme if I taught more than 40 or 50 students per year (which many of the teachers at the meeting were/are doing).
One idea that I had was about doing more of the student initiated assessments during class since, even if you could get 150 kids to quickly and earnestly buy in, it would become pretty unwieldy to meet with all, most, or honestly even half of them every week or two outside of class. These in-class opportunities would have to give students choice over what they want to test, but it would also be impractical for a teacher to be making 150 different quizzes each time. Even if most of the students had many of the same problems on their quiz, putting them together in nearly endless combinations would still take too long to be a sustainable practice.
But, so. My idea* is to offer a menu with a few flavors of test (say, 3 or 4) and let students choose a day or two in advance which test they’d like to have. It’s not so much trouble to make up only 3 or 4 different tests (especially compared to 20, 50, or 150). It would be pretty easy to have kids mark their names down on a sign up sheet as they left class one day, or to throw together a Google form if students were likely to use it. Here’s a quick example of four choices I might offer around this time:
It’s easy to scan ActiveGrade and see what flavors would be useful to a lot of kids at the moment (just look for vertical stripes of trouble). Here’s a snapshot of one of my Honors classes at the moment (well, a bit behind, actually, since I haven’t finished looking at the quiz they took right before they left for break, but still). Click to make it a little bigger.
So without a lot of work, it would be easy to see what the menu choices should probably be and to make up a few quizzes (tests, whatever) along those lines.
A couple of ideas for implementing this kind of menu test on a regular basis:
- Alternate weeks of teacher-chosen tests and student-choice tests. Teacher-chosen tests could still cover any objective (even quite old ones), but everyone would take the same test. Student-choice weeks would have them picking from a menu (of 2, 3, 4… whatever makes sense in the number of choices) created that week.
- 50/50 tests — Students get a choice every week (after the first few when there are too few objectives on the table for there to be any choosing), but they only choose the flavor of the 2nd half of the test. So everyone has the same first page, but they get their choice of a few different 2nd pages.
In my classes, we’ll return from break for just over 2 weeks of classes before the exam. I’m thinking about trying this method of student-choice testing to help my students feel more in control of how they are preparing for the end of the semester and to give them more than the two Sunday test opportunities to choose skills that they want to show me. I think they will like it.
Does anyone have other ideas for low-maintenance in-class assessments (or re-assessments, if you call them that) that can lower the stakes for teachers trying standards-based grading? And let me know if you try something like this in your classes (and how it works, for better or for worse…)!
—
* I’m sure that I’m not the first to come up with an idea like this one.
Physics! Blog! 
This is very timely. I have about 30 students per class, and I had presumed everyone had a similar number of students. I am designing an SBG protocol to implement starting January 4th. I presumed I would be able to tweak the customized assessment systems I read about here and on other websites, but I realize at least two of the best authors of cool SBG related stuff teach at private schools with presumably smaller class sizes.
If I survive this next term, I will have some battle tested systems for grading lots of students at once — or at least some cautionary tales.
I’ll keep checking here in hopes that others post their experiences and ideas for testing larger class sizes.
Have you seen what Frank and John have been doing with having students write on their own quizzes when they finish? I’ve tried it in a few of my classes and students L O V E it. It could also make it a lot faster to get the grading done because if you don’t have time to write on all of them again yourself, you can just score the papers and trust them to give themselves enough feedback based on your answer keys.
Kids serious love this. I mean, even students who I’ve previously seen throw their quizzes in the trash without looking at them as soon as they get them back are actually putting time into figuring out what went wrong with their solution and how to do things correctly.
Some kids like trying to score them themselves (of course I do the official scoring later), too.
And this would be easy enough to do with different flavors of tests, too. Just put the different answer keys in different spots.
It’s a win-win-win for grading time, engagement, and shortening the feedback loop. The only loss is on class time, so I’ve struggled with using it in Honors, but I’m all in with it in my regular classes now.
I did read those two entries and I have them queued up to incorporate in my procedures, but I haven’t considered them in a while… I have a LOT to do. I think I’ll use a single uniform test the first three weeks, then move to a test form with 1/2 current questions and the second half in 2-4 different “flavors” chosen by me based on their “holes” and after a few more weeks, let them ask for questions the week in advance and I’ll prepare a set of 2-4 semi-custom based on demand — I could even proposed study groups based on the requested problems… Someday, I could even prepare custom study packets based on the set of re-test questions…
I’m most concerned right now with annotating the quizzes and tests in the modeling packets with the appropriate learning goals identified for each problem and creating a bit of a question and problem bank that I can expand and modify.
What do you think is the essential toolkit for a first time SBG implementer?
Any suggestions on how to organize, store and deploy all these aligned questions? I remember a screenshot of a Sugarsync folder with all sorts of personalized tests for your students in one of your posts, and I’ve read you mention recording images of your students’ assessments // re-assessments in Evernote… However, I’m still unclear about how you do this and how I’ll do this…
I’m thinking a mail-merge type process might work for some types of problems — with a spreadsheet with different values and ranges… Other problems it would be nice to develop and keep a dozen or so rewordings of a basic concept, designed to flush out common misperceptions… ?Sounds like a job for a simple database?
It sounds like you crack out new questions on demand with little thought required. Maybe I just need to do it, make mistakes, improve, and develop the feel instead of a question bank… Kind of like what I ask of my students…
It’s so easy to get overwhelmed with lots of new ideas. My advice would be to try to imagine how a new policy or idea will play out over the course of the semester (especially imagining how it will look during “crunch” times like the end of a quarter, before spring break, or the last week of school) and see whether it looks like it’s going to become overwhelming. I would also say… it helps to make the basic structure of your grading policy pretty rigid, but leave the implementation a little more flexible. It’s easy to paint yourself into a very tight path with lots and lots of rules, and you want some room to try little shifts out without unsettling the natives.
As far as assembling a bunch of questions… just grab any physics textbook that your kids don’t have at home (so if you aren’t using a textbook in class, then any physics textbook). They have millions (well, not quite millions…) of problems in them. Just take the situation that they give and drop the questions from the end (or leave them, or whatever). Easy. Done. Don’t spend your whole life writing millions more problems. You’ll drive yourself crazy and you’ll never keep up with the kids.
Kids can make their own problems, solve old ones backwards (change what is given and what you’re finding), or cover up old work and try solving anew. Don’t let them demand that you do more work (like getting them new practice problems) before they are willing to do any work (practicing the skills they haven’t mastered). But do teach them the strategies for how to use what’s already in front of them. And let them know that if they search for something like “projectile motion homework” they will find all the problems of their dreams available on the internet for them to practice. Don’t worry about them finding problems from the books. 1) They’ll never remember a problem they saw before online if they don’t know ahead of time they’re going to see that question again. 2) If you still worry, just drop the question asked in the book and ask about something else. Or just make it a goal-less problem and make them ask themselves something else.
If you are going to give questions on the fly (rather than printing out customized individual tests), then put the problems onto notecards or cut them into strips. That way you can just hand them a card, tell them to do that problem, and then collect the card or strip back from them when they’re done. That was super useful to me last year (but not really applicable to what I’m doing this year).
Read all of the posts on Shawn’s blog. And after a month, when you start feeling inadequate about fielding questions / complaints / concerns / strange looks from students or parents or other teachers or yourself, read all those posts again.
If you like Evernote, you could easily put the questions in there. Either as individual notes, tagged with objectives, or group them together by model (it’s okay to have lots of PMPM notes, each with a few problems in them… when it comes time to look for a problem, clicking a tag or just searching for PMPM will bring all of them up and it is easy to look through them all). Make your system flexible enough that you’ll actually follow through on using it as you start adding problems along the way (again, don’t over-do it with the rules). Set yourself up for success by thinking about how many steps you’d have to do to maintain and then use it, and keep it pretty simple.
My problems are sort of all over the place. In old files from previous years, in various notes in Evernote, in a little yellow notebook that I carried around last year, in books as I find them, in my head, and all over the Internet. Somehow that works for me, and I kind of know where everything that I want is located. But that’s probably not the best model to follow…
In terms of selling it… Be open with your students about why you think this is a really important and also really cool idea, why they should think it is a really cool idea. That you are trying it for the first time and know that there will be little bumps along the way, but that you’ll look out for problems and fix them as they happen. In my experience, kids can be really forgiving and usually appreciate explanations about why you’re doing something and acknowledgements about when you make a mistake or don’t do something as well as you could have. But they also have very little patience for hypocrisy (especially when you change the rules in the middle of the game) or when the adult seems like they have no idea what they’re doing. So you want to cultivate this sense of: I have a strong idea about how and why this should happen, and I am also ready and willing to make some minor adjustments along the way because I care about you and am going to look out for you.
Hmmm… my reply is starting to go all over the place. The “essential toolkit for a first time SBG implementer” question is a really good one. I think I need to give it more thought for a more coherent answer.
Wow! Your answer totally resonates with me and gives me heart.
Particularly poignant: students have no patience for hypocrisy and adults who don’t know what they’re doing and students have tons of patience and forgiveness in most other situations…
I’m relieved to read that — and it’s exactly what I experienced this first term teaching. I’m shocked at how genuine students were and how well things went when I was demanding, flexible and open, and how angry they got when they were really confused… because I was confused.
I feel equal parts terrified and sublimely sanguine! The sanguine part comes when I read all the cool stuff you guys are doing and making it sound easy — the terrified part when I’m actually writing and planning step-by-step how I will implement this on my own.
I sure wish there were a better teacher preparation process…
I’m going to follow this thread with interest for several reasons.
1) I just jumped into the “modeling” pool and it is moving soooo sloooooowly that in 17 weeks we got through an introduction to energy pie graphs, the CVPM and through the first multipart worksheet (Rex Rice) for the CAPM – we did the inclined cart demo with loggerpro and the “wheel and axle” lab. But have only had time for 2 formal assessments (outside of our required midterm exam)
2) so with time already proving to be an issue … I am dumbfounded as to how you have time assess once per week. (I’d love to but … )
3) after struggling to create assessment questions that are so heavy in the graph / graphics department … test item creation seems to be a major concern when budgeting time. (I did finally figure out that formatting my entire word doc as a 2-column table with the columns merged for the question and 4 cells for the various representations (verbal, graphical, pictorial, analytical) …
I’ve a lot to learn in this regard
Tim
Hey Tim,
Modeling definitely goes much more slowly than traditional instruction, and once you start asking kids questions and realizing how much they don’t understand, you could basically spend the entire year on the first unit. I definitely sympathize with the struggle to keep things at a reasonable pace. What level of students are you working with? (Age, math level, amount of warm/fuzzy feelings about computational things?) My honors kids (mostly sophomores in Algebra 2 Honors) go about twice the pace of my “regular” kids (mostly juniors in Precalculus). In the regular classes, we’ve covered CVPM, BFPM, CAPM, and UBFPM.
One suggestion for the next time through… do BFPM before you do CAPM. I think it really does make CAPM faster (because they’ve already drawn v-t graphs with slopes and already have a good gut feeling about (a) what the slope represents and (b) why you’d care about doing this unit at all), but it also makes it seem like you’ve covered a lot more even if you cover the same number of units per time. Since BFPM is an explaining model, it feels a lot more (to you and to them) like you’re doing physics than when you spend forever and ever on descriptive models (CVPM and CAPM… and probably also the pie charts you’re talking about).
Re: testing once per week… Many of these tests are < 20 minutes (some 10 minutes). In my Honors classes, I've given a few of them as take-home quizzes to buy back a little class time (I don't assign specific homework, so giving them as take-home tests doesn't eat any homework time away from my pace). I think I have been able to keep such a fast pace in Honors this year without the anxiety and angst that the kids expressed about our speed last year (which was not super justified, given the unbelievable job they did on the exams, but a lot of them did feel like we were way too speedy) because of the test every week. Since we don’t have end-of-unit tests, and since tests every week are now the norm and don’t signal moving on to a new unit, we can keep moving forward when I think they’re ready without them getting worked up about having a test again. I think I just explained that really poorly, but maybe you get the idea.
But, yes. I think moving up the pace comes with more experience doing Modeling. And then cutting down the activities to the essential ones (which is different between my Honors and regular classes, so depends on the students). And being willing to move forward when they get it “enough” even if they haven’t 100% mastered it yet (or even 90% mastered it yet) because you know that they’re going to practice the heck out of those skills a unit or two down the road and will really get it then so long as they have the basics and can draw the graphs reliably.
Okay, I’m going to stop rambling about pace things now.
Kelly.
I’ve thought about doing what you’re thinking about in letting the students choose their own test, but haven’t gotten to it. Making up suitably standards-based tests and quizzes has been time-consuming enough…
How is it any more time consuming than making a normal test or quiz? Are you trying to isolate objectives? I think that’s terribly counter-productive. I never do that unless I have a student in dire trouble that needs to isolate the A objectives and get to a place where they can pass the class (hasn’t happened yet this year… fingers crossed!!). For me, it’s just like making any old kind of test/quiz, but I’m just more thoughtful about hitting particular skills. What’s causing all the extra time spent? Maybe we can brainstorm some time savers! 🙂
I am trying to isolate objectives. Partly because I do have students in dire trouble (given a v-t graph and asked for v at a particular t, many students couldn’t do it…) and partly to make my grading more consistent. I’m one of 6.5 physics teachers in the building and the only SBG-er, so I feel I need to be able to point to particular questions to explain what the students need to work on.
Is there a requirement for taking the class? If students are consistently having trouble even reading values from a graph, it seems like they’re pretty far from being ready to handle much more than that… Unless— I can’t remember whether you teach 9th graders?
They’re allegedly sophomores who passed both algebra and biology as 9th graders, but the prerequisites are ignored all too often. And many of the students have the mentality that they don’t have to remember anything after they’ve passed the unit test. They complain mightily when they find out that physics has a cumulative final…
Kelly,
Great post, as always. I’m thinking about doing this, and I have been doing reassessments on Friday’s with the whole application online/show me your practice thing so that you can reassess thing. My question is, do you want them to show you that they’ve practiced for the in-class student choice assessment? I guess this goes along with the question of whether or not all of the students are required to take an assessment during this period (e.g. if they have all 2’s).
I think this is a great idea, especially for my lower level students who have a hard time coming in for reassessment.
Hey Chad,
For this, I’d imagine that I wouldn’t ask ahead of time to see the practice (as I don’t for any other in-class test). But since they’re choosing which objectives they want to show, if it is clear from their work that they have no idea what to do on this new test, I would probably catch them outside of class and ask to see how they practiced for it (which they probably didn’t) and coach them a bit on how to prepare for the next time.
As far as kids with all 2’s… they’d still have to choose a flavor. I tell them not to think of it as “having” an objective, especially if they’ve only gotten one 2 on it so far. I tell them to only start thinking of it in that way (if they must at all) once they’ve gotten three 2’s in a row. When you’re learning something new, you tend to go back and forth (1 2 1 1 2 2 1 etc) as you’re gaining understanding and consistency. Hopefully it levels off as a 2 eventually! But the first time you kick a soccer ball correctly (play a certain note on the french horn, tie your shoes, etc etc), that doesn’t mean you’ve 100% mastered it, right? It means you’re getting a lot closer and now need to keep trying it to make sure you can do it correctly every time.
But another option for a kid with all 2’s might also be letting them opt for a goal-less problem. I have a few kids who come in for weekend tests like that. I give them a much tougher open-ended problem and let them show me what they can do, then give them feedback. If it ends up with them getting a 1 on some objective they’d shown before, then great! We both definitely want to know that they weren’t completely solid on that skill so they don’t head into the exam practice time unaware. And there are plenty more chances to fix that up, anyway.
What are you thinking?
Kelly.
Thanks for your reply Kelly.
Something that I really haven’t had the guts to do is do whole class assessments with old objectives, that is, I only assess in class on an objective once. How do students respond to this? Also, do you let them know ahead of time what objectives are going to be on all assessments?
I can see that if I had this in place, then reassessing students who have all 2’s with another assessment wouldn’t be anything different. I just don’t think I have that culture in place this year. I’m thinking about trying that with my next batch of students next semester, I’m just curious as to how you get them to buy in. Is it something like, “you’re responsible at all times for every standard…”? I can see this as being easy to justify and kind of gets at the heart of SBG…. kind of sounds like what I’m supposed to be doing!
Trying the “student reflection immediately after the test” thing as we speak.
Yes yes yes yes yes! It is a total given (in my classes) that they will be assessed on everything multiple times. Only the most recent score counts, so every grade is only a temporary grade that will just be updated (for better or for worse) at some point in the future. For most skills you can’t help but assess it more than once anyway (like drawing CAPM graphs which you need to do in order to solve CAPM problems, model UBFPM situations, solve UBFPM or PMPM problems, etc).
Since I think about (and talk about) tests as taking data on the kids, it’s easy to sell it in the same way (and for the same reasons) that you convince them to take multiple trials of the same data point during an experiment. Actually, it’s an even better analogy than that. You want to measure the same thing with a few different questions around the same time (multiple trials of the same data point), but you also want to keep taking data points to see how their understanding varies as time goes on (taking multiple data points to see the shape of the graph).
But then which value do you want to use when it comes time to give a final judgement as part of the grade? Well, you could average all the points and find an average value for the function (which has some usefulness, but doesn’t have the meaning that I want a grade in my class to have). You could do a linear fit and find the slope if you wanted the grade to mean something about the rate of change in their understanding. Or you could use the graph to get a good estimate of the value for the function at the final time (which will be different from the average value in most cases!). I like that one the best (and there are ways to use multiple data points near the end to get an estimate of that value, but I stick with a simpler view of just taking the most recent score at the end of the semester… though I do take a look at the other data points if they bungle something on the exam and see whether this final data point fits their pattern on that objective or not).
Now, of course… no one likes “losing” an objective (I can’t find just the right language to replace that term yet). I would never call it “losing” an objective, but the kids definitely do. Except for the week before the end of the term, though, they just grumble a bit about it rather than having panicky episodes (at least in front of me).
And finally… all objectives are always fair game all year (once we’ve practiced them a bit at least in class). I throw old skills at them all the time. I haven’t had any push-back on that at all. (Actually, they often make comments about how easy a quiz was when it has something on it that’s old… since they have to keep practicing until they get it, they don’t have problems showing me skills that they know they can do).
Also… (okay, I guess that previous point shouldn’t have been “finally”…) since they know I’m going to hit every skill multiple times, it helps a lot with (a) the calm attitude about the temporary scores being temporary and (b) their willingness to take tests with very very recent material on it. They are totally okay with having tests every week because none of them are really going to have any kind of effect on their final grade. (And yet, they still take them really seriously! To the surprise of many non-SBGers.)
Awesome. Thank you.
So here’s my thought for next semester. More frequent (roughly once per week) assessments of the shorter variety to keep myself sane, addressing standards related to the current model. So, if it’s the old Impulsive Force Model, I can easily throw BFPM, UBFPM, and CVPM/CAPM standards on the same assessment even if we’re in the middle of momentum. All of these may or may not come up again next week, but they’ll all probably show up again soon. Ahh, the possibilites…
This will most likely cut down on my Friday before/after school reassessment volume, which is getting ridiculous as the semester gets toward the end.
Crazy stuff O’Shea, keep posting this kind of stuff. Inspiring.
Now on to grading hopefully what is one of my last hour long Grand Ole’ End of Chapter tests. I’m probably going to tell them I’m going to give them another shorter assessment on this when they get back from break!
Thanks a bunch!
A bit pie-in-the-sky, but the problem is begging for a software solution. It should be possible to auto-generate a quiz for each student based on the current grade book and a database of problems tagged with what standards they apply to (and possibly a few other tags like difficulty).
Absolutely! We were talking about such a thing at physics teacher camp this summer, and some people even started working on it (we were picturing a web-based thing rather than individual software), but I think the idea lost the summer momentum once classes started again. I’d really like to see it happen, though, because it would be so useful to so many people. Maybe this coming summer it can get some renewed energy. 🙂
I have a reasonably large collection of questions in ExamView format, which is some proprietary flavor of XML. Anyone have contact with the ActiveGrade folks? Maybe they want to integrate something like this…
Are you using ActiveGrade? Riley’s really responsive! I’m sure something like that is on the list of dream add-ons eventually, though we still aren’t able to refine the view by tag yet, so probably a long way off…
Sadly, no. We use PowerSchool.
We use WebAssign for reassessments. Students can choose which standards they need to reassess and access those standards through WebAssign at school. WebAssign allows us to randomize values which is helpful. I administer these reassessments before and after school one day a week. I may have 40 students in the computer lab working on a variety of different reassessments. This is a software solution that we use that had made this more manageable.
Are they all multiple choice, then? Are your regular in-class tests also multiple choice?
WebAssign had some non-MC options the last time I used it. Unfortunately, it costs $$ and my district didn’t want to continue using it.
Great post! I actually do something like this for finals. I invite students to tell me in advance what learning goals they want on their own personalized final. Of course, not that many students take me up on that offer. So I pick the LGs (aka standards) for most of the students, and can pick common needs to make my test-making easier. This semester it turned out to be about two and a half exams-worth of questions which I would mix and match to make the four-plus exams—tailored to the needs and requests of the individuals.
Works great, and is not all that hard to grade. The upsides are enormous: they’re each working on something that they need, not wasting their time regurgitating what’s easy or being in over their heads. I think everybody finished the exam feeling pretty good.
[…] also considering Kelly O’Shea’s “test menus” for in-class assessments. It sounds fairly easy to manage for a large number of students (which is […]