I don’t do lab reports.

Maybe that’s a strange way to begin, here, but stick with me. I haven’t found lab reports to be useful assessment tools in the past. Instead, I have claimed that our work in the lab is too important to grade. It’s the basis of all of our knowledge. On the one hand, it’s not acceptable to do that work poorly. On the other hand, I don’t want to reduce it to whatever aspects are getting assigned a grade. So instead, we do the experiments to begin a unit. We base our models on what we find in the lab. We learn the process of how to design and do an experiment, and we get better over time. We whiteboard and discuss the results, and we refer back to them often. We practice writing, and especially writing arguments, all the time on assessments. So who needs a lab report?

The other side here, though, is that I am not giving students much individual or formal or consistent feedback about their work in the lab. And maybe that’s not as easy of an omission to dismiss.

Enter the Scientific Abilities

At the start of the year, I posted this document (also pasted below) on my 180 blog. At some point in the past year or two (maybe at a workshop or an AAPT meeting or something), I came across this work by Eugenia Etkina (and others). I tried out some of the Physics Union Mathematics (PUM) curriculum last year in various classes, and although the PUM materials weren’t a total fit for me and my classes, that experience also reinforced my interest in the abilities.

For one thing, thinking about a distinction between an observational experiment (what we do in paradigm labs in my classes to build a model) and a testing experiment (which would come when you have a model in mind, so they would involve a hypothesis) was a huge step forward for me. It’s one of those things that, once you see it, you can’t believe you never thought about before.

So, mostly (gratefully) stealing these from Dr. Etkina, but narrowing and adapting a little for what seemed to make sense for my high school students, I made this document with 6 named abilities.

These abilities have also been the background for an ongoing discussion in my department about how we will revise and restructure the 4-year science curriculum at my school (and the classes themselves within that curriculum).

9th Grade Abilities Work

This year, I teach a one-trimester (about 10 weeks, or about 30 classes) physics course with 9th graders (4 times over the course of the year). For a few different reasons, I’ve chosen to focus on mechanical energy with them (energy storage and transfer, a little bit of simple harmonic motion, and a little bit of mechanical waves).

There’s only so much you can do, both content- and abilities-wise in 10 weeks. And that’s true even before you start thinking about the time necessary for developing class culture at the beginning and the time necessary for assessment throughout. You can’t try to do justice to all 6 of those ideas in 10 weeks.

I decided to focus the 9th grade class on abilities 1, 2, and 4. Longtime readers of this blog might know that Ability 1 (multiple representations) is already the bread and butter of my physics classes. Abilities 2 and 4 (observational experiments and collecting/analyzing data) have also been part of my class, but my work with them hasn’t been quite as strong.

Experiment Objectives

In the fall, the abilities were in the background for me. I was putting together this new-to-me class, and although we certainly worked on multiple representations and doing experiments, I wasn’t explicit with the students about how I was approaching that work. In the winter (trimester 2), I committed to really focusing on abilities in the 9th grade classes and to attempting to assess them.

I focused our simple harmonic motion unit even more on doing several experiments and even less on that particular set of content. We worked a lot on using Desmos for making graphs and doing regressions. Some of them use it a little in their math class, and all of them have school iPads, so it’s convenient for my classes.

The abilities are big ideas and contain overlapping pieces—I wanted something (at least a little bit) grainier to use for giving feedback on their work. For my first try, I came up with these three objectives:

• Make observations—describing in words what is observed without trying to explain
• Identify what physical quantities can be measured and what measurement tools can be used
• Identify which of those quantities can be controlled and what might depend on those quantities
• Formulate experiment objectives based on those possible relationships (in the form, “To find a relationship between…”)

• Identify how to use a tool properly
• Read markings accurately and precisely from the tools
• Think about how to minimize the range and uncertainty involved in each measurement
• Record and represent data in a meaningful way

• Notice and write a statement about a pattern in a set of data
• Graph the data in desmos.com
• Fit a function to the data to show the mathematical relationship

So, with these in hand, I was ready to try to assess students on them. I wanted to be able to do this work individually, not during an experiment when they were working in groups. How can I be sure what each student really understands? How can I keep the stakes low for learning—keeping it separate from judgement? That is, how can I protect the experimental work of the class as something that is still too important to grade?

Writing the Abilities Assessments

Here are the questions from my first attempt at assessing the Experiments objectives.

First up, Experiments.1 (designing an observational experiment). In the back of the room, I set up a long track and a fan cart. While they were working on the test (which also included some energy content objectives, so there was plenty to do before their turn), I took a few of them at a time over to the lab table and wordlessly showed them the fan cart. I ran it a few times across the track using different fan speed settings. If they wanted to see something again, I let them. I let them inspect the cart itself. Surprisingly, they were really satisfied with that experience and didn’t try asking questions about it that I wouldn’t answer for them (during a test).

I structured the page to basically pace out what we do during a pre-lab in class. The annotations section should prompt them to note which of the measurements on their list are things they can choose values for (control themselves) and which of the measurements might depend on those things that they can change. They should eliminate measurements where the values can’t change and/or if they don’t seem to depend on anything else and nothing seems to depend on them. It’s their process for narrowing down their list to figure out what relationships make sense to investigate.

And later on that test, they had this question to test Experiments.3 (analyzing data and finding patterns). I grabbed the data set from a set of physical science materials from AMTA (and hope it is okay to post here since I am not posting the question itself from their materials). It would be easy enough to also make up a set of data, find it elsewhere, or use data from students in a different class.

I wanted to give them data that didn’t connect to an experiment that they had already done because I wanted them to analyze it themselves, not remember what kind of relationship we had found in class. I didn’t get all of the wording right here. It’s a little clumsy, and I don’t talk about intercepts on the graph well. I fixed that a little on later assessments in that class, but it’s still a work in progress.

Of course, it’s not 100% the ideal way to assess analyzing data since it isn’t as authentic when they didn’t take the data themselves. They are going to have a better idea about physical meanings of parts of the graph if they actually interacted with them physical phenomena.

On the front of the test, along with the other things-that-are-silly-to-memorize box, I gave them the set of generic equations that I write on the board for them all the time in class. These are 9th graders, so many haven’t seen or thought about a lot of different functions in detail. They can still recognize a shape and use it to do a fit, so I give them this menu to think about.

They can put all of these into Desmos, too, so they can see the shapes (and don’t have to memorize them) before they make their decision on the shape of their own data.

Finally, for Experiments.2 (collecting data), I kept trying to think about how I wanted to do this authentically, but I couldn’t figure out how to do it well. How can I have students do this individually, at the same time, without being influenced by what others are doing? And if I do that, is that even authentic anyway, since when they actually practice this skill in class, I would never ask them to do it without consulting anyone else? And how, logistically, can I manage that during one class period? As the trimester starting running out of time, I decided to try again in the spring instead of doing it shoddily in the winter.

Some Student Work

I was really excited about and fascinated by their work on these types of questions. It improved over the few weeks we had together, and it helped us have conversations in class while doing other experiments or activities (about how to make better observations, for example).

Here are a few interesting student responses to that first test on the abilities.

Observations

One of the common problems on the first test was students writing things that they couldn’t have actually observed.

Measurements

Here are three different students listing what they think they could measure for the fan cart experiment.

Physical Meaning of Graphs

Here are a variety of first attempts at students talking about the physical meaning of slope and intercept on a graph. “did we learn this in physics??” is an instant classic.

Here’s the Desmos graph that the third student above emailed me.

And here’s a full page response from another student.

Next Steps

I get one more try with a new set of 9th graders starting on Monday, so I will get to keep refining these ideas—finding ways to not over-structure my questions but not over-confuse what I want them to do. I will also get to try to figure out a way to test Experiments.2 that seems fair and useful.

I would like to write some TIPERs-type questions to come at the assessment in different ways. With everything else going on this winter, I basically just gave them the same structures again with different things to observe and different data when they retested. I want to have a broader plan for assessing these, just like any other objective that I test.

Looking forward, I can start thinking about bringing this work more explicitly into my other classes as well, including possible yearlong classes I might be teaching next year where the work can be deeper, more coherent, and reach more abilities.

Another person in my department (a biology/chemistry teacher) has been working on similar ideas with his 9th grade classes, so I am hoping to do more work in having conversations with him and making more connections between what we are doing. As we move forward in the redesign process, I hope it will be something the whole department starts to talk about more, too.