Assessing Scientific Abilities

I don’t do lab reports.

Maybe that’s a strange way to begin, here, but stick with me. I haven’t found lab reports to be useful assessment tools in the past. Instead, I have claimed that our work in the lab is too important to grade. It’s the basis of all of our knowledge. On the one hand, it’s not acceptable to do that work poorly. On the other hand, I don’t want to reduce it to whatever aspects are getting assigned a grade. So instead, we do the experiments to begin a unit. We base our models on what we find in the lab. We learn the process of how to design and do an experiment, and we get better over time. We whiteboard and discuss the results, and we refer back to them often. We practice writing, and especially writing arguments, all the time on assessments. So who needs a lab report?

The other side here, though, is that I am not giving students much individual or formal or consistent feedback about their work in the lab. And maybe that’s not as easy of an omission to dismiss.

Enter the Scientific Abilities

At the start of the year, I posted this document (also pasted below) on my 180 blog. At some point in the past year or two (maybe at a workshop or an AAPT meeting or something), I came across this work by Eugenia Etkina (and others). I tried out some of the Physics Union Mathematics (PUM) curriculum last year in various classes, and although the PUM materials weren’t a total fit for me and my classes, that experience also reinforced my interest in the abilities.

For one thing, thinking about a distinction between an observational experiment (what we do in paradigm labs in my classes to build a model) and a testing experiment (which would come when you have a model in mind, so they would involve a hypothesis) was a huge step forward for me. It’s one of those things that, once you see it, you can’t believe you never thought about before.

So, mostly (gratefully) stealing these from Dr. Etkina, but narrowing and adapting a little for what seemed to make sense for my high school students, I made this document with 6 named abilities.

lreiscientificabilities

These abilities have also been the background for an ongoing discussion in my department about how we will revise and restructure the 4-year science curriculum at my school (and the classes themselves within that curriculum).

9th Grade Abilities Work

This year, I teach a one-trimester (about 10 weeks, or about 30 classes) physics course with 9th graders (4 times over the course of the year). For a few different reasons, I’ve chosen to focus on mechanical energy with them (energy storage and transfer, a little bit of simple harmonic motion, and a little bit of mechanical waves).

There’s only so much you can do, both content- and abilities-wise in 10 weeks. And that’s true even before you start thinking about the time necessary for developing class culture at the beginning and the time necessary for assessment throughout. You can’t try to do justice to all 6 of those ideas in 10 weeks.

I decided to focus the 9th grade class on abilities 1, 2, and 4. Longtime readers of this blog might know that Ability 1 (multiple representations) is already the bread and butter of my physics classes. Abilities 2 and 4 (observational experiments and collecting/analyzing data) have also been part of my class, but my work with them hasn’t been quite as strong.

Experiment Objectives

In the fall, the abilities were in the background for me. I was putting together this new-to-me class, and although we certainly worked on multiple representations and doing experiments, I wasn’t explicit with the students about how I was approaching that work. In the winter (trimester 2), I committed to really focusing on abilities in the 9th grade classes and to attempting to assess them.

I focused our simple harmonic motion unit even more on doing several experiments and even less on that particular set of content. We worked a lot on using Desmos for making graphs and doing regressions. Some of them use it a little in their math class, and all of them have school iPads, so it’s convenient for my classes.

The abilities are big ideas and contain overlapping pieces—I wanted something (at least a little bit) grainier to use for giving feedback on their work. For my first try, I came up with these three objectives:

Experiments.1 (Design) I can design (or help design) an observational experiment.
  • Make observations—describing in words what is observed without trying to explain
  • Identify what physical quantities can be measured and what measurement tools can be used
  • Identify which of those quantities can be controlled and what might depend on those quantities
  • Formulate experiment objectives based on those possible relationships (in the form, “To find a relationship between…”)
Experiments.2 (Measurement) I can use available equipment to make measurements.
  • Identify how to use a tool properly
  • Read markings accurately and precisely from the tools
  • Think about how to minimize the range and uncertainty involved in each measurement
  • Record and represent data in a meaningful way
Experiments.3 (Analysis) I can analyze data and identify patterns.
  • Notice and write a statement about a pattern in a set of data
  • Graph the data in desmos.com
  • Fit a function to the data to show the mathematical relationship

So, with these in hand, I was ready to try to assess students on them. I wanted to be able to do this work individually, not during an experiment when they were working in groups. How can I be sure what each student really understands? How can I keep the stakes low for learning—keeping it separate from judgement? That is, how can I protect the experimental work of the class as something that is still too important to grade?

Writing the Abilities Assessments

Here are the questions from my first attempt at assessing the Experiments objectives.

First up, Experiments.1 (designing an observational experiment). In the back of the room, I set up a long track and a fan cart. While they were working on the test (which also included some energy content objectives, so there was plenty to do before their turn), I took a few of them at a time over to the lab table and wordlessly showed them the fan cart. I ran it a few times across the track using different fan speed settings. If they wanted to see something again, I let them. I let them inspect the cart itself. Surprisingly, they were really satisfied with that experience and didn’t try asking questions about it that I wouldn’t answer for them (during a test).

Experiments1

I structured the page to basically pace out what we do during a pre-lab in class. The annotations section should prompt them to note which of the measurements on their list are things they can choose values for (control themselves) and which of the measurements might depend on those things that they can change. They should eliminate measurements where the values can’t change and/or if they don’t seem to depend on anything else and nothing seems to depend on them. It’s their process for narrowing down their list to figure out what relationships make sense to investigate.

And later on that test, they had this question to test Experiments.3 (analyzing data and finding patterns). I grabbed the data set from a set of physical science materials from AMTA (and hope it is okay to post here since I am not posting the question itself from their materials). It would be easy enough to also make up a set of data, find it elsewhere, or use data from students in a different class.

I wanted to give them data that didn’t connect to an experiment that they had already done because I wanted them to analyze it themselves, not remember what kind of relationship we had found in class. I didn’t get all of the wording right here. It’s a little clumsy, and I don’t talk about intercepts on the graph well. I fixed that a little on later assessments in that class, but it’s still a work in progress.

Experiments3

Of course, it’s not 100% the ideal way to assess analyzing data since it isn’t as authentic when they didn’t take the data themselves. They are going to have a better idea about physical meanings of parts of the graph if they actually interacted with them physical phenomena.

On the front of the test, along with the other things-that-are-silly-to-memorize box, I gave them the set of generic equations that I write on the board for them all the time in class. These are 9th graders, so many haven’t seen or thought about a lot of different functions in detail. They can still recognize a shape and use it to do a fit, so I give them this menu to think about.

Common Physics Relationships

They can put all of these into Desmos, too, so they can see the shapes (and don’t have to memorize them) before they make their decision on the shape of their own data.

Finally, for Experiments.2 (collecting data), I kept trying to think about how I wanted to do this authentically, but I couldn’t figure out how to do it well. How can I have students do this individually, at the same time, without being influenced by what others are doing? And if I do that, is that even authentic anyway, since when they actually practice this skill in class, I would never ask them to do it without consulting anyone else? And how, logistically, can I manage that during one class period? As the trimester starting running out of time, I decided to try again in the spring instead of doing it shoddily in the winter.

Some Student Work

I was really excited about and fascinated by their work on these types of questions. It improved over the few weeks we had together, and it helped us have conversations in class while doing other experiments or activities (about how to make better observations, for example).

Here are a few interesting student responses to that first test on the abilities.

Observations

One of the common problems on the first test was students writing things that they couldn’t have actually observed.

Observations1

Observations2

Measurements

Here are three different students listing what they think they could measure for the fan cart experiment.

Measurements1Measurements2Measurements3

Physical Meaning of Graphs

Here are a variety of first attempts at students talking about the physical meaning of slope and intercept on a graph. “did we learn this in physics??” is an instant classic.

PhysicalMeaning1PhysicalMeaning2PhysicalMeaning3

Here’s the Desmos graph that the third student above emailed me.

DesmosGraph

And here’s a full page response from another student.

Exp3page

Next Steps

I get one more try with a new set of 9th graders starting on Monday, so I will get to keep refining these ideas—finding ways to not over-structure my questions but not over-confuse what I want them to do. I will also get to try to figure out a way to test Experiments.2 that seems fair and useful.

I would like to write some TIPERs-type questions to come at the assessment in different ways. With everything else going on this winter, I basically just gave them the same structures again with different things to observe and different data when they retested. I want to have a broader plan for assessing these, just like any other objective that I test.

Looking forward, I can start thinking about bringing this work more explicitly into my other classes as well, including possible yearlong classes I might be teaching next year where the work can be deeper, more coherent, and reach more abilities.

Another person in my department (a biology/chemistry teacher) has been working on similar ideas with his 9th grade classes, so I am hoping to do more work in having conversations with him and making more connections between what we are doing. As we move forward in the redesign process, I hope it will be something the whole department starts to talk about more, too.

 

13 thoughts on “Assessing Scientific Abilities

  1. I’ve had some success assessing the ability to collect and analyze data using PhET simulations. Not anywhere near as formally as I should, but it’s harder for the kids to watch their peers on a laptop than when they’re determining spring constants with an actual spring, for example.

    1. I thought about doing something like that, but I also kept thinking about some of the students I have who might be challenged more by problems on paper, but who have excellent ideas about taking data and dealing with something physically in front of them. I also think that to do a good job of assessing it, I would need to see them doing the work. I’m really stuck on this one right now. PhETs (and DMVs) are a good idea in the mean time, I agree! I will try that this trimester.

  2. Thank you for posting this! I also have not done lab reports this year (for the first time)….in part because I already have so much else to do, to grade, etc.; and in part because as I am working on my modeling instruction and also making the switch to SBG, I wasn’t sure how lab reports would really fit in as an assessment in my class(even after looking at Frank Noschese’s lab standards).

    I like this idea and am thankful you posted now, as I will have time to work through my own classes with these new ideas over the summer. I have been feeling like I was somehow “shorting” the kids by not doing lab reports, but at the same time knowing that their experience has been so much richer this year after really working on my implementation of the modeling (last year I was trying to model but hadn’t been to a workshop, so it was piecing together what I could learn from blogs like yours and others).

    So thanks!

    1. One of the other reasons I stopped doing lab reports is because one of the reasons I hear for using them is that they need to practice it for college. And my experience in college was that the kind of writing we were asked to do was so different from what usually looks like a lab report in high school. So I don’t buy that one. And I think the best preparation for doing the college-type writing is more practice writing thoughtful arguments—which we can practice in a less stressful way (and more frequently on a smaller scale). Or at least, that’s part of my justification to myself! 🙂

      Another option that I’ve thought about, but haven’t really done, is having the kids take a photo of their whiteboard after a lab and then submit the photo along with a paragraph explanation of the relationship we determined and how we decided on it (basically a summary or “report” on our whiteboard discussion rather than a “lab report”). That could be easy to do in Google Docs and could be graded for completion (or just not graded at all, really).

      Glad to hear your classes are going well! So exciting! 🙂

      1. Hi Kelly. Thanks for the post. Though I’ve been teaching physics for many years, I’m somewhat new to modeling. My junior class spent lots of time whiteboarding this year (they love it and so do I!) and always took photos of their work. As you suggested, they placed the photos in a google doc and wrote either a “conclusion”, essentially an argument justifying the relationship(s) determined based on the evidence or occasionally an abstract, where they gave a very brief statement (1-3 sentences) about the lab and method, and then went on to develop their arguement. It allowed me to give them more individual feedback without overburdening them with tedious reports. So far, it’s been successful and their writing is much more meaningful than what I’ve received from full lab reports in the past.

  3. I absolutely love this post. I have been thinking about this idea for years, and planning on doing this all year. I wrote objectives (some inspired by ISLE and PUM) and made a tentative plan, but I haven’t been able to put it together. I even talked about it with Tiffany Taylor last summer. As usual, you are out in front and showing me the way!

  4. Wow, you seem like an amazing teacher! I am inspired by this approach to assessing students, and I love your focus on short answer (for lack of a better term) questions that require analysis and not just regurgitation. So many ideas!! This post makes me want to A) get back into teaching, and 2) retake Physics so that I can actually understand it.

    1. Aww, thank you!! 🙂 I agree that teaching is amazing. And teaching physics is also just obviously much better than teaching math, of course. 😉 Since the last time we talked about teaching, I’ve met so many math teachers (online and in person) that you’d be interested in hearing about, too. There is a lot of collaboration, thinking, etc happening online now. It’s a whole awesome community. Here’s one place to start learning more about (some of) it: https://exploremtbos.wordpress.com/overview/

  5. Hi Kelly!

    I have came across your blog this year and have become very interested in modeling (second year teaching 9th grade physics). I am also interested in attending the high school physics teacher camp in Sacramento this summer. Can you tell me more? I teach at a modified year round school and our first staff day back is Monday July 18th. Before presenting this to my administrators, I am looking for more information on what I might expect/experience.

    Thanks for sharing your adventures!

  6. Thanks for posting this Kelly! Like some of the comments above, I’ve been toying around this idea for years have come up with a set of objectives gleaned from the Modeling framework (aka what might the lab objectives look like from a Modeling lens…). The way you structure it lends itself to SBG as well.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s