A Further Call to Arms

Entering the Discussion

Two Thursdays ago I jumped in the middle of an ongoing discussion about assessment by posting this. It was essentially a call to arms, a request to join forces in strengthening the quality of our assessments.

Then, just after midnight on Saturday morning Daniel Schneider blew my mind by posting this. If you haven’t read it yet, stop messing around on my blog and get yourself over to his. Make sure you follow all of the rabbit-holes provided in the links. (He wasn’t kidding when he said he was a master aggregator.)

(Update: While I was fiddling with this draft, Daniel threw this down as well. Go ahead and put another educator on the list of people I want to be like when I grow up.)

What’s Next

So where do we go from here? What should we do with all of the interest and enthusiasm surrounding assessment? I see one thing as a no-brainer:

Let’s create a centralized location with links out to quality posts and articles about creating excellent mathematics assessments.

I’m hopeful that a certain master aggregator will lend a hand here. (Update: He will!) Over time we can add more links to resources and even invite members of the so-called better assessments movement to write articles addressing specific topics of need.

Archive or Conversation?

Beyond that, I see two lines of attack: (1) Build an archive; (2) Foster a conversation.

Let me explain.

It would be incredibly valuable to have access to a well-populated, easily-searchable database full of rich assessment questions.

It would be even more valuable—incomparably so—if we learned, as an entire community, to write such rich questions.

In the first case (the archive-building scenario) the focus is more on writing and/or gathering good questions and assessments, and then serving them up in a helpful way. A noble task, to be sure. One I hope others will take up and carry to great heights. (By the way, if that’s you, check out OpusMath, follow them on Twitter, and start uploading like crazy.)

But more than a comprehensive archive of excellent problems with a slick user interface, I think our most essential need right now is to develop an army of amazing assessment authors.

If you agree that the ongoing, teacher-developing conversation is at least as important as the creation of a fantastic archive, then I invite you to join me in the following challenge.

Write An Assessment You’re Proud Of

In a recent email exchange with Daniel, I shared a massive vision I have for creating this group of great assessment writers. He suggested starting with something smaller (something vital, yet attainable) and building from there.

With that in mind, here is my/his/our challenge to ourselves and to you:

I want to challenge the blogotwittersphere to write an assessment they’re proud of. To pick a skill/concept/objective and write a targeted assessment that measures this objective at various depths. Or, if they already have an assessment they’re proud of, to share why they’re proud of it—what is it about this question/this series of questions that makes this assessment meaningful? That finds a way to assess both procedural and conceptual understanding. That gives students an opportunity to exceed expectations. That has an ‘explain’/’justify’ component. I don’t know if these things are possible for all skills and objectives, but this is why I want others to be thinking about it too.

The end goal is: create an assessment you’re proud of in terms of format or questions or depth or all of the above, and explain why you think this is something worthwhile.

(I tried to recast his challenge in my own words, and realized I was better off stealing the thing wholesale, with his permission, of course.)

Where Do I Sign Up?

If you’re interested in playing along at home, start with a one-minute survey. Then hop on the Twitter and spread the word! The more voices we have in the conversation, the better we’re all going to get.

What You Can Expect From Me

Once this blog post goes live, I’ll do the following:

  • Set up betterassessments.wordpress.com (the “home” for the better assessments conversation)
  • Gather already-existing resources together in an Assessment Authoring Boot Camp section of the blog (Daniel Schneider has agreed to lend a hand; additional volunteers welcome)
  • Put out an official “call for assessments” over a two week (?) period in the near future, including submission guidelines
  • Feature a small number (one to three?) of these assessments each week as guest posts on the blog (including the “explain…” bit from Daniel’s challenge), and invite the “better assessments” community to offer feedback, constructive criticism, etc.

As with any group project (and I hope this turns into a massive group project), better ideas will come as soon as brains other than my own start their wheels-a-turnin’. I’m not married to any of the details sketched out above, so long as we find a way to establish an ongoing and positive conversation about assessment.

The Dailyness is the Key

I feel I should explain the last bullet under “What You Can Expect From Me.”

Let’s dream big. Imagine that 100 math and science teachers from all sorts of different grade levels and courses submit assessments when the call goes out. The assessments range from decent to amazing, and we’re all stoked because 100 people (100 people!) played along with this little experiment and we have heaps of assessments to look at and learn from.

If we’re not careful, we’ll squander most of the opportunity for conversation presented by 100 such submissions. My intention is to highlight a few assessments at a time so that busy teachers (that’s us!) have an opportunity to dig into each assessment in depth, over the course of an extended period of time. (Whether or not my suggested approach will achieve the goal is open to discussion.)

I think we’ll experience the most growth as a community if we employ a slow-and-steady approach, rather than go after this all at once. It’s the reason many of us find blogs and Twitter more powerful tools for sustained professional growth than fantastic-but-isolated conference experiences once every year or two. The dailyness of our practice is the key to our growth.

What To Do With a Head of Steam

If this project isn’t dead in a few months, then I’ll share some details of the bigger vision I have for this assessment conversation. It has to do with recruiting and organizing people at various levels and in various subjects into assessment-writing cohorts. They’re exciting plans (at least to me), but possibly unrealistic. In the coming weeks I’ll invite some of you to tell me what has potential, what’s a waste of time, and what needs tweaking to become realistic.

Some Closing Thoughts

I think multiple-choice questions are generally inferior to free-response questions. I also think that both styles of “one-off” question are completely inferior to well-crafted performance assessments. However, I also believe that poorly-written MC and FR questions are inferior to well-written MC and FR questions. With that in mind, I think it’s entirely appropriate to allow MC and FR questions into the discussion at betterassessments.wordpress.com.

With that said, I think the MC/FR/Performance Assessment classification of assessment questions isn’t as helpful as the conceptual/procedural/synthesis approach described by Daniel here and here. I certainly like this latter three-part structure better than then conceptual/procedural/application framework I’d been mulling over prior to reading all these great blog posts on assessment.

Comment Time

Heart beating with uncontrollable excitement? Bored out of your mind and wondering how you made it to the end of another lackluster post? Have a suggestion? A critique? An idea? Drop a line in the comments and keep the conversation rolling.

Comments 20

  1. In all likelihood it won’t be a daily discussion. But I do hold out hope for a weekly discussion of sorts. In any case, the ongoing nature of the discussion is key for me.

    Here’s one way it might play out:

    Suppose a handful of people (including you and me) submit assessments sometime in the near future. The submissions include the actual assessment, as well as a detailed description of what we had in mind, what we consider to be the strengths of the assessment, what we aren’t sure about, what we want specific feedback on, etc.

    Soon after the submissions come in we post one of them (say, yours) on the blog, let’s say on a Monday. My hope is that through the rest of the week, or at least during part of it, a discussion arises (in the comments) focusing on this one assessment. Strengths, weaknesses, lessons to be learned, and so on. Depending on the course of the conversation, the author might revise and repost the assessment at the end of the week with a commentary on the changes made and the rationale behind each one.

    Then on the following Monday, we post another assessment (say, mine) and open the floor for a new discussion.

    I think the process of writing assessments, giving and receiving criticism in a constructive manner, and revising assessments in light of others’ insights will be a healthy practice as we strive to develop our understanding of what a quality assessment is and grow in our ability to write such assessments.

    And if this can happen on an ongoing basis (daily, weekly, or whatever) then I think we’ll be that much better off because of it.

    Focusing on one assessment at a time (or at least not more than a couple of assessments at once) may allow each person/assessment to get it’s fair share of the community’s attention. Otherwise, I’m afraid a lot of the assessments will get lost in the shuffle, and we’ll only focus on assessments from people whose names we recognize. Plus, if we dump all the assessments on the blog at once and invite everyone to discuss everything all at the same time, I imagine the discussion will flare up and then burn out pretty quickly.

    I hope I answered your question somewhere in the midst of my rambling reply. Remind me next time not to reply from my phone. It seems that I go on and on and on when I can’t easily see the beginning of my response on the screen. 🙂

  2. Love that you’re linking to Mathy and I have commented a few times over there already. Crazy week has kept me on the sidelines for a few days. I do have one concern and maybe I just need to either change my status or accept being kept out. My concern is this – I don’t tweet. I don’t have a smart phone (can’t see affording the data bill on top of the rest of my life’s charges.) Is there an efficient way to engage in the twitter world without such a device in my pocket? I may be displaying a pretty fundamental ignorance of this world here, but it strikes me as a portable device driven communications engine. Is there an efficient way to engage without such a device in my pocket? I’d love some helpful tips and would love to hop on this train of sharing assessment ideas and examples.

  3. Thanks for the comment. I hope you’re able to join in the discussion, and I don’t think it’s a smartphone-required sort of conversation. Two thoughts on Twitter: One, if you have unlimited text messages then you can engage in Twitter-based conversations that way. I think it gets unmanageable pretty quickly (before I got a smartphone, I followed like two or three people on Twitter using a non-smartphone, and that was about all I could handle), so that may not be a viable option.

    A better option might be this: Create an account, download Tweetdeck (Mac, PC, or browser-based: http://tweetdeck.com/) and follow/participate in the conversation that way. It won’t be as real-time as with a smartphone (unless you keep your computer in your pocket), but you can certainly keep tabs on things and chime in daily or a few times a week by checking in from your computer.

    Hope this helps! We’d love to have you be a part of this assessment conversation!

  4. Michael

    My situation with the phone is even worse than I let on, I guess. I don’t have a text plan for my phone at all. I still communicate with my voice on my phone. I can examine the laptop version, I suppose.

  5. This is a REALLY thoughtful way to manage this conversation. One observation, though, and I am interested in hearing the opinion of others. It seems that an inherent difficulty lies in understanding context. Without knowing course/curricular sequence, text information, classroom background, etc. it may be hard to make meaningful, informed observations about an assessment. It seems that context is HUGE here. I know this can be overcome – I just don’t know if I am smart enough on my own to see how.

  6. I think you’re absolutely right about context being important, and difficult to quickly and/or effectively convey to others. I’ll think about how to address this, but defer to others’ comments for the time being. I’d love to hear how other people think we could work through this challenge.

  7. Pingback: Another Day In… Honors Precalculus with Trigonometry | Reason and Wonder

  8. Pingback: Balancing Assessment/Pedagogy and SBG | Right Brained Math

  9. Michael

    What’s the current status of the assessment project idea? I checked the page this morning and it looks like it has not gotten off the ground yet.

  10. Pingback: Better Assessments: It’s Time to Begin | Reason and Wonder

  11. Thanks for the extra motivation to get this thing moving. Check out my latest post for details about how to submit your assessment. I’m excited to get the conversation started!

  12. I put assessment on back burner to get through a difficult month at school, but I think it’s time to start this in earnest. Check out my latest post for more info. Thanks for the added motivation to begin.

  13. Mrs Reilly

    First, thanks for sharing! I have a few questions and I think that they are largely contextual ones.
    For questions 12 and 13 – what types of answers would you accept/reward with credit? I can easily see my more smart-aleck students offering 1 * x^5 and ((x^2)/(y^2))/1 as answers here. My guess is that your students know some parameters for these type of questions already.
    How long do you allot for quizzes? I try to keep mine in the 15 – 20 minute range. What is your target time for an assessment like this?
    Questions 14 – 18 seem a bit out of place with the context for the first part of the quiz. Is this review here?
    Finally, I LOVE question 19. Great way to get students to explain their thinking. I’d love to know what kinds of answers you got.

    Thanks again!
    Jim

  14. Thanks for reading the quiz!
    1) For the make-your-own problems, I didn’t give them any parameters or rules. We have done these before, so maybe they just *know* what I expect and not to be a smart-alec. Or perhaps they aren’t bright enough to determine a smart-alec response, they just try to make up a problem like something we have done in class.
    2) This quiz took about 20 min, about a half period. We have 42 min periods.
    3) Yes – #14-18 are a bit disjointed in terms of content! This is how our textbook is aligned, so I went with it. The book reviews exponent rules then operations with polynomials — leading up to factoring higher-order polynomials and eventually synthetic division, etc.
    4) #19…ahhh, very eye-opening. Most were absolutely stumped. I thought the clue that there were two solutions would lead them to think about factoring, but most didn’t get anywhere. Here’s what most did: they tried to solve it like they would a one-variable equation in Alg1. They moved the constant to the other side of the equation, giving x^2-x=42. Then they were stumped. I think a solid year of Alg1 equation manipulation trumps our one month of solving quadratics.

  15. Curious if anyone followed that process of moving the 42 by then finding a common factor. I could see a relatively resourceful Alg II student recognizing that if x(x-1) = 42 that I would just need consecutive numbers that are factors of 42. Did anyone go down that path?

  16. No. At that point they were stuck. Most tried to add the x^2 and the -x to something they could work with, like 2x. Others ignored the -x and took the square root. Several tried to complete the square, which would have worked, but they got into fractions and faltered. I may look more closely at these and add to mathmistakes website.

  17. Pingback: An Overdue Beginning | Better Assessments

  18. Pingback: How To Play the Better Assessments Game | Better Assessments

Leave a Reply