Doug Johnson

Getting What You Ask For
Head for the Edge, Jan 1999

I’ve discovered a great technique for getting what I want for Christmas. I describe the hoped-for gift – precisely. I’ve learned that by simply asking for a tie, heaven only knows what I’ll receive. If I ask for a red and gray tie, my chances improve. But, if lead my daughter by the hand to the Jerry Garcia’s at the local department store and “ooh” and “ah” over one or two, I am pretty sure to get something to my taste.

As educators begin to work with students on performance skills that cannot be evaluated by standard paper and pencil tests, the ability to write an assessment instrument that clearly articulates a desired quality level becomes critical. Whether in the form of a rubric, a checklist, or a benchmark, creating tools that describe what is expected of learners can help educators dramatically improve instruction.

Library media specialists have a leadership role in implementing these new forms of assessment. Having had experience with “project-based” learning, we can use our experiences to teach teachers effective means of evaluating performances and projects through both inservices and by modeling the assessments of joint library/classroom projects. Our media specialists and teachers are becoming increasingly proficient at writing good assessment instruments. Here are some of their “secrets:”

1) Describe what you want in observable terms  
Remember the tie analogy? The more specific you can be with the indicators of quality, the easier it will be for students to determine the quality for themselves. A hypermedia stack about a historical period might include checklist items such as:

• the location and the years
• proper clothing
• correct transportation
• tools and weapons
• people doing their daily work
• key events. What happened that was so important that we’re still studying it today?
• main geographical features
• symbols (religious, job-related or holiday)  that were important to the people in your region
• important or famous people, sayings or documents
  

2) Two strands: content and container.
Remember getting back English papers that had two grades: one for content and one for mechanics? Projects that use technology to help communicate the content really need two separate sets of assessment criteria – one for the content and one for the electronic container of that content. Whether it is a videotape, a hypermedia stack, an electronic slideshow, a word processed document, a desk-top published brochure, a spreadsheet, or a database, an assessment tool that describes the effective use of the container needs to be developed. Quality container criteria for the hypermedia stack above might include:

• a minimum of eight cards, each with a uniform background and layout style
• easily seen and understood navigation buttons
• a logical organization and structure for the stack
• readable text
• graphics, sounds and movies used to add to the understanding of the topic
  

3) Use examples of past high quality work.
Using past student work, students need to see or read actual examples of quality. The “critical elements,” as Mankato media specialist Kathy Wortel describes them, need to be listed. One of the dangers that using examples presents is that students may be tempted to copy the examples too closely. One way to prevent this is to change the assignment enough that this becomes impossible. If a research assignment looks at the attributes of effective leaders, one year ask students to choose scientists as subjects, the next year social activists. If geographic regions are being studied, questions one year can be about environmental issues, the next year about the effect of geography has on living conditions.

4) Give criteria to the learner at the time of making the assignment.
Assessment tools need to be shared with students at the time the assignment is given, not after it is complete. That way students have a roadmap to follow as they work on the project. The goal should be: NO SURPRISES. Here is the task. Here are the quality indicators. Go to it.

5) Use the assessment tool to help guide revisions.
Jean Donham at the University of Iowa reminds us that the term assessment has its roots in a Latin word that means “to sit down beside.” One of the great philosophical differences between doing an assessment and an evaluation, is that an assessment is a tool which encourages continued growth rather than simply judging a completed task. The assessment tool should be able to help students see where they are strong and where they can improve. And by using the these tools while the project is being completed rather than simply when it is completed, such growth and improvement can be actively encouraged.

6) Use multiple assessors.
The best checklists I’ve seen have places for input from multiple sources. The teacher, of course, should comment whether a quality indicator has been met, as should the student. The media specialist can add his or her own unique perspective. Parents should be given the opportunity to review with their children the progress of their work. And in special cases, experts in either the subject of the research or the use of the media can provide insights unavailable elsewhere.

7) Revise your tools each time they are used.
No assessment instrument is perfect the first time it is used. Criteria can be unclear. Too many indicators might restrict creativity or originality. We have found and eliminated nearly all uses of superlatives (good, better, best) in creating rubrics.. The terms are empty without precise descriptors of what actually makes something “better” than something only “good.” Keep your assessment tools in digital format, a word processing document or database, for easy updating and reuse.

Writing good assessments take time, practice, and thought. And this includes not just instruments that measure student performance, but tools that measure the quality of programs and of professional performance, as well. The more experience we as educators get in articulating what we hope to get, the better chance we have of getting it. Remember Johnson’s Law of Assessment: You’ll only get what you want if you can describe what you want. And that applies to both Christmas ties and student performance.