Meta-assessment

The other day I wrote a quick blog post from school, making noise about assessments and a new format and the like.  As I was finally assembling the tests, I was having second thoughts.  It wasn't a grammar-and-vocabulary test, which is what I think of by force of habit as a test.  This assessment was only communicative skills--listening, speaking, reading, and writing.  I keep telling students that I don't care if they memorize vocabulary lists or whether they can conjugate verbs or not, that what matters to me is their ability to communicate.  In the first year class, the emphasis is strongly on the comprehension skills, with production being fairly limited in length and scope.  This is in line with language acquisition theory, which says that comprehension will develop before production, and will always occur at a higher level.  It's also in accordance with the ACTFL national standards and the corresponding levels of performance for Novice-Mid to High.

But my tests haven't really reflected that--until this year, the tests have always been (1) listening comprehension (2) reading comprehension with a cultural trivia component (3) grammar and vocabulary sections.  Largely, this is because I took as much of the text-book-provided tests as my students could reasonably do in a day, copied them off and stapled them together.  But I've started redesigning my unit plans the way they're supposed to be designed (see here and here)--which means my assessments needed to be re-written to match learning goals.  And if I don't care if they can memorize vocabulary words, I shouldn't test them on memorizing vocabulary words.

I've hesitated to do this, for three reasons.   1.  It marks a dramatic departure from what I think of as a Spanish test, and I had a hard time wrapping my head around it.  2.  I was worried that a sampling of communicative tasks would overestimate students' abilities to use the language comprehensively.  3.  I wasn't really sure that I was good enough at communicative assessments and using rubrics to assess communicative ability to create a reliable assessment.

I've learned to live with (1) in other contexts--the job of professional educator is not at all what I thought it was.  It's a great deal deeper, more exciting, and science-based than I expected it to be.  (If I'd known what my job was actually going to be, I would have taken a lot of laboratory science and social science classes, and not, for example, Astronomy, Ocean Systems, or Health and Well-Being.)  So, I'm just kind of getting used to the idea that almost everything I thought I'd be doing is the wrong thing, on some level, to do.  (Still trying not to throw out the baby with the bath water...)

Having run a couple of tests, I can now address (2) fairly accurately.  When the learners are participating in good faith in the assessment, their communicative performance gives at least as accurate picture of their language skills as the previous test formats.  And, of course, it has the added advantage of, y'know, actually testing what I want them to know.  So we can pretty well set (2) to bed.

Concern (3) remains a concern.  I'm not sure that this first communicative-based exam I gave really gets to the heart of the matter.  But neither did what I was doing before, and change and learning have to start somewhere.  I've changed, and now I'm learning.  A greater sampling of communicative tasks is probably in order, although the format is about right.  My primary concern is with the speaking/listening section.  The first time I tried this, I asked the students a question, and graded them on how well they answered it.  They then had to ask me a question, and write down the answers.  It was the first chapter test of a high school Spanish I class, so an open-ended question probably would have been too much.  But that section as it stood was much more reactive than creative, I think.

The other part of that is the rubrics I'm using.  Given the nature of a limited testing scenario, I don't think I can break the comprehension skills up the way I did.  So, I'll be looking for another way of doing it, so that the comprehension grade really tells the learners what they need to know in order to improve.