17 December, 2009

Assessing Student Performance in Online Classes

Lisa Lane posted some of her experiences about assessment in online environments that are very relevant to my classes.  Assumptions made by her students are pretty much the same ones mine have:

  • Anything called a test or quiz is important.
  • Anything called discussion you can miss.
  • The textbook is the heart of the class and should be read carefully.
  • Anything called an assessment could go one way or another.
  • If it’s online, it’s self-paced and you don’t need to “show up”.
  • All online classes are in Blackboard.
Lisa is particularly concerned about the second point because like me, she wants students to extend their understanding of the topic at hand through discussion in forums.

In terms of course design, I don’t consider the discussion 20% of the course, just 20% of the grade. It’s more like half the class, because it’s the processing and sharing of the knowledge learned via presentation and reading. It’s the heart, not a side activity. It’s lower stakes (not 50% of the grade)  because I want the students to feel free to explore.

This seems simple enough, but my experience corroborates Lisa's - the students just don't get it.

I have found a way to resolve this problem, at least partially. From the beginning of my courses I make it clear that grades will be based on summative assessment only, and that all other activities are formative and for that reason are not graded. For the first few times I gave online classes, this caused so much confusion because students didn’t know where to focus their energy. So, I decided to give a mark to each activity, a number according to its value. I keep this on a Google spreadsheet linked to the Moodle so it is always up to date and visible to students. That changed everything because students can’t stand to see a low number, even if it doesn’t count towards the grade.

Also, if a discussion is designed to last 2 weeks and it is worth 6 points (marks), then I assign 3 to the first week and 3 to the second week. This gets students to participate more constantly and not just at the end of the designated period for that discussion.

I have told them that because activities are formative, they can be improved by going over my feedback and the rubrics. This of course means being flexible with due dates and very patient with problems each student has in doing assignments on time. It has motivated them to interact more with me, with classmates and with the rubrics and it has focused their attention, even if it is inadvertently, on the learning process - writing, editing, consulting, re-writing, re-editing, consulting again - and less on the grade itself.

It is kind of sad to use numbers for this purpose, but it seems to be a language symbol that communicates a message far clearer than many of my attempts to explain and motivate.


Margie Nunez said...

I always enjoy reading your blog because it contains helpful information for me. But now i have a question about my course based on what you believe in. I have never used a text book for my EAP class, (although i would have found it very helpful if i did have one. I was told to just prepare material, which is why i never had a good syllabus, so it was always changing. I would love to have a text book. If not, it means that I have to put up a lot of material on the mmodle and internet, and the students will have to be connected all the time. So what do you suggest? That i speak to Karen and get a text book? I would love that! It would make my work so much easier and more organized and consistent. And I would be happier.

Justin said...

Margie - I agree that it is always nice for students to have course readings all in one place and in physical form at the beginning of the semester. This way students don't have to read extensively on line or print out too many pages, both of which could be expensive for them. Try to get a hold of a good text book so that both your life and that of your students is more organized and filled with greater happiness. If you can't find one, then compile all of the readings before the semester begins so you can hand them out during the first day of class.

Karen said...

Hi Margie (Hmmm - I wonder if Margie will ever look for an answer on Justin's blog - wouldn't a Moodle forum be better for this interaction?),
I have a book for Academic writing at UCG that we have used in the past. We have also tried to design a syllabus using the TOEFL Preparation book. I think you need to start with essential questions before designing the syllabus and then decide on the material. What do you want the students to know, understand and be able to do by the end of the course. Also you need to talk to Susan Jensen. Don't make the book the course.

Karen said...

Hi Justin
My experience indicates that students like grades to know where they stand and I guess the marks help them know what the teacher is looking for. Have you found that marks and feedback help work improve? I have trouble getting students to use feedback, however, I have not used marks.

HugFab said...

In our culture students are used to see grades or numbers. According to my experience, there are very few students, specially in high school , that are interested in knowing what they did well and what they would need to improve.There fore , this is an issue we need to deal with, trying every day to make our students reflect that a grade doesn't mean you are the best or the worst.

Margarita "Mitoh" Ortiz said...

I also agree with Hugo and karen that students feel the need to have a grade in order to see where they stand. I like the system you came up with and with us and it's true. In our case, we have the change to edit and change according to feedback but it happens because we are teachers and we really want to learn this. In the case of your students I assume they are teachers or training to be teachers right? What I'm not sure is if this can be applied for students in general (e.g when you have an Interfacultades class). What do you think?

Irma Illonka said...

Grading has always been a challenge. I think keeping track of the "clicks" and the dates that automatically appear in all the answers, hwks our students do, will help us keep track of what they do, how they do it and when they do it. that is fantastic.

Yes, we all need grades. They are a great feedback. They are the best source to know how the teachers is getting to know the students. rubrics are important too, specially to avoid subjectivity.

No matter my grade, my learning, my individual/personal changes, my vision of my own potential for next classes online can not be graded because they are huge. Thanks Justin!

Justin said...

Margarita - Why would this strategy be any different with normal university students? The challenge is to get them to understand that you will not be grading homework or class participation anymore, but you will instead use a performance-based summative assessment to see if they have fulfilled the learning objectives or not. This will become clearer as the semester goes along and the students gain more confidence that they can do assignments several times to improve their marks and gain more practice for the final exam.

Justin said...

Karen - Yes, marks seem to get student performance to improve because they go back over the assignment to make it better and thus raise their mark. This is a win-win because they get a better mark and I get them to look over the feedback and re-do the assignment accordingly.