How not to use peer assessment in a MOOC

Earlier this week, after much thought, I unenrolled from what until that point had been an excellent course on Global Public Health.

I'd viewed all the lectures, taken all the quizzes, done all the readings, and all that was left was the final assignment.

Students were given a reasonable amount of time in which to write a ~1500 word paper on one of three topics related to the course. One topic was somewhat rigidly defined, one was wide open, and one was in between.

I'd settled on the second option - write an essay on an infectious disease, focusing on the sorts of issues brought up by the course: international public goods, global health initiatives and so on. Fair enough. I was already familiar with leprosy from research I'd done years ago, and was ready to write a report on that illness.

It was on deciding on my topic that I realized I'd be drawing mostly on my existing knowledge and Google searches, not what I'd learned in the course, for my essay. That prompted me to take a closer look at the requirements.

Here are the ones that stood out as troubling or inappropriate:

a) The essay was worth 80% of the course mark. Placing such a heavy weight on the essay cheapened the rest of the course material. It is entirely possible to ignore the course, pick an infectious disease one is familiar with (or which has been extensively written about), pander to the specific points on the posted rubric and walk away with a statement of accomplishment. Conversely, it is also possible to pay careful attention to the lectures and readings, submit all the quizzes on time, and fail the course due to insufficient attention to the rubric, troubles with the English language, and so on.

b) The essay would be peer-assessed. This in itself is not a problem. I have been privileged to take other Coursera courses where peer-assessed essays are an invaluable and worthwhile component of the course. This was not one of them. (I'll post my thoughts on what I believe is needed for peer assessment to work in a later blog entry.)

c) This was the only essay required in the course. Students have no chance to learn how to grade, and there is no guarantee that a random subscriber to a free online course will have the knowledge, experience or language skills necessary to gauge the worth of a university-level academic essay on a nuanced, complex scientific and political subject.

d) The assessment rubric required students to establish that the essay writer had encyclopedic knowledge of a subject (failing this, a minimum mark should be given for the relevant rubric entry). This is a fundamental, fatal flaw in the grading rubric, especially since the course was billed as an introduction to the subject. This asks non-experts to know what it is that they do not know. How can a beginner in a subject, acquainted with only a modicum of information, judge whether a peer has not only more knowledge, but all-encompassing encyclopedic (the word used in the rubric) knowledge?

For all of the above reasons, a statement of accomplishment for this course provides at best a very mixed signal of whether a student was engaged with or understood the material. Though confident I could in short order write an essay that would earn acceptably high marks, I did not find the task intellectually honest. There was too much of a disconnect between learning from the course and scoring well on the final assignment, and so I chose, with some regret, to un-enroll at the eleventh hour.

Peer assessment CAN work, and work brilliantly - but this is not the way to do it.

The other side of the coin: a beautiful video and thoughtful design post about Bitcoin

Some weeks ago I wrote a long post detailing some of my current thoughts about Bitcoin.

This morning, the excellent design blog Fast Company posted a fascinating article on a beautiful video about Bitcoin.

I'm still uncomfortable about equating scarce tradable goods with currency or even currency candidates, but the post provides a lot of food for thought.

If nothing else, the video is worth a view or ten simply as a work of art. It's a breath-takingly gorgeous animation.