Assessing Creativity through Common Consensus Discussion
The assessment result
The study obtained high coefficients in the statistical tests for reliability and inter-reliability. One possible reason of course is the success of the research method in generating a rubric that is agreed on by teachers in the context. Instead of adopting a standard instrument, the study applied the Delphi method to generate a set of criteria that was agreed on by the majority of teachers as suitable for assessing students' creativity in filmmaking. Hence, when they applied these criteria in the actual assessment activity, high agreement among them was achieved. As Amabile put it, "...experts in a domain do share creativity criteria to a reasonable degree." (1983, p.38).
However, one has to be aware that even with the rubric created, if assessors assess a creative work with an overall impression instead of examining it item by item, it may also result in high inter-reliability among assessment items. This was actually reflected by one of the assessors in the discussion after the assessment activity. She found the session was too long and felt fatigued by the end and tended to grade students' films according to overall impression. This shows that even with a good assessment tool, if the way of applying it is inappropriate, negative effects may arise.
However, at the minimum, there is no indication that the process or tool has a negative impact on the ability to assess for creativity. It appears that participating in a long discussion process and using a rubric that breaks down creativity into an array of items in several categories does not confuse or prevent teachers from identifying and assessing the creativity of a given work. A further study to support or refute these possibilities is in the planning stage.
