Assessing Creativity through Common Consensus Discussion
The Delphi survey
In the stage 1 Delphi survey, participating teaching faculty held different opinions towards the study in the beginning. "This is the first time I look into the creativity of students' films in such a structural way. Interesting!", said one of the teachers who welcomed the idea. However, some were skeptical, "We are professionals. We should be able to tell if a film is creative or not, no need to discuss" commented another colleague.
After the preliminary round, in which panelists were encouraged to spell out their opinions, it was discovered that there were many differences among them in what to assess in student's work. For example, one teacher thought that one essential assessment criterion is communication,
"Film is a form of communication. Therefore, a film should be judged by how well it communicates its message to the audience. If creativity hampers communication, it should be taken out."
He weighted communication over creativity. However, others did not share his opinion and discussion through emails was opened in an anonymous way with me acting as the moderator. The results of the two surveys revealed that some colleagues did change their mind and support this argument later while many of them continued to hold their original opinion (increase from 12.5% in the 1st round to 37.5% in the 2nd round).
As shown from Figure 2, higher levels of consensus were reached in the 2nd round survey. For example, only one item reached 100% consensus in the 1st round while in the 2nd round 5 items did. As for items that reached 75% or more, there were 7 in the 1st round but increased to 11 in the subsequent survey.
One significant difference and interesting change over the two surveys is the decrease in the choice of 'no opinion' for items. In the 1st round 4 items received zero responses of neutral as indicated by a "3".
In the second round the number of items receiving no neutral responses moved up to 13 of the possible 25. This indicates that after repeated surveys and sharing of others' ideas in discussion, most panelists made up their mind whether or not they considered certain items important in assessing student's creativity. The Delphi method may be considered for other complicated issues, not just to establish consensus but also to provide a mechanism for people concerned to clarify their thoughts.
