تسجيل الدخول

تسجيل

Delphi method Primary round one

المؤلف:  Gipsy Chang & Josephine Csete

المصدر:  Enhancing Teaching and Learning through Assessment

الجزء والصفحة:  P459-C38

2025-08-23

616

Delphi method

Primary round one

Designing the instrument

A total of 26 assessment items were proposed in the questionnaire of the first survey. These items were derived from two sources. One source was the aggregated and synthesized responses of the 9 panelists in the preliminary round. The other source was an established set of criteria, Creative Product Analysis Matrix (CPAM) (Besemer & Treffinger 1981). The CPAM comprises of 14 items divided into 3 sections: novelty, resolution, elaboration and synthesis. It is used frequently in assessing design and visual art products. The criteria of CPAM were adapted for the context of filmmaking and formed part of the assessment items in the questionnaire.

Table 1 summarizes the 26 criteria in 4 groups for assessing the creative outcomes of students' films. These criteria were used to develop the instrument for the subsequent rounds of the Delphi survey.

The design of the Delphi questionnaire for round one followed that of other Delphi surveys reported in the literature. It was designed as a closed questionnaire. A 5-point Likert rating scale was used to indicate the importance of each assessment item. Panelists were required to respond to all items by circling the corresponding score. A rating of 4 or 5 was considered to be important, 3 was seen as a 50/50 choice, and 2 or 1 indicated the item was considered unimportant in assessing the creativity of a student's film. Panelists were also asked to make comments at the end of the questionnaire on either the items or the instrument. These comments allowed them to show to what degree they understood the thoughts of others They also served the purpose of modifying the questionnaire as necessary.

Analyzing data

Data collected from the first round were analyzed both qualitatively and quantitatively. The importance of each assessment item was looked at. The responses were divided into three categories, items receiving scores of '4 - 5' were considered important, items scoring 3 were seen as representing no opinion from the panelists, and items scoring '2 - 1' were considered unimportant. The percentage of panelists’ responses on each item was determined. The result was reported back to the panel together with the second-round questionnaire. Any item receiving 100% consensus from the panel did not need to be considered again in the second survey. Written comments were also compiled and delivered in the subsequent round.