Can Intrinsic Graduate Qualities be Developed Through Assessment? Mapping Assessment Practices in IT Degree Programs
Our findings
The database generated, and the accompanying graphs (see below) indicate a number of surprising, and perhaps uncomfortable, factors about the current assessment being applied in the three linked programs.
Within the 21 courses mapped (see Figures 1 and 2) for graduate qualities, there are many apparent discrepancies between what the course objectives state and what the assessment tasks ask the students to do.
Figure 1 indicates that in most of the courses, course objectives focus almost exclusively on Graduate Quality 1 'Body of Knowledge' which would indicate that course developers write course objectives to reflect the course content only. In fact, only four of the 21 courses explicitly address more than one graduate quality. In the case of assessment tasks, nine of the 21 courses appear to explicitly address only Graduate Quality 1 (see Figure 2), although a number of courses address 5 out of 7 graduate qualities.
An examination of the allocation of graduate qualities using indicative unit weightings (see Figure 3) made by course developers in their course statements indicates a higher correlation with the course assessment tasks rather than the course objectives.
A comparison of Figure 2 and Figure 3 shows a high correlation between graduate qualities evident in assessment tasks and graduate qualities weightings allocated by course developers in their course statements. University policy requires all course descriptors to include a chart indicating which graduate qualities are expected to be gained by a student undertaking the course. Nevertheless, this result was surprising given that course developers generally allocate graduate quality weightings to a course during course development whereas assessment tasks are designed during delivery and often without reference to graduate quality weightings. It may be the case that course developers allocate graduate qualities in the expectation that these graduate qualities will develop as a result of doing typical assessment tasks.
Given such results we then undertook a further, and more critical, analysis of assessment tasks in four courses. In all cases we ascertained that the assessment tasks seemed based on an assumption that students already possessed a number of skills that could be viewed as graduate qualities. Some examples of these assumptions are that to adequately address a task, students:
• Possess adequate presentation skills
• Have the capacity to engage in student-centred learning and have the ability to find information
• Are able to work effectively in groups and manage conflict resolution
• Have a sound command of academic level written and spoken English
• Can critically analyze text
• Understand relatively complex commercial activities
More profoundly, in several other cases we determined that the assessment tasks seemed to actually test whether students already possessed a number of graduate qualities rather than developing them.
