Formative Assessment for Progress Tests of Applied Medical Knowledge
This article outlines features of the progress test at the Peninsula Medical School. It then presents the rationale for the development of Formative Assessment of Applied Medical Knowledge. The employment of students as item writers is described and data on the use of the assessments are reported. The advantages and disadvantages of the features of the item writing learning environment are discussed.
Progress testing of medical knowledge is a method of assessment in which all cohorts of students sit the same test, set at the standard expected of a newly qualified doctor. Progress tests are typically composed of many items which may be multiple true/false, extended matching or multiple choice. At Peninsula Medical School each test consists of 125 best-of-5 multiple choice items. Each item consists of a clinical vignette, a question, 5 options and a don't know option. Tests are sat 4 times per year with each test being a structured sample from a large item bank.
Students are able to see their knowledge grow over the course of their studies, and patterns of student scores over time can be used to identify variables of interest to the student and to the faculty. Progress tests are intended to assess deep learning because each item is designed to engage the student in clinical reasoning and functional knowledge rather than recall of isolated facts, and because each test exposes the student to a structured sample of the whole domain of applied medical knowledge appropriate to that of a newly qualified doctor.
It is particularly difficult to provide useful feedback to students during the early years of their study, as they will have only minimal knowledge at the assessed level, and do not attempt many of the questions. Scores in the first 2 years are around 10-20% with most questions eliciting a 'Don't know' response. In order to generate more detailed knowledge of what our students have learned in the first 2 years, we decided to develop a formative question bank focussed on knowledge specific to these early years.
Inevitably for a new medical school, the staff focus has been on developing summative assessments. A more innovative approach was needed to develop the formative question bank. We recognized that students have a close involvement with the curriculum and under appropriate circumstances form a valuable educational resource. We also recognized the potential of the item writing environment for learning and noticed that opportunities for learning were being missed by excluding students from important parts of the assessment process.
The purpose here is to outline the rationale for the use of students to write assessment items and report on the use and performance of the test. It proposes a design-based approach to enhancing teaching and learning through Learning Oriented Assessment.
