Formative Assessment for Progress Tests of Applied Medical Knowledge Conclusions
The potential for the Learning Oriented Assessment of Applied Medical Knowledge project has been identified here and a method has been detailed that can be used to improve the implementation of the project locally and also contribute to the wider understanding of learning and assessment. Future developments will track the progress of the students involved in item writing to formally evaluate the impact on their learning, and develop the management of the test to provide more information on knowledge acquisition.
We also intend to systematically adjust aspects of the Learning Oriented Assessment project in successive iterations and observe the influence of the adjustment.
In terms of the item writing environment we are in a position to change the organizational structure of the item writing process to tease out those factors that appear to be most effective. We are able to adjust the detail of the review process to explore the effect on the quality of the items. We can vary the location of the source of items (PBL class, community placements, plenaries, Life Science Resource Centre) and explore how that influences the items created and the process of item writing. We are able to adjust the motivations of the item writers; so far we have employed a small group of students to write items and rewarded them financially, but we could embed item writing into the course, remove and or adjust characteristics of the financial incentive and explore the impact on how the task is perceived.
In terms of the test we are in a position to vary the conditions of the test such as the number of items in each test, whether each test contains fixed items or a random sample from the database, and the nature of the feedback on each item. We can then explore the impact this has on the use of the test, and on learning.
A key priority in this project is the quality of the items written. The training and item writing process has to be robust enough to develop the student's cognitive skills to that which is typical of the 'expert'. This is no easy task and the question of how to do this is the central focus of this project.
