Introduction
Competency-based medical education (CBME) has been implemented across India from the current academic year by Medical Council of India. The emphasis is on shift from knowledge based traditional curriculum to outcome based, student centred approach in our country. The principle of CBME paradigm is centred on the need for producing multi-dimensional, contextual and developmental competent physician.1
CBME necessitates a robust and multifaceted assessment system , which facilitates developmental progression of competence. New assessment tools and approaches need to be incorporated to current system to fulfil the essence of new curriculum.2 Hence redesigning assessment tools to align with curricular objectives and instructional methods would be essential step towards successful implementation.
Assessment drives learning is universally accepted dictum. But the selection of an appropriate assessment tool mainly depends upon learning objectives. There is no golden rule that a particular assessment tool would be the best.3 Adapting suitable combination would ensure constructive alignment between goals and learning outcomes. The objective structured clinical examination (OSCE) is one of the established valid and reliable, formative and summative tools for assessing the clinical skills.4
Objective Structured Practical Examination (OSPE) is a concept adapted from Objective Structured Clinical Examination (OSCE) into basic sciences. Though many have tried and used for formative evaluation, only few have incorporated into summative assessment in our country.5 By using OSPE as an assessment tool, students can be made to learn methodically and develop skills which are crucial for successful and expert practice of medicine. The study intended to evaluate OSPE as a formative assessment tool for first year medical students in Anatomy. The perspectives would direct in judicious utilization of this tool surmounting the hiccups associated in the newly introduced CBME curriculum.
Materials and Methods
This study was conducted during formative assessment in the department of anatomy with Institutional Ethical Committee approval. The faculty (n = 16) involved were sensitized on the process of OSPE and preparation of stations through half a day workshop. The students (n=60) were randomly selected from the batch of 200 first year MBBS students. An informed consent was taken from students for participation in the study. They were briefed about the aims and process of OSPE through an audio- visual presentation. The syllabus and the schedule were announced one month prior to them. The OSPE blue print (Table 1) was prepared. The OSPE action plan was constructed with 6 stations testing psychomotor domain, 4 of analysis and application level of cognitive domain and 2 as rest stations. Validity was ascertained by review from senior subject experts and medical education unit members. During OSPE students were made to rotate through 10 stations. (Figure 1) Each station was 4 minutes carrying 4 marks. The OSPE was conducted in two parallel circuits of ten stations each. Each circuit had similar stations. The students were randomly divided in two groups of 30 students for each circuit. Further they were subdivided into 3 groups of 10 students each to be rotated through the stations. This was done to ensure homogeny of stations and timely completion of examination. Care was also taken to see that those who finished did not communicate with the rest of them. The faculty were asked to provide their opinion about OSPE in comparison with routine examination. The respondents provided their overall views on OSPE process by ticking one of the five alternatives viz. Poor, Not adequate, Satisfactory, Good, and Excellent (Table 2). The faculty feedback questionnaire had 22 statements and students’ questionnaire had 12. The respondents had to indicate their level of agreement/disagreement on this new tool based on a five-point Likert’s scale. The overall experience was also gathered through open ended questions. Descriptive statistics was applied and data was analysed.
Table 1
OSPE blueprint
Results
Following observations were made
From the faculty (n=16): 97% the faculty agreed that OSPE was better tool in assessment than conventional method. 26% of the faculty rated overall OSPE programme as excellent, 57% as good, 14% satisfactory and 3% not satisfactory. The opinion was also obtained regarding quality of stations prepared: 26% felt stations were excellent, 57% as good, 16% satisfactory and 1% unsatisfactory. 60% of the faculty expressed that planning for the new assessment tool and stations were good, 22% felt satisfactory and 18% unsatisfactory. 2% thought excellent, 61% felt good, 35% felt satisfactory and 2% unsatisfactory regarding implementation of OSPE.
The perceptions of faculty and students are illustrated in Table 3, Table 2 respectively. The overall experience is depicted in Table 4.
Table 2
Faculty feedback on OSPE (n=16)
Table 3
The opinion of the students about the OSPE method of assessment (n=60)
Table 4
Overall experience of faculty and students
Discussion
CBME curriculum has been implemented all over India. Redesigning assessment as per needs of new curriculum is required at every institution. An ideal assessment tool must be valid, reliable, feasible and acceptable by stakeholders. A single examination doesn’t fulfil all the above criteria. The conventional assessment methods utilized in Anatomy includes spotters, discussion and viva voce examination. These assessment modalities have raised concerns on examiner variability, bias and objectivity. In traditional practice, skills are not directly observed but are assessed based on questions asked at the end of the session.6 Consequently whether students have attained the necessary skills are not tested genuinely.
OSPE process
The key factors determining successful implementation of OSPE as an assessment tool would be meticulous planning, prior sensitization and briefing to the students (before examination), preparation of procedure/response stations in an appropriate ratio (matching the number of students/groups.7 The present study in addition also indicated that team efforts, systematic conduct with clear instructions, repetitions and experience were significant aspects for successful implementation of OSPE. Good rapport between colleagues, commitment in terms of time and personnel for elaborate process involved, proper organisation, repetitive efforts is essential to evolve OSPE as better assessment tool.10, 9, 811
Faculty perspective
Faculty expressed that it was objective, unbiased tool that could emphasize on testing wide range of skills, assess higher domains of knowledge with depth of understanding of topic. The main advantage of OSPE for any subject is that both the examination process and the examinee are evaluated by giving importance to the individual competencies. OSPE can test both the knowledge and skills, better than a conventional examination.7 It beneficial for competency based performance discrimination and improving students’ performance quality in laboratory exercise.12
It was comparable test for all students. Feroz M et al in their study found that OSPE had highest discrimination index and acceptance rate among students. Also there was no room for subjectivity as expressed by many investigators by removing both experiment and examiner variability thus ensuring validity and objectivity.9, 8 In addition, structured nature of this evaluation method offers less opportunity to factual recall and luck.
Though OSPE was a learning experience, change acceptance, stress of managing time, efforts involved in the process were expressed as tedious and taxing by faculty. Probably the current faculty student ratio, resource poor settings are hindrance towards implementation. Hence the time that the educator can spend on planning, preparing and executing an OSCE would be at stake. 13 This emerged out to be prime reason for 72% of faculty not favouring the inclusion of OSPE into CBME assessment.
Students perspective
In general, student feedback on OSPE was overwhelmingly positive. They were not only satisfied with the process of OSPE i,e number of stations, time allotted, clarity of instructions and observers but also felt that it could test wide range of knowledge and skills in greater depth. It was a comparable mode of examination than the Conventional system.16, 15, 14
Excellent acceptance and wide appreciation has been observed from students’ perspectives in many studies. 17, 1411OSPE was rated as a reliable, effective, useful, interesting and challenging examination, although considered taxing, both mentally and physically. Disapproval was only with respect to performance station as they expressed threatening sense while performing in observer station.19, 18, 171111This is also reasonable, since it was their first encounter with new means of assessment. Gupta P et al opined that, care must be taken while introducing OSCE especially in basic sciences as students might find performing in front of observer a threatening experience. But this can be overcome by explaining the purpose and effectiveness of direct observation in providing a good feedback and making learning better.13
Regardless majority (84%) wanted it to be repeated regularly due to its impact on learning. Students could perceive the novelty of this method in acting as a catalyst to trigger the learning process. Some studies have reported that a combination of OSPE and CPE was preferred22, 21, 20 and few others a complete change-over to the OSPE.23 But only 65% were supportive towards inclusion into university examination in the study conducted by Kundu et al. 14 Our study too showed that only 55% of students agreed for inclusion of OSPE into summative assessment and 35% were neutral. Any change introduced into a system is always expected face with resistance. If proper implementation strategy is carefully administered, the adaptation to new method could be favourable.
Conclusion
OSPE is definitely a better assessment tool that is well accepted by both faculty and students. It can be incorporated into CBME curriculum to align with objectives with assessment tools. In basic sciences especially anatomy, OSPE could justify as an advantageous tool if it is beyond glorified spotters. But our experience proposes that perseverance and practice would unquestionably yield better OSPE stations that would fulfil purpose of choice. Another major practical concern is to develop strategies to overcome observer fatigue with increasing undergraduate admissions. The emphasis on skills assessment is definitely needed to prepare the students for clinical years thus contributing towards competent physicians.
Article
DOI
