Introduction
Assessment is a vital component in the medical education curriculum. It has a powerful influence over the learning process. Anatomy is one of the most essential basic subjects in medical education regardless of nation or speciality.1 New medical terminologies are being introduced to the undergraduate students for the first time. It is a vast subject and assessment of anatomy is complicated as it has many subdivisions like general anatomy, gross anatomy, histology, embryology, surface marking, osteology and genetics.2 For practical sessions in anatomy, more time is being invested by students than theoretical sessions and the conventional method of assessment does not assess all the skills, therefore there arises a need for an exam which is more reliable, able to assess higher cognitive domain, psychomotor and affective skills. This article is derived from curriculum innovation project which was a part of Advance course medical education at Medical council of India nodal center, Gujarat.
Objective structured practical examinations (OSPE) are a well accepted tool in the practical examination for both Pre and Para-clinical subjects.3 Therefore we introduced OSPE in the anatomy department of our institute. OSPE is a practical exam where there is a set chain of stations at which students work through a task designed to test their skills. In this method of assessment, competencies are evaluated in a comprehensive, consistent and structured manner.4 It is more objective, can be used with a large number of students, valid, reproducible, reliable method and it eliminates examiner bias along with variability.
The primary significance of OSPE is that, it promotes an integrated approach of learning which includes knowledge, skills and communication which in turn increases the depth of understanding the subject to the students and to become a more competent Indian medical graduate along being a successful clinician.5 OSPE also helps students to obtain feedback at the end of each session on their strength and weaknesses.6 This encourages and directs them for self-evaluation and self-assessment which finally leads to self-directed learning. Similarly, a feedback can be taken from students which help the faculty to plan more systemically their integrated curriculum, integrated teaching schedule and assessment methods.7
It is a useful tool in discriminating good and not so good performers. Students feel more comfortable and score more in this exam. It is motivating, inspiring, interesting and demands higher cognitive skills.8 The limiting factors for OSPE are that a lot of effort is required in careful planning which is time-consuming and trained dedicated staff is required. Most of the existing literature supports OSPE to be superior to conventional methods of examination; however, each institute must evaluate the acceptability and feasibility criteria before adopting it.9,10,11,12 Also, substantial literature regarding the use of OSPE as an assessment tool in medical education in India is lacking especially in Anatomy subject.13 Thus, present study aimed to assess student and faculty perception regarding the feasibility of OSPE implementation for the first time in the practical examination of first professional undergraduate students.
Materials and Methods
This is a prospective educational interventional study which was conducted in the Department of Anatomy at LN Medical College & JK Hospital, Bhopal (M.P). The study was conducted only after obtaining approval from the Institutional Ethics Committee and administrative authorities. A batch of 150 students is there in the first year of MBBS (Bachelor of Medicine and Bachelor of Surgery) out of which 135 students (75 girls and 60 boys) appeared for the study. Duration of study was from 15th Oct 2017 to 15th Feb 2018.
Since OSPE was introduced for the first time in the Dept of anatomy, sensitization of all students and faculty was done in the orientation program for OSPE. After sensitization, two practice sessions were conducted for students before the main exam. Fifteen days prior, students were informed verbally about the main exam and the topics on which OSPE had to be conducted.
The present study comprised of total twelve OSPE stations which included six observer stations, four non observer stations and two rest stations. The OSPE stations included charts, bones, histology slides, embryology models, Digital X-rays skill performing exercises and communication skills. Students were allotted a time of three minutes for each OSPE station. Each OSPE station carried five marks. Students were instructed to rotate in the clockwise direction and attend all the stations. Structured questions were asked in OSPE stations and all the details of each station are described in Table 1 and observation was done by faculty on every observation station. A cognitive domain was checked on all ten stations, psychomotor skills were observed on six stations and communication skills were assessed on two stations.
During the present study, the method of assessment was standardized as follows:-
Establishment of a blueprint of assessment – Mapping of test items for specific learning outcomes and domains to be checked. Candidate instructions must be concise and clear.
Formation of checklist – All individual stations was reviewed in details through focus group discussions of faculty and a checklist for scoring individual stations was maintained. A checklist was complete and main components were included in it.
Review of OSPE stations - All the stations were assessed for content clarity and avoidance of duplicity across various stations.
The content validity was established by focus group discussions involving faculty of the department and face validity was established by involving faculty of other departments too.
The scores obtained by the students were informed to them but it was made clear that these scores would not have any weight age in academic assessment.
Feedback from students and faculty - The perceptions of 135 students and 5 faculties were assessed using ten closed-ended questionnaires. At the last of the feedback form, there was one open- ended question in the form of “ comments’’ so that everyone can freely express positive and negative points and feelings about the study.
Results
Response of the students and faculty are tabulated in Table 2 and Table 3.
Table 1
Details of OSPE stations
Table 2
Student’s perspective regarding OSPE
Table 3
Faculty perspective regarding OSPE
Table 4
Comparison of perception of students regarding OSPE on anatomy
Feedback from students
97% of the students felt that questions asked were relevant, 96%said it is fair when compared with old method, 91% found it is can be passed easily, 97% favored OSPE should be followed as a method of assessment in anatomy, 98% stated it helps to improve because it is a better stimulus for study, 94% supported OSPE because it helps score better, 91% said it helps to eliminate bias and 88% supported that it makes the student think in more than one way. Nevertheless, 27% felt that it was more stressful since it was adopted for the first time and 20% students felt that the time given (3 minutes) was not sufficient especially where skills have to be performed.
Feedback from faculty
Majority of the faculty gave a positive feedback associated with OSPE. 100% of the faculty felt that OSPE is more objective, more uniform when compared with the traditional method, it can measure better practical skills, helps to eliminate examiner bias and can be followed as a method of assessment of practical skills in anatomy. But 60% of the faculty said it is not easy to pass OSPE since it assess es the higher level of cognitive skills. 80 % faculty said it limits the assessment of knowledge.
Feedback given by faculty motivates us to implement OSPE in routine exams. Since we have a limited number of faculty and to prepare OSPE stations, its checklist, its implementation is a tedious and time-consuming process. But with due course of time department can have its own OSPE bank for future. Therefore, in the beginning, OSPE can be used for formative assessment.
Discussion
In the present era, it is a well known fact that the medical education is undergoing extensive re-evaluation to achieve educational objectives which are being defined to make medical graduates more competent. As we all know that assessment drives learning, so developing a better assessment tool will enhance learning and further, it will help to fulfill the objectives of medical education.8
Recent trends show a shift in the paradigm of basic science, particularly of anatomy curriculum. Now it has been designed in such a way where there is an increa sed integration of anatomy with clinical subjects and promoting the use of technology in the form of E-learning for the better understanding of a subject. Anatomical knowledge can be assessed by both written and practical examination. Written examinations assess core knowledge by case based study, multiple choices, short answer and long answer questions. But for being an efficient practitioner, practical skills are essentials and must be mastered thoroughly.14 When compared with the conventional methods which test only a few learning outcomes, it is essential to implement new assessment methods to make examination more effective.15
If the assessment strategies are well planned, designed and implemented, they help to achieve all the competencies along with improvisation in the learning and feedback process.16
Harden and his colleagues, in 1975, introduced OSCE as an ideal method of assessment of clinical competencies. This was later modified to assess practical knowledge and skills in the basic medical sciences and was termed OSPE. OSPE is an objective and structured method of skill-based assessment by direct observation of the student’s performance in an adaptable examination set-up.17 It was first introduced as a tool to assess the practical skills of students in 1986 by Nayar and colleagues.18
It is a well- accepted tool for practical examination in most of the advanced countries and also in few Indian medical institutes for both pre and Para clinical branches.19 In the present study, an attempt has been made to introduce OSPE and test its feasibility in the Anatomy department of our institute. OSPE is not merely an evaluation tool but it helps to improve teaching-learning methods too.
In the present study, total 135 students (75 girls and 60 boys) participated and were assessed by OSPE. Five faculties helped in arranging OSPE since it requires a lots of pre-planning with collaborative teamwork. It contains a total chain of twelve stations out of which two we re rest stations. After successful conduction of OSPE, students and faculties were supposed to fill the feedback form.
Majority of the students gave the positive feedback, 97% of students agreed that relevant and well-constructed questions were asked. When compared with a traditional method, 96% appreciated OSPE to be fairer since all the student s were exposed to the similar questions having a same difficulty level. 98% of students stated that it helps to improve because it leads to self-directed learning. 94% confirmed that it helps to score better, 91% suggest that it eliminated subjectivity and examiner bias and 88% said that it makes student to think in more than one way which are in accordance with the study of Rajkumar et al. 27% stated that it is more stressful since it w as introduced and 20% complained that time w as not sufficient to attempt all questions at each station. Favorable and satisfied feedback was given from the faculty side too.
Previous various studies proved that OSPE is a reliable assessment tool but per se a very few studies are done independently on Anatomy subjects.20,21 As shown in Table 4 one of the studies forwarded by Rajkumar KR et al emphasizes that OSPE is a better assessment tool to assess students for anatomy, physiology and biochemistry.2 The study conducted by Yaqubuddin et al also found OSPE to be an efficient tool to assess the practical aspect of anatomical knowledge.3 Cherian SB proved that Computer-a ssisted OSPE can be enabled for a uniform assessment of students for assessing practical skills of a natomy among undergraduates.22 Feroze and Jacob conducted OSPE for practical assessment in pathology and found that OSPE was more objective and measured practical skills better as compared with traditional methods.20
Therefore from the present study, we concluded that both students and faculty favored OSPE because it enables evaluation of all the domains with a higher level of students’ satisfaction. It not only improves assessment but also provides a forum for the improvement of both teaching and learning through feedback. The limitation of the study was in regarding of a limited number of trained faculties, a formation of high standard question banks and dedicated time required for planning till implementation of OSPE. During the conduct of OSPE, it was found that there is a risk of observer fatigue if the observer has to record the performance of several students on the lengthy checklist. Thus the participating faculties suggested and emphasized the need for continuous faculty development in the field of medical education for better and efficient conduct of newer methods of assessment. Another limitation was the lack of gender analysis of the data collected from the students in the form of feedback. Only the perception of students and faculties were analyzed irrespective of their gender, caste and creed.
Conclusion
A majority of the participants in the study were in favour of using this method in the future both initially as formative and then as summative assessment. Thus, this study demonstrates that OSPE is a relevant, meaningful and feasible tool for the assessment of practical skill in undergraduate students in anatomy at our institute.
Article DOI
