- Visibility 17 Views
- Downloads 2 Downloads
- DOI 10.18231/j.ijcap.2020.018
-
CrossMark
Objective structured practical examination: As an assessment tool in newly introduced competency based anatomy curriculum
Introduction
Competency-based medical education (CBME) has been implemented across India from the current academic year by Medical Council of India. The emphasis is on shift from knowledge based traditional curriculum to outcome based, student centred approach in our country. The principle of CBME paradigm is centred on the need for producing multi-dimensional, contextual and developmental competent physician.[1]
CBME necessitates a robust and multifaceted assessment system , which facilitates developmental progression of competence. New assessment tools and approaches need to be incorporated to current system to fulfil the essence of new curriculum.[2] Hence redesigning assessment tools to align with curricular objectives and instructional methods would be essential step towards successful implementation.
Assessment drives learning is universally accepted dictum. But the selection of an appropriate assessment tool mainly depends upon learning objectives. There is no golden rule that a particular assessment tool would be the best.[3] Adapting suitable combination would ensure constructive alignment between goals and learning outcomes. The objective structured clinical examination (OSCE) is one of the established valid and reliable, formative and summative tools for assessing the clinical skills.[4]
Objective Structured Practical Examination (OSPE) is a concept adapted from Objective Structured Clinical Examination (OSCE) into basic sciences. Though many have tried and used for formative evaluation, only few have incorporated into summative assessment in our country.[5] By using OSPE as an assessment tool, students can be made to learn methodically and develop skills which are crucial for successful and expert practice of medicine. The study intended to evaluate OSPE as a formative assessment tool for first year medical students in Anatomy. The perspectives would direct in judicious utilization of this tool surmounting the hiccups associated in the newly introduced CBME curriculum.
Materials and Methods
This study was conducted during formative assessment in the department of anatomy with Institutional Ethical Committee approval. The faculty (n = 16) involved were sensitized on the process of OSPE and preparation of stations through half a day workshop. The students (n=60) were randomly selected from the batch of 200 first year MBBS students. An informed consent was taken from students for participation in the study. They were briefed about the aims and process of OSPE through an audio- visual presentation. The syllabus and the schedule were announced one month prior to them. The OSPE blue print ([Table 1]) was prepared. The OSPE action plan was constructed with 6 stations testing psychomotor domain, 4 of analysis and application level of cognitive domain and 2 as rest stations. Validity was ascertained by review from senior subject experts and medical education unit members. During OSPE students were made to rotate through 10 stations. ([Figure 1]) Each station was 4 minutes carrying 4 marks. The OSPE was conducted in two parallel circuits of ten stations each. Each circuit had similar stations. The students were randomly divided in two groups of 30 students for each circuit. Further they were subdivided into 3 groups of 10 students each to be rotated through the stations. This was done to ensure homogeny of stations and timely completion of examination. Care was also taken to see that those who finished did not communicate with the rest of them. The faculty were asked to provide their opinion about OSPE in comparison with routine examination. The respondents provided their overall views on OSPE process by ticking one of the five alternatives viz. Poor, Not adequate, Satisfactory, Good, and Excellent ([Table 2]). The faculty feedback questionnaire had 22 statements and students’ questionnaire had 12. The respondents had to indicate their level of agreement/disagreement on this new tool based on a five-point Likert’s scale. The overall experience was also gathered through open ended questions. Descriptive statistics was applied and data was analysed.
| General Anatomy and General Histology | Focus and Identify the given slide | |
| General embryology | Chart Analysis and questions | |
| Topic | Upper LIMB | Lower LIMB |
| Osteology | - | Hip bone |
| Joints | Shoulder joint | - |
| Radiological anatomy | Carpal bones | - |
| Muscles | - | Popliteal fossa |
| Nerve supply | Erbs palsy | - |
| Arterial supply/ venous drainage/ lymphatic drainage | Median cubital vein | Peripheral pulse palpation |
| Surface anatomy | - | Sciatic nerve |
Results
Following observations were made
From the faculty (n=16): 97% the faculty agreed that OSPE was better tool in assessment than conventional method. 26% of the faculty rated overall OSPE programme as excellent, 57% as good, 14% satisfactory and 3% not satisfactory. The opinion was also obtained regarding quality of stations prepared: 26% felt stations were excellent, 57% as good, 16% satisfactory and 1% unsatisfactory. 60% of the faculty expressed that planning for the new assessment tool and stations were good, 22% felt satisfactory and 18% unsatisfactory. 2% thought excellent, 61% felt good, 35% felt satisfactory and 2% unsatisfactory regarding implementation of OSPE.
The perceptions of faculty and students are illustrated in [Table 3], [Table 2] respectively. The overall experience is depicted in [Table 4].
| S. No. | Your Observation | Strongly agree % | Agree % | Neither agree nor disagree % | Disagree % | Strongly disagree % |
| 1 | 41 | 59 | 0 | 0 | 0 | |
| 2 | Principles and rationale of implementing of OSPE are clear | 40 | 33 | 7 | 10 | 10 |
| 3 | OSPE was conducted systematically and there was no place for confusion | 69 | 31 | 0 | 0 | 0 |
| 4 | The instructions during the examination were clear | 98 | 2 | 0 | 0 | 0 |
| 5 | Time allotted for each station was adequate | 0 | 0 | 0 | 51 | 49 |
| 6 | OSPE is more comfortable than regular Examination | 0 | 36 | 31 | 33 | 0 |
| 7 | OSPE is more objective form of Assessment | 74 | 36 | 0 | 0 | 0 |
| 8 | OSPE was a learning experience | 60 | 31 | 9 | 0 | 0 |
| 9 | OSPE is less biased than traditional assessment | 94 | 6 | 0 | 0 | 0 |
| 10 | OSPE is less stressful than TPE | 0 | 0 | 2 | 20 | 78 |
| 11 | OSPE can concentrate more on skills with Knowledge | 93 | 6 | 1 | 0 | 0 |
| 12 | OSPE can assess the depth of understanding and concept of the topic | 79 | 21 | 0 | 0 | 0 |
| 13 | It is a comparable test for all students | 82 | 20 | 4 | 2 | 2 |
| 14 | There is no examiner bias | 99 | 1 | 0 | 0 | 0 |
| 15 | Motivation of students on OSPE was sufficient | 60 | 22 | 0 | 8 | 10 |
| 16 | Need repetitions and experience to improvise stations | 98 | 2 | 0 | 0 | 0 |
| 17 | Need to apply on students regularly to get acquainted with this new tool | 60 | 36 | 4 | 0 | 0 |
| 18 | Students need time and more exposure to get oriented to new format | 75 | 25 | 0 | 0 | 0 |
| 20 | More taxing and time consuming | 77 | 18 | 5 | 0 | 0 |
| 21 | Examiners fatigue was observed | 100 | 0 | 0 | 0 | 0 |
| 22 | OSPE can be included into regular examination in CBME curriculum | 3 | 5 | 20 | 40 | 32 |
| S. No. | Your Observation | Strongly agree % | Agree % | Neither agree nor disagree % | Disagree % | Strongly disagree % |
| 1 | Instruction given and briefing of the new method was clear | 72 | 26 | 0 | 2 | 0 |
| 2 | Time given for stations were sufficient | 3 | 18 | 10 | 43 | 26 |
| 3 | Stations were difficult | 8 | 11 | 10 | 33 | 38 |
| 4 | OSPE method is more stressful than traditional Method | 54 | 27 | 9 | 0 | 0 |
| 5 | OSPE helps learning in depth | 35 | 38 | 10 | 7 | 10 |
| 6 | Assessment of skills must be given Weight age in practical examination in first year itself | 44 | 50 | 6 | 0 | 0 |
| 7 | OSPE tests w ide range of knowledge and skills are tested | 39 | 55 | 6 | 0 | 0 |
| 8 | The observer stations were threatening and Fearful | 25 | 30 | 20 | 17 | 8 |
| 9 | Observers were friendly and non threatening | 78 | 22 | 0 | 0 | 0 |
| 10 | OSPE can be repeated again in the department | 35 | 49 | 10 | 4 | 2 |
| 11 | OSPE can be included in the university exams | 0 | 55 | 30 | 15 | 10 |
| 12 | Overall we were comfortable with the new method | 17 | 27 | 22 | 22 | 12 |
| 13 | OSPE is very much suited to CBME new curriculum | 3 | 15 | 12 | 37 | 33 |
| From faculty | From students |
| Need an expert to assist and train in preparing stations and to guide in item analysis | Were happy that there was no place for examiner bias Helps to develops critical thinking |
| Benefits of skills assessment were realised | Helps to study the topic in detail, including the skills |
| Some senior faculty were apprehensive that the delineation from glorified spotting was not much. | They felt that it will help them to develop confidence before entering clinical rotation. |
| Time constraint and examiner fatigue was a major complaint | They expressed anxiety in initial observer stations. |
| Students were not oriented to new tool and change requires time to be accepted. | Students were not oriented to new tool and change requires time to be accepted. |
Discussion
CBME curriculum has been implemented all over India. Redesigning assessment as per needs of new curriculum is required at every institution. An ideal assessment tool must be valid, reliable, feasible and acceptable by stakeholders. A single examination doesn’t fulfil all the above criteria. The conventional assessment methods utilized in Anatomy includes spotters, discussion and viva voce examination. These assessment modalities have raised concerns on examiner variability, bias and objectivity. In traditional practice, skills are not directly observed but are assessed based on questions asked at the end of the session.[6] Consequently whether students have attained the necessary skills are not tested genuinely.
OSPE process
The key factors determining successful implementation of OSPE as an assessment tool would be meticulous planning, prior sensitization and briefing to the students (before examination), preparation of procedure/response stations in an appropriate ratio (matching the number of students/groups.[7] The present study in addition also indicated that team efforts, systematic conduct with clear instructions, repetitions and experience were significant aspects for successful implementation of OSPE. Good rapport between colleagues, commitment in terms of time and personnel for elaborate process involved, proper organisation, repetitive efforts is essential to evolve OSPE as better assessment tool.[10], [9], [8][11]
Faculty perspective
Faculty expressed that it was objective, unbiased tool that could emphasize on testing wide range of skills, assess higher domains of knowledge with depth of understanding of topic. The main advantage of OSPE for any subject is that both the examination process and the examinee are evaluated by giving importance to the individual competencies. OSPE can test both the knowledge and skills, better than a conventional examination.[7] It beneficial for competency based performance discrimination and improving students’ performance quality in laboratory exercise.[12]
It was comparable test for all students. Feroz M et al in their study found that OSPE had highest discrimination index and acceptance rate among students. Also there was no room for subjectivity as expressed by many investigators by removing both experiment and examiner variability thus ensuring validity and objectivity.[9], [8] In addition, structured nature of this evaluation method offers less opportunity to factual recall and luck.
Though OSPE was a learning experience, change acceptance, stress of managing time, efforts involved in the process were expressed as tedious and taxing by faculty. Probably the current faculty student ratio, resource poor settings are hindrance towards implementation. Hence the time that the educator can spend on planning, preparing and executing an OSCE would be at stake. [13] This emerged out to be prime reason for 72% of faculty not favouring the inclusion of OSPE into CBME assessment.
Students perspective
In general, student feedback on OSPE was overwhelmingly positive. They were not only satisfied with the process of OSPE i,e number of stations, time allotted, clarity of instructions and observers but also felt that it could test wide range of knowledge and skills in greater depth. It was a comparable mode of examination than the Conventional system.[16], [15], [14]
Excellent acceptance and wide appreciation has been observed from students’ perspectives in many studies. [17], [14][11]OSPE was rated as a reliable, effective, useful, interesting and challenging examination, although considered taxing, both mentally and physically. Disapproval was only with respect to performance station as they expressed threatening sense while performing in observer station.[19], [18], [17][11][11]This is also reasonable, since it was their first encounter with new means of assessment. Gupta P et al opined that, care must be taken while introducing OSCE especially in basic sciences as students might find performing in front of observer a threatening experience. But this can be overcome by explaining the purpose and effectiveness of direct observation in providing a good feedback and making learning better.[13]
Regardless majority (84%) wanted it to be repeated regularly due to its impact on learning. Students could perceive the novelty of this method in acting as a catalyst to trigger the learning process. Some studies have reported that a combination of OSPE and CPE was preferred[22], [21], [20] and few others a complete change-over to the OSPE.[23] But only 65% were supportive towards inclusion into university examination in the study conducted by Kundu et al. [14] Our study too showed that only 55% of students agreed for inclusion of OSPE into summative assessment and 35% were neutral. Any change introduced into a system is always expected face with resistance. If proper implementation strategy is carefully administered, the adaptation to new method could be favourable.
Conclusion
OSPE is definitely a better assessment tool that is well accepted by both faculty and students. It can be incorporated into CBME curriculum to align with objectives with assessment tools. In basic sciences especially anatomy, OSPE could justify as an advantageous tool if it is beyond glorified spotters. But our experience proposes that perseverance and practice would unquestionably yield better OSPE stations that would fulfil purpose of choice. Another major practical concern is to develop strategies to overcome observer fatigue with increasing undergraduate admissions. The emphasis on skills assessment is definitely needed to prepare the students for clinical years thus contributing towards competent physicians.
Source of Funding
None
Conflict of Interest
None.
References
- Jason R. Frank, Linda S. Snell, Olle Ten Cate, Eric S. Holmboe, Carol Carraccio, Susan R. Swing. Competency-based medical education: theory to practice. Medical Teacher 2010. [Google Scholar]
- Eric S. Holmboe, Jonathan Sherbino, Donlin M. Long, Susan R. Swing, Jason R. Frank, . The role of assessment in competency-based medical education. Med Teacher 2010. [Google Scholar]
- R. M. Harden, F. A. Gleeson. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979. [Google Scholar]
- M Iqbal, B Khizar, Z Zaidi. Revising an Objective Structured Clinical Examination in a Resource-limited Pakistani Medical School. Educ Health 2009. [Google Scholar]
- N Ananthakrishnan. Objective structured clinical/practical examination (OSCE/OSPE). J Postgrad Med 1993. [Google Scholar]
- Shivani Jaswal, Jugesh Chattwal, Jasbinder Kaur, Seema Gupta, Tejinder Singh. Assessment for learning with Objectively Structured Practical Examination in Biochemistry. Int J Appl Basic Med Res 2015. [Google Scholar]
- M A Azeem. A brief overview regarding various aspects of Objective Structured Practical Examination (OSPE): Modifications as per local needs. Pak J Physiol 2007. [Google Scholar]
- M Feroz, A J Jacob. OSPE in pathology. Indian J Pathol Microbiol 2002. [Google Scholar]
- S Hasan, S Malik, A Hamad, H Khan, M Bilal. Conventional/Traditional Practical Examination (CPE/TDPE) versus Objective Structured Practical Evaluation (OSPE)/Semi Objective Structured Practical Evaluation (SOSPE). Pak J Physiol 2009. [Google Scholar]
- R M Harden. Twelve tips for organizing Objective Structured Clinical Examination. Med Teach 1990. [Google Scholar]
- N Rahman, S Ferdousi, N Hoq, R Amir, J Kabir. Evaluation of objective structured practical examination and traditional practical examination. Mymensingh Med J 2007. [Google Scholar]
- Piyush Gupta, Pooja Dewan, Tejinder Singh. Objective structured clinical examination (OSCE) revisited. Indian Paed 2010. [Google Scholar]
- Dipankar Kundu, T Mandal, Manish Osta, Gargi Sen, HN Das, Divyendu Gautam. Objective structured practical examination in biochemistry: An experience in Medical College, Kolkata. J Nat Sci Biol Med 2013. [Google Scholar]
- V Roy, U Tekur, S Prabhu. A comparative study of two evaluation techniques in pharmacology practicals: Conventional practical examination versus objective structured practical examination. Indian J Pharmacol 2004. [Google Scholar]
- A E Nemer, N Kandeel. Using OSCE as an Assessment Tool for Clinical Skills: Nursing Students feedback. Aust J Basic Appl Sci 2009. [Google Scholar]
- Russell B Pierre, Andrea Wierenga, Michelle Barton, J Michael Branday, Celia DC Christie. Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BMC Med Educ 2004. [Google Scholar] [Crossref]
- M Verma, T Singh. Attitude of Medical students towards objective structured clinical examination (OSCE) in pediatrics. Indian Pediatr 1993. [Google Scholar]
- M V Natu, T Singh. Student’s opinion about OSPE in pharmacology. Indian J Pharmacol 1994. [Google Scholar]
- S. L. Malik, S. K. Manchanda, K. K. Deepak, K. R. SUNDERAM. The attitudes of medical students to the objective structured practical examination. Med Educ 1988. [Google Scholar]
- U. NAYAR, S. L. MALIK, R. L. BIJLANI. Objective structured practical examination: a new concept in assessment of laboratory exercises in preclinical sciences. Medical Education 1986. [Google Scholar] [Crossref]
- A S Mahajan, N Shankar, O P Tandon. The Comparison of OSPE with Conventional Physiology Practical Assessment. Journal of the International Association of Medical Science Educators 2004. [Google Scholar]
- Ahmed Yaqinuddin, Muhammad Zafar, Muhammad Faisal Ikram, Paul Ganguly. What is an objective structured practical examination in anatomy?. Anat Sci Educ 2013. [Google Scholar]