Research Article

The Variation in Pharmacology Subject Achievement Levels Between two Assessment Methods in Undergraduate Medicine

by Shahid Akhund1*, Joanne Ehemann2,3, Shahmina Naz4

1Department of Medical Education and Department of Anatomy College of Medicine, Alfaisal University, Riyadh, KSA

2Griffith University School of medicine and dentistry, Gold coast, Australia

3Gold Coast University Hospital, Queensland Health, Australia

4Department of Anatomy College of Medicine, Alfaisal University, Riyadh, KSA

*Corresponding author: Shahid Akhund, Department of Medical Education and Department of Anatomy College of Medicine, Alfaisal University, Riyadh, KSA. Email: sakhund@alfaisal.edu

Received Date: 20 August, 2023

Accepted Date: 24 December, 2023

Published Date: 26 January, 2024

Citation: Akhund S, Ehemann J, Naz S (2024) The Variation in Pharmacology Subject Achievement Levels Between two Assessment Methods in Undergraduate Medicine. J Surg 9: 1987 https://doi.org/10.29011/2575-9760.001987

Abstract

Introduction: The examination system is one of several variables that affects students’ academic performance. Examinations taken once a year or every two semesters are typical practices. Students prefer the semester system because they do academically much better under it. The British method of preclinical and clinical years of instruction followed by end-of-year examinations was mainly used in Pakistani medical education. The semester system of examination with objective assessment was recently adopted in medical institutes in Pakistan as a result of the reform of medical education. Regarding how this move will affect the academic performance of medical students, there is no empirical data. The purpose of this study was to evaluate if the semester examination system has affected medical students’ academic performance in any way compared to the annual examination system.

Method: Pharmacology percentage scores of four two batches of students who sequentially took annual and semester examinations were collected from two medical universities of Pakistan. Following a normality check on the data, descriptive and inferential statistical tests were run.

Results: The data of 1943 students (semester = 463 and annual = 1480) were entered for analysis. The students who took semesterbased examination performed (M= 71.2, SD= 8.7) higher than the students who took annual examination (M=58.63, SD=9.50). The difference in mean percentage scores was statistically significant (t (1941) = 25.32, P <.01, CI 95% (11.60-13.55).

Discussion: The outcomes showed that students who took the semester-based examination had higher scores. The research’s conclusions corroborated those of other studies that claimed that shortening course duration and overhauling the assessment process had improved students’ performance. Additionally, the objective assessment approach reveals enhanced academic performance.

Conclusion: According to the study, students who took semester examinations scored much higher on the pharmacology knowledge evaluation than those who took end-of-year examinations. To see whether this difference is seen in other basic sciences courses and persists into the performance clinical years, more research is required.

Keywords: Annual; Examination; Medical education reform; Pharmacology; Semester 

Introduction

In every educational institution, its assessment policy, course length and content all have an impact on how well its students perform [1,2]. There are two examination-related methods that are frequently used: semester-based examinations and annual examinations. Students take an end-of-year examination in an annual system and a six-monthly examination in a semester system. These examinations, which are due at different points throughout the course, have an impact on the students’ academic performance [3]. Additionally, there are differences between annual and semester systems in terms of how students learn [4]. It has been demonstrated that the structure of assessment tools, such as essay writing and objective assessment questions like MultipleChoice Questions (MCQs), has an impact on student performance, particularly in domains that evaluate more sophisticated cognitive capacities [5]. Students employ various learning tactics depending on a variety of criteria, like as the course they are enrolled in and the type of examination they take [2,6]. The teaching, learning, and assessment have all been impacted by the widespread movement for medical education reform [7]. The semester system, which is thought to offer more opportunity for assessments and feedback, has taken the place of the annual examination system in this environment of higher education reform [8]. The fundamental change to the semester system is the adoption of objective assessment methods. These methods frequently use MultipleChoice Questions (MCQs) as a tool for evaluating students’ knowledge of particular subjects and ask students to choose the right response from the given options [9].

Biggs [9] suggests that the institution’s environment for educational assessment influences students’ learning strategies. He asserts that assessments with only MCQs foster low cognitive learning based on the acquisition and memory of facts and knowledge, whereas assessments using essays, examine higher cognitive levels. Hence, MCQs cannot accurately assess the more sophisticated cognitive talents [5]. Students who solely answered MCQs in their earlier academic examinations struggled to write essays on basic science subjects when they were included in examinations later [10]. Smith and Miller [6] stated that students employ a higher level of cognitive processing in the essay-based examination than the MCQ examination when discussing students’ perceptions of these two types of examinations. In a study by Mujeeb and Pardeshi [11] the scores of medical students answering MCQs and Short Essay Questions (SEQs) was evaluated. A statistically significant positive correlation (r = 0.64, p=0.01) was discovered between the scores on MCQs and SEQs. However, for students who either failed an examination (r = 0.11, p = 0.08) or had a high distinction (r = -0.27, p = 0.13), no significant correlation was observed. The Modified Essay Question (MEQs) format, which combines MCQs and whole essays, is another type of essay test. However, it has been noted that they are challenging to create and frequently fall short of testing highly developed judgment and reasoning abilities as measured by essays, hence not frequently used [12].

The Bologna Declaration promoted the widespread adoption of semester system, encouraging many higher education institutions, including those in Pakistan, to adopt this scheduling strategy [13]. This declaration aspires to provide a consistent degree system from bachelor’s to doctoral levels [14] and to bring uniformity in higher education in European countries [13]. While data suggests the semester system could produce superior outcomes, unfortunately, the implementation of declaration is underdeveloped in some areas of medical education [15]. In a study published in 2016, Masic and Begic [16] looked at the academic performance of medical students in Bosnia and Herzegovina enrolled in the Bologna system’s annual examination model or semester examination model. The results of their study showed that students in the semester system outperformed academically those in the annual system (p 0.05). This beneficial impact transcends the realm of medical education, as Yousaf and Hashim found comparable results of enhanced performance among students in the semester system within the context of commerce education [17]. Students at several universities indicated greater satisfaction with the semester system of examinations than did teachers [18]. Despite the distinctions between the two systems, there is no discernible difference in the examination anxiety levels of the students [19]. The annual examination-based education system gives students more time to grasp the subject as compared to the semester-based education system with its semester-ending examinations [4]. Additionally, with frequent teacher-student contacts and evaluation feedback, the semester system offers students the chance for continual learning and assessment [20].

Pakistan followed the educational legacies left by postcolonial rule, especially the design of the medical curriculum. This covers the preclinical and clinical parts of medical education that are carried over from the British system of medical education, which included yearly examinations at the conclusion of academic years. However, alternative teaching, learning, and evaluation strategies have recently been introduced by several universities in Pakistan. The pre-clinical and clinical years made up the traditional and new curriculum structure used by medical universities in Pakistan. In the traditional curriculum, during preclinical years, courses like pharmacology were taught all year long and evaluated at the conclusion. The large lectures, small-group tutorials, and lab sessions were all used as teaching techniques. The written and oral assessments made up the parts of examinations. The written papers required in-depth responses to essay questions to be completed within a three-hour time limit. The appropriate faculty completed the grading manually. These techniques were widespread in Pakistani medical schools and universities before.

The higher education institutions of Pakistan’s now use a standardized semester system, according to policy guidelines circulated by the Higher Education Commission of Pakistan (HEC). The introduction of this method is being monitored and evaluated by the National Committee on Examination method (NCES). After deliberations with stakeholders the group created rules and advised semester-based examinations as the ideal method to standardize degree programs in Pakistan. According to the suggestions, the adoption of this system was supposed to be finished by 2008 [20]. The medical institutions also adopted the semester system in accordance with HEC guidelines. The Pakistan Medical and Dental Council (PMDC) regulates the courses, curricula, and evaluation procedures in the medical colleges and universities The PMDC, did not suggest any modifications to the course structure or modes of instruction to be used in semester system. As a result, with the previous course contents the single end-of-year examinations were replaced with end-of-semester examinations as part of other changes to assessment procedures. Accordingly, in semester examinations, the format of the question papers changed from lengthy descriptive essays to objective questions. The PMDC, however, has voiced discontent with the results of introducing the semester system in medical institutes and is thinking about going back to the original decision to abandon the annual system [21].

In pre-clinical undergraduate medicine, pharmacology is an essential subject for future doctors. However, the method of teaching, learning and assessment varies depending on the program context [22]. Pharmacology is a field that is constantly evolving. The use of newer drugs with increased and potentially dangerous drug interactions and toxicities for a wide range of important illnesses is commonly seen. A full understanding of the pharmacokinetics, pharmacodynamics, drug-drug interactions, and other characteristics of the substance being prescribed is therefore necessary for accurate medication dosing today. For doctors, nurses, and physician assistants, inadequate clinical pharmacologic training can frequently be traced back to undergraduate education. Hence, it is generally agreed that it is essential to regularly evaluate the teaching programs and changes adopted to improve those. Due to the constantly evolving types and repositories of pharmaceuticals as well as technology developments in educational processes, pharmacology education has gone through many stages in the evolution sequence. Revisions and updates to learning objectives, teaching-learning materials and techniques, and evaluation at the Undergraduate (UG) and Postgraduate (PG) levels were significant turning points in the evolutionary sequence. Teaching and evaluation methodologies used for pharmacology are undergoing several improvements [23-25].

Research Question

This aim of this research is to analyse the impact semester system has on the academic performance of students in the subject of pharmacology. There is a dearth of actual data in this regard. Information comparing academic achievement between the two systems, particularly in the field of medical education in Pakistan, is largely based on anecdotal evidence. The objective of this study was to compare between academic performance of students in the revised semester and conventional annual examination systems in preclinical discipline of Pharmacology.

Methods

This is a retrospective cohort study conducted in Pakistan. The data from public medical universities were required. There were two criteria of selection of the study sites. First, the site should have graduates who took annual and semester-based examinations. Second, the site should have at least one hundred students admitted each year. The universities which fulfilled the criteria were contacted to seek permission to have access to the data. Five universities were contacted by the researcher after ethical approval of the research project. Only two universities consented to provide the data. The access to the data was granted by the Liaquat University of Medical and Allied Health Sciences (LUMAHS) and the Peoples Medical College. The Liaquat University of Medical and Allied Health Sciences (LUMAHS) in Pakistan, formerly Liaquat Medical College was established in 1947. It evolved from an institution founded in 1881. Initially located in Hyderabad, it later moved to Jamshoro in 1963. LUMAHS offers MBBS and BDS courses and gained the status of a postgraduate medical institute in 1989. In 2001, it was elevated to the level of a university of medical and health sciences, admitting over 350 students each year. The Peoples Medical College, a prestigious public medical school for women, opened its doors in Nawabshah, Pakistan, in April 1974. The college has produced over 6,000 female doctors. In 2010, it received a name change to Shaheed Mohtarma Benazir Bhutto Medical University (SMBBMU). About 200 students are accepted each year into the MBBS program at SMBBMU, which has a similar curriculum to other public medical schools in Pakistan.

The data were the official records showing students’ assessment scores achieved during undergraduate medical education in the Pharmacology written examination component. The data were de-identified by the university staff with the student’s name replaced by a code before supplying it to the researcher. The IBM Statistical Programme for Social Sciences (SPSS) version 29 SPSS for windows was used for statistical analysis. The individual student data with missing values was omitted from the analysis. The data included end-of-semester and annual examinations scores in percentage format. A detailed descriptive analysis including test for normality of the data was carried out at the initial stage of study to explore and understand the characteristics of the population under study. Later, the student t test was conducted to analyse the dis/similarities between different groups of students who graduated overtime. Jaccard and Becker [26] pointed an argument between the use of parametric and non-parametric techniques among researchers. In social science research, the ideal normality of distribution is observed rarely. They argued that the parametric techniques are considered robust in producing results even if there are minor violations of the distributional assumptions. Hence, the frequency of different errors and accuracy of conclusions made are relatively unaffected compared with conditions when assumptions are met.

Results

The dataset provided was results of semester-based and annual examination results of various batches of students. The data of students (N=463) who graduated in year 2014 having semester system of examination and students (N=1480) who graduated in year 2013, 2012 and 2010 having annual system of examination were collected. The initial data analysis suggested that the data did not violate the assumptions of normality (Skewness = -0.46). The current data fulfilled the assumptions of parametric statistics. The descriptive statistics of the data are presented in Table 1 shows the of Pharmacology knowledge scores achieved by two groups of students. The students who took semester-based examination achieved (N=463, M= 71.2, SD= 8.7) higher scores than the students who took the conventional annual examination (N=1480, M=58.63, SD=9.50). The mean percentage scores of students who took the semester-based examination was statistically significantly higher (t (1941) = 25.32, P <.01, CI 95% (11.60-13.55) as shown in Table 2.

Examination system

n

Mean

SD

Semester system

463

71.21

8.73

Annual System

1480

58.63

9.5

Table 1: Descriptive data of students’ achievement scores in Pharmacology written examination component in two examination systems.

t

df

Sig. (2-tailed)

Mean difference

Std. Error difference

95% confidence interval of the difference

Lower

Upper

25.32

1941

0

12.58

0.5

11.6

13.55

Table 2: Student t Test to analyse the significance of difference between students’ achievement scores in Pharmacology theory component in two examination systems

Discussion

The results unequivocally show that students who took the semester-based examinations fared better than those who took the annual examinations in terms of accomplishment scores in written component of pharmacology subject in preclinical years. It shows that the HEC’s recommendations have brought positive changes in assessment process and its outcome. The changes, which shortened the length of the courses and adopted newer assessment process, seem to have improved students’ achievement. As the study by McCreary and Hausman [3] suggested that the interval between the time of the learning activities and the examinations has an impact on students’ academic achievement, this study also supports the deduction that the results from examinations held after a shorter duration in semester-based system are significantly higher than those from the longer duration annual system. Along with the length of the course after which examinations are given, assessment format, particularly when measuring more complex cognitive abilities, influences students’ academic progress [5]. The students of semester system did better because the semester system examination format with the use MCQs was distinct from the annual examination system format, supporting Frederiksen’s claim [5]. This is evident from the difference in the ranges of scores reported between two groups as seen. However, this difference can also be a result of the different learning strategies that students used to prepare for the two different types of examinations, as indicated by Smith and Miller [6].

Although students who took the semester system performed substantially better than those who took the annual examination, the ranges of scores are still important to note. The disparity in SDs, range, and interquartile range between two groups raises concerns. The narrow ranges of statistical values for the semester system cohort could be due to acquisition of the limited content needed to learn and recalled during the examination, as alluded to by Biggs (9). Biggs’ (9) theories on the cognitive talents measured in two different formats of assessment may explain the small range of scores in semester-based tests with objectively structured questions as opposed to descriptive essay type questions in the annual-based examination. He believed that examinations with essays measure greater cognitive levels than multiple-choice tests, which only instil low cognitive learning focused on the acquisition and retention of facts and knowledge. Examinations which include essay components are perceived more challenging by students [6]. This study supports this finding by demonstrating that students who wrote essays were more likely to receive lower grades than those who took objective tests, but with a wider range of scores. This suggests that some students will create far better descriptive analysis of pharmacology subject knowledge than others. This study also confirms the findings of Masic and Begic [16] showing students in the semester examination perform statistically higher (p < 0.05) as compared to the annual system. It is advised that more research be done to determine whether the difference in achievement scores is actually caused by the timing and format of assessment, condensation of the course material, or if other factors, such as gender, age, and the accessibility of teaching and learning resources, for example, are responsible. Additionally, studying the future treatment practices of two groups of graduates from two systems will be worthwhile.

Conclusion

This study emphasizes the educational impact on student performance that adjustments to the curriculum and evaluation procedures may have. The semester system-based examinations implemented in medical education institutions of Pakistani following the HEC’s instructions has shown positive outcome. Examinations in this system follow the guidelines of using objective assessment tools like MCQs. According to the study, students who took semester-based examinations fared better on the pharmacology knowledge evaluation than those who took endof-year examinations. This study only focused on the variations in academic performance in one fundamental science course that was taught and evaluated during the first years of medical school in two systems. To assess the effects of the semester system in the later clinical years, additional research is advised. This research project highlights the effects of changes made in medical education in Pakistan. Medical schools in Pakistan followed a traditional curriculum delivery and assessment methods. Recently there has been a great drive for the reforms in medical education including the assessment. A part of reform emphasizes the use of semester system of curriculum delivery and assessment with the use of objective assessment tools. The semester system of examination with objective assessment was recently adopted in medical institutes in Pakistan as a result of the reform of medical education. Regarding how this move will affect the academic performance of medical students, there is no empirical data. The purpose of this study was to evaluate if the semester examination system has affected medical students’ academic performance in any way compared to the annual examination system. This study adds to the fact that students who took the semester-based examination had higher scores. This study confirms studies done in other contexts and geographical locations. This study adds that shortening course duration and overhauling the assessment process had improved students’ performance. Additionally, the objective assessment approach reveals enhanced academic performance.

The impacts of the modifications made to Pakistan’s medical education are highlighted by this research effort. Pakistani medical schools used conventional methods for delivering their curricula and evaluating student progress. There has been a strong push recently for changes to medical education, particularly the assessment. The semester system of curriculum delivery and assessment with the use of unbiased assessment methods is emphasized as part of the reform. As a result of the reform of medical education, the semester system of examination with objective assessment has lately been established in medical colleges in Pakistan. There is no factual data on how this change will impact medical students’ academic performance. The goal of this study was to determine whether the semester examination method, as opposed to the annual examination system, had any impact on medical students’ academic performance. This study supports the finding that students who took the exam administered during a semester performed better. This study supports research conducted in various contexts and regions. According to this study, cutting the length of the course and changing the evaluation procedure has enhanced student performance. The objective assessment strategy also displays improved academic performance.

References

  1. Jama MP, Mapesela MLE, Beylefeld AA (2008) Theoretical perspectives on factors affecting the academic performance of students. South African Journal of Higher Education 22: 992-1005.
  2. Fardows N, Nayer D, Nayer S, Yousuf H (2016) Impact of education system (semester and annual system) on students’ performance in Pakistan. 8th International Conference on Education and New Learning Technologies; JUL 04-06, 2016 Barcelona, Spain Iated-Int Assoc Technology Education A& Development, Lauri 2016: 581-588.
  3. McCreary J, Hausman C (2001) Differences in Student Outcomes between Block, Semester, and Trimester Schedules. Report No. EA 031 346. Arlington, VA: Educational Research Service, Service  2001: 457-590.
  4. Khattak ZI, Ali M, Khan A, Khan S (2011) A study of English teachers and students’ perception about the differences between annual and semester system of education at postgraduate level in Mardan. Procedia - Social and Behavioral Sciences 15: 1639-1643.
  5. Frederiksen N (1984) The real test bias: Influences of testing on teaching and learning. American Psychologist 39: 193-202.
  6. Smith SN, Miller RJ (2005) Learning approaches: examination type, discipline of study, and gender. Educational Psychology 25: 43-53.
  7. Thompson AR, Braun MW, O’Loughlin VD (2013) A comparison of student performance on discipline-specific versus integrated exams in a medical school course. Adv Physiol Educ 37: 370-376.
  8. Virdi MS (2011) Reforming undergraduate dental education in India: introducing a credits and semester system. J Dent Educ 75: 15961602.
  9. Biggs J (1993) What do inventories of students’ learning processes really measure? A theoretical review and clarification. British Journal of Educational Psychology 63: 3-19.
  10. Rothman AI, Kerenyi N (1980) The assessment of an examination in pathology consisting of multiple-choice, practical and short essay questions. Med Educ 14: 341-344.
  11. Mujeeb AM, Pardeshi ML, Ghongane BB (2010) Comparative assessment of multiple choice questions versus short essay questions in pharmacology examinations. Indian J Med Sci 64: 118-124.
  12. Palmer EJ, Devitt PG (2007) Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ 7: 49.
  13. Patricio M, Den Engelsen C, Tseng D, Ten Cate O (2008) Implementation of the Bologna two-cycle system in medical education: Where do we stand in 2007? Results of an AMEE-MEDINE survey. Med Teach 30: 597-605.
  14. Neave G (2003) The Bologna Declaration: Some of the historic dilemmas posed by the reconstruction of the community in Europe’s systems of higher education. Educ Policy 17: 141-164.
  15. Lobato RD, Lagares A, Alen JF, Alday R (2010) [Implementation of the Bologna system in medical education. Current status and future prospects]. Neurocirugia 21: 146-156.
  16. Masic I, Begic E (2016) The Actual (Un)usefulness of the Bologna System in Medical Education. Med Arch 70: 158-163.
  17. Yousaf A, Hashim M (2012) A Case study of Annual and Semester Systems of Examination on Government College of Management Sciences, Peshawar, Pakistan. International journal of academic research in business and social sciences 2: 53.
  18. Hassan DA, Younis A, Sheik AA, Mahar M, Abbasi AZ (2012) Analyzing factors affecting students’ satisfaction regarding semester system in universities of Pakistan. Journal of American Science 8: 163-170.
  19. Rafiq (2007) Test anxiety in students: semester’s vs. annual system. Journal of behavioural sciences 2007: 17.
  20. Naqvi AS, Fatima S (2015) Hybrid Semester System in MBBS Program-Ziauddin Medical College Experience. Pakistan Journal of Medicine and Dentistry 4: 78-80.
  21. Pakistan Medical and Dental Council (2019) Minutes of the 195thsession of the council held on 2nd May 2019, at PM&DC Office 2019.
  22. Sarker SK, Bhas G, Moitra P, Paul R, Yusuf MA, et al. (2021) editors. Appraisal of the Current Pharmacology Curriculum, Teaching Methodology and Effectiveness in Undergraduate Medical Education: A Student-Centric, Questionnaire-Based Study 2021.
  23. Badyal D (2018) Evolution of pharmacology education in India: Past and future. Indian J Pharmacol 50: 159-168.
  24. Prasad, Kudthni RH, Santhosh R (2016) A questionnaire based study of students’ opinion on the teaching and learning methods in pharmacology in a medical school of India. International journal of basic and clinical pharmacology 5: 2589-2593.
  25. Wiernik PH (2015) A dangerous lack of pharmacology education in medical and nursing schools: A policy statement from the American College of Clinical Pharmacology. J Clin Pharmacol 55: 953-954.
  26. Jaccard J, Becker M (2002) Statistics for the Behavioral Sciences The United States of America: Wadsworth 2002.

© by the Authors & Gavin Publishers. This is an Open Access Journal Article Published Under Attribution-Share Alike CC BY-SA: Creative Commons Attribution-Share Alike 4.0 International License. With this license, readers can share, distribute, download, even commercially, as long as the original source is properly cited. Read More About Open Access Policy.

Journal of Surgery