Email this article 
Hide
Show all
A problem-based learning curriculum and undergraduate performance in the final psychiatry examination at the Nelson R Mandela School of Medicine
1 Department of Psychiatry, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa
2
Department of Clinical and Professional
Practice, Nelson R Mandela School of Medicine, University of
KwaZulu-Natal, Durban, South Africa
Corresponding
author:
S Saloojee
(saloojees1@ukzn.ac.za)
Background. Medical education reformers must consider disease patterns, health system expectations and clearly specified outcomes to ensure that revised curricula are relevant. South Africa needs clinically competent doctors in adequate numbers to address the burden of psychiatric illnesses.
Objective. To evaluate the impact of a curricular reform, this study compared undergraduate students’ clinical competence in psychiatry following a change from a six-year traditional lecture-based (LB) curriculum to a five-year problem-based learning (PBL) curriculum.
Method. The psychiatry examination records of 936 students enrolled in a PBL curriculum were compared with those of 771 students enrolled in a LB curriculum, covering a nine-year period from 2001 to 2009. Records covered the long case, case vignette and oral examinations.
Results. Students in the PBL group performed significantly better in the problem-solving case vignette examination (p<0.02). There were no statistically significant differences in the mean marks for the long case and the oral examination. Because the revised curriculum is shorter, one additional class of 200 students graduated during the duration of the study than would have been possible under the previous curriculum.
Conclusion. The new PBL curriculum
produced more doctors, but there was no change in their
psychiatric knowledge and skills compared with graduates from
the old LB curriculum. Clinical teachers need to define outcomes
prior to curriculum revision, because these are essential for
evaluating the curriculum’s success.
S Afr J Psych 2013;19(4):218-221.
DOI:10.7196/SAJP.437
In 2001, a problem-based learning (PBL) curriculum replaced the traditional, didactic lecture-based (LB) curriculum at the Nelson R Mandela School of Medicine (NRMSM) in Durban, South Africa (SA). This was prompted by the call from the Health Professions Council of South Africa (HPCSA’s) Medical and Dental Board to modernise curricula at all the country’s medical schools.1
PBL as a heterogeneous entity has gained wide acceptance across courses and disciplines in higher education.2 This approach considers both the content and instructional process of a curriculum, and is innovative in that it is a method of teaching as well as a philosophy of learning.3 Central to the approach is the use of problems, which are posed to students to initiate learning.
The approach was conceived at Canada’s McMaster University in 1969, and thereafter adopted at medical schools in Africa either fully or in a hybrid format. Costs for the successful implementation of PBL include library and resource development, refurbishment of small meeting rooms, laboratories for skills training, appointment of educationists and retraining of staff and facilitators. The costly implementation of PBL in the resource-constrained African setting left many clinical teachers wondering whether the reform would be worthwhile.4 In addition, despite decades of experience with PBL, the debate surrounding its benefits or superiority over traditional methods continues unabated.5-8
SA has a major shortage of doctors in the public sector and faces concomitant epidemics of HIV and tuberculosis.9 Due to high patient loads and limited supervision from specialists, it is important for students to be clinically competent on graduation. During their internship they will have to manage illnesses independently and often for prolonged periods under challenging circumstances.
There is evidence that students experience PBL environments as more nurturing and enjoyable, and some have argued that this positive learning environment is reason enough for PBL’s adoption.10
Psychiatry is one of the six major clinical disciplines at NRMSM, and the faculty-wide curriculum revision presented an opportunity to test the effect of the reform on students’ clinical competence in this discipline. We hypothesised that the revision should affect students’ performance positively in the final psychiatry clinical examinations, because the PBL curriculum provides students with (i) earlier clinical exposure to psychiatric patients than LB education and (ii) psychiatric cases that allow the application of basic science knowledge. It also offers (iii) systematic and sequential exposure to clinical presentations, which introduce foundational core principles before more complex presentations; (iv) active learning; and (v) enhancement of communication skills.
Curriculum
comparison studies that focus on psychiatric knowledge and
clinical skills are limited. A study conducted by McParland et
al. 11 at a British medical school
reported that the students from their PBL curriculum achieved
statistically significant higher clinical and knowledge-based
examination scores than students graduating from the LB
curriculum. To date, no published studies have compared the
knowledge and clinical skills of graduating students following
the revision, in any of the six major clinical disciplines at
our school. This study therefore compared the impact of the
curricular revision on the psychiatric knowledge and clinical
skills of undergraduate final-year medical students in the MB
ChB programme at the NRMSM.
Methods
This quantitative assessment audit examined the extent to which
the curricular reform affected the assessment outcomes of 10
cohorts of final-year medical students. The psychiatry
examination results of five cohorts of students who graduated
from the LB curriculum (2001 - 2005) were compared with those of
five cohorts who graduated from the PBL curriculum (2005 - 2009)
at the NRMSM. The use of archived student assessment data was
approved by the Humanities and Social Sciences Research Ethics
Committee of the University of KwaZulu-Natal.
Educational background
Both curricula covered the same content, and students rotated
through blocks of equal duration to exactly the same teaching
hospitals with the same exit outcomes expected from graduates.
There were, however, concerns that the shorter PBL programme
would hamper students’ ability to achieve clinical competence,
and this was the main reason why the examination format remained
unchanged. At the end of their degree, the final exit
examination for the discipline required students to demonstrate
their clinical competence, and consisted of a long case
assessment, a written examination which included short questions
based on two case vignettes, and an oral examination (viva).
The LB curriculum
Under the LB curriculum, MB ChBVI students spent three years learning pre-clinical outcomes, followed by three years focusing on clinical outcomes.
Students were introduced to the discipline of
psychiatry only in their fourth year, and the undergraduate
training programme in psychiatry aimed to instill students with
the necessary disciplinary content knowledge and clinical skills
to identify and treat psychiatric disorders prevalent in the
South African primary healthcare and general hospital setting.
The teaching method employed a number of didactic lectures.
Clinical teaching and training in psychiatry continued through a
series of clinical methods modules, offered as a four-week
rotation to fifth-year students and as a six-week rotation in
their final year. The exit examination occurred at the end of
the six-week rotation in the final year.
The PBL curriculum
The PBL curriculum was five years long (MB ChBV) and has been described in detail elsewhere.12 The NRMSM adheres to Barrows’13 core characteristics for PBL curricula: the programme is student-centred and employs a recognisable sequence of events, where students work on a problem while meeting in small groups under the guidance of a facilitator. The authentic problem serves as a tool to direct students’ learning, as they gather new information through self-directed learning strategies and share it with the group in subsequent sessions.2 Learning was supported by lectures, skills training and clinical exposure, making this a hybrid rather than a pure PBL curriculum.
Psychiatry was introduced in a neuroscience
module in the second year of study. Students were thus exposed
to disciplinary content much earlier in the PBL curriculum than
the LB curriculum. After the second-year module, students were
expected to understand the diagnostic classification and
clinical presentation of psychiatric disorders, and to be able
to perform a mental state examination. As well as basic science,
psychiatry was also integrated with other clinical disciplines,
e.g. childhood psychiatric disorders were introduced in the
‘fever’ theme, which focused on growth and development and
delirium. There was therefore a major shift from a
discipline-based model to a more problem-based model. In keeping
with the way in which PBL has been implemented globally, our
model emphasised PBL to a greater extent in the pre-clinical
than in the clinical years.
Assessing outcomes
The long case examination
In the final year of study, a traditional long case examination
assesses students’ clinical skills. Students are given 90
minutes to take a history, conduct a mental state and physical
examination, formulate a differential diagnosis and present a
management plan for an untrained patient they have not
previously encountered. The examination is uninterrupted and not
directly observed. Students then present their findings to two
examiners in a structured format. A final mark is based on the
consensus of both the examiners.
The case vignette
A 2-hour pen-and-paper examination is used to assess students’
diagnostic reasoning skills. They are required to answer short
questions with reference to two clinical scenarios. The
questions related to each case are marked independently
The oral examination ( viva )
For 15 minutes, two examiners assess students’ knowledge on any
topic covered in the clinical methods course in psychiatry. The
oral examination is primarily a test of factual recall, although
students are also expected to answer questions based on
simulated case scenarios, demonstrating their ability to apply
their knowledge.
Statistical analysis
Curriculum type was the independent variable in this study, and the dependent variables were the results of the long case examination, the case vignette and the viva. Gender, age and matriculation entrance points were the confounding variables.
Mann-Whitney
U tests were performed on each of the outcome performance
measures and p-values were calculated to show
the differences between curricula. The level of significance
was p<0.05. Stepwise multiple
linear regression was used to control for possible confounding
in the final year.
Results
Demographic characteristics
The current study analysed the psychiatry examination records of 771 students in the final year of the LB curriculum from 2000 to 2005, and compared them with those of 936 students in the final year of the PBL curriculum from 2005 to 2009.
Although there were more female students than males in both curricula, the proportion of female students graduating from the PBL curriculum was greater (Table 1). The mean age for final-year students was higher in the LB curriculum, because it was six years long while the PBL was only five years long.
The results of the multiple linear regression
(Table 2 ) showed that students in the PBL curriculum performed
significantly better in the problem-solving, case vignette-based
examination than LB students (p=0.02).
After adjusting for the confounding variables, there were no
significant differences in the mean marks for the long case and
oral examination between the two groups.
Effect of gender, age and matriculation entrance points on the examination marks
Generally, females, younger students and those who entered
medical school with higher matriculation entrance points
performed better in both curricula than males, older students
and those with lower entrance scores.
Discussion
As assessed by their performance in the long case and oral examination, the psychiatric clinical skills of five cohorts of students emerging from a PBL curriculum were no different from those of five cohorts of students emerging from an LB curriculum. However, the students from the PBL cohort fared better when their ability to solve written case scenarios was assessed. This is hardly surprising, because written case scenarios are the primary trigger for learning in a PBL curriculum. Previous studies examining the effects of PBL on students’ problem-solving ability were unequivocally positive.3 , 7 However, although the PBL students were successful problem solvers, this was not generalisable to their clinical reasoning skills as applied to real patients in the long case examination. Clinical reasoning is a complex task and involves much more than the recall of previously learnt information.3
Our results are in keeping with several other studies which failed to show significant differences in medical competence following the introduction of PBL curricula.5 Hecker and Violato,14 utilising United States Medical Licensing Examination performance data from 116 medical schools, showed that various curricular reforms had little effect on students’ performance in the examination.
In keeping with the international literature, previous studies showed that our PBL graduates had positive perceptions of their own clinical skills, the PBL learning environment and respect for diversity within the student population, and were generally satisfied with the approach.15 , 16 These studies were retrospective evaluations of student and faculty perceptions regarding the curricular revision. Data were collected through surveys or interviews soliciting information from current and former students of the PBL curriculum. The impact of curriculum reform was examined mainly through an evaluation of students’ own perceptions of their clinical competencies. Also, none of the studies at our medical school reported on the satisfaction or perceptions of students enrolled in the old LB curriculum.
Notably, our findings support many earlier studies which found that PBL graduates did not receive lower ratings on assessments of clinical competency.5-7 The positive attributes of PBL were therefore not attained at the expense of clinical competence.
Additionally, the shorter PBL curriculum allowed the medical school to produce an additional 200 graduates compared with what was possible with lecture-based learning (LBL). SA has a low physician-to-population ratio of only 36 doctors per 100 000 in the public sector and there is a need to increase the annual number of medical graduates.9
By the time the curriculum was restructured at the NRMSM, it was globally recognised that teaching and learning in small groups requires more time, effort and resources. SA has a shortage of both resources and teaching staff. Clinical teaching staff at SA medical schools are mainly public sector specialists, but the number of specialists in this sector has declined.9 Clinical teaching staff have large case loads and are salaried employees of the Department of Health, for whom patient care is a priority. Clinical teachers also face considerable pressure to publish. The NRMSM Department of Psychiatry has no clinical teachers employed by the university or Department of Education. Innovative pedagogical approaches demand extra time and effort from clinicians, and this must be worthwhile.
Because this study was not an opinion poll and did not rely on student or staff surveys, its findings are not plagued by response fatigue or responder bias. It is also based on a large number of students graduating from one medical school. Furthermore, while many clinical rotations were revised during the nine-year period this study covered, psychiatry was unaffected by the revisions. The method of teaching psychiatry differed significantly between the two curricula but the assessment methods remain unchanged. Identical examination practices make the comparison more robust because there is a perception at the school that the examinations for PBL curricula were more lenient than those for LBL.
However, the study is limited by reporting on the clinical competency in only one discipline within the medical school, and these results may not reflect the overall clinical competencies of the students in the curriculum reform cohort. Comparative education research based on examination performance is also limited by assessment methods.7
Conclusion
Regardless of curriculm, the two cohorts of students displayed the same level of clinical skills in psychiatry.
This study did find that PBL had a positive effect on students’ ability to solve problems based on written case scenarios, but no striking difference in their knowledge or clinical skills. We therefore conclude that although doctors produced under both curricula possessed similar clinical psychiatric skills, it is not possible to make definitive statements regarding the success or failure of the revision. The expected outcomes of the revision were not clearly specified at the outset, so whether it was worthwhile or not is debatable.
We advocate an outcome-based approach for future curriculum innovations in order to provide a framework for the evaluation of such revisions.
This study’s contribution to global medical
educational discourse is the finding that students’ graduating
from the PBL curriculum at the NRMSM did not have improved
psychiatric clinical skills, but were certainly not worse off
than their LBL counterparts. Students trod an alternative path
from novice to doctor, but although the road was shorter the
outcomes were similar. Our study also emphasises that curricular
evaluations are more meaningful when informed by well-defined
predetermined outcomes.
References
1. Seggie JL. MB ChB curriculum modernisation in South Africa – growing doctors for Africa. African Journal of Health Professions Education 2010;1(2):8-14.
2. Maudsley G. Do we all mean the same thing by ‘problem-based learning’? A review of the concepts and a formulation of the ground rules. Acad Med 1999;74(2):178-185. [http://dx.doi.org/10.1097%2F00001888-199902000-00016]
3. Patel VL, Groen GJ, Norman GR. Effects of conventional and problem-based medical curricula on problem solving. Acad Med 1991;66(7):380-389. [http://dx.doi.org/10.1097%2F00001888-199107000-00002]
4. Mufunda J, Chatora R, Ndambakuwa Y, Samkange C, Sigola L,Vengesa P. Challenges in training the ideal doctor for Africa: Lessons learned from Zimbabwe. Med Teach 2007;29(9):878-881. [http://dx.doi.org/10.1080/01421590701813007]
5. Colliver JA. Effectiveness of problem-based learning curricula: Research and theory. Acad Med 2000;75(3):259-266. [http://dx.doi.org/10.1097%2F00001888-200003000-00017]
6. Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: A meta-analysis. Learning and Instruction 2003;13(5):533-568. [http://dx.doi.org/10.1016%2FS0959-4752%2802%2900025-7]
7. Gijbels D, Dochy F, Van den Bossche P, Segers M. Effects of problem-based learning: A meta-analysis from the angle of assessment. Rev Educ Res 2005;75(1):27-61. [http://dx.doi.org/10.3102%2F00346543075001027]
8. Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract 2009;18(1):1-9. [http://dx.doi.org/10.1159/00016303]
9. Strachan B, Zabow T, Van der Spuy ZM. More doctors and dentists are needed in South Africa. S Afr Med J 2011;101(8):522-528. [http://dx.doi.org/10.1080%2F10246029.2011.630808]
10. Albanese M. Problem-based learning: Why curricula are likely to show little effect on knowledge and clinical skills. Medical Education 2000;34(9):729-738. [http://dx.doi.org/10.1046%2Fj.1365-2923.2000.00753.x]
11. McParland M, Noble ML, Livingston G. The effectiveness of problem based learning compared to traditional teaching in undergraduate psychiatry. Med Educ 2004;38(8):859-867. [http://dx.doi.org/10.1111/j.1365-2929.2004.01818.x]
12. Van Wyk J, Madiba T. Problem-based learning at the Nelson R Mandela School of Medicine. East and Central African Journal of Surgery 2006;11:3-9.
13. Barrows HS. A taxonomy of problem-based learning methods. Med Educ 1986;20(6):481-486 [http://dx.doi/10.1111/j.1365-2923.1986.tb01386.x]
14. Hecker K, Violato C. Medical school curricula: Do curricular approaches affect competence in medicine? Fam Med 2009;41(6):420-426.
15. Singaram VS, Dolmans DH, Lachman N, van der Vleuten CPM. Perceptions of problem-based learning (PBL) group effectiveness in a socially-culturally diverse medical student population. Education Health (Abingdon) 2008;21(2):1-9.
16. Reddy S. Experiences of Clinical Practice in a Problem-Based Learning Medical Curriculum and Subsequent Clinical Environments. PhD Thesis submitted to the University of KwaZulu-Natal School of Education; 2010.