SAJP 437

A problem-based learning curriculum and undergraduate performance in the final psychiatry examination at the Nelson R Mandela School of Medicine

S Saloojee,1 MB ChB, FCPsych (SA); J van Wyk,2 BSc, BEd, PhD

1 Department of Psychiatry, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa

2 Department of Clinical and Professional Practice, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: S Saloojee (saloojees1@ukzn.ac.za)


Background. Medical education reformers must consider disease patterns, health system expectations and clearly specified outcomes to ensure that revised curricula are relevant. South Africa needs clinically competent doctors in adequate numbers to address the burden of psychiatric illnesses.

Objective. To evaluate the impact of a curricular reform, this study compared undergraduate students’ clinical competence in psychiatry following a change from a six-year traditional lecture-based (LB) curriculum to a five-year problem-based learning (PBL) curriculum.

Method. The psychiatry examination records of 936 students enrolled in a PBL curriculum were compared with those of 771 students enrolled in a LB curriculum, covering a nine-year period from 2001 to 2009. Records covered the long case, case vignette and oral examinations.

Results. Students in the PBL group performed significantly better in the problem-solving case vignette examination (p<0.02). There were no statistically significant differences in the mean marks for the long case and the oral examination. Because the revised curriculum is shorter, one additional class of 200 students graduated during the duration of the study than would have been possible under the previous curriculum.

Conclusion. The new PBL curriculum produced more doctors, but there was no change in their psychiatric knowledge and skills compared with graduates from the old LB curriculum. Clinical teachers need to define outcomes prior to curriculum revision, because these are essential for evaluating the curriculum’s success.

S Afr J Psych 2013;19(4):218-221. DOI:10.7196/SAJP.437


In 2001, a problem-based learning (PBL) curriculum replaced the traditional, didactic lecture-based (LB) curriculum at the Nelson R Mandela School of Medicine (NRMSM) in Durban, South Africa (SA). This was prompted by the call from the Health Professions Council of South Africa (HPCSA’s) Medical and Dental Board to modernise curricula at all the country’s medical schools.1

PBL as a heterogeneous entity has gained wide acceptance across courses and disciplines in higher education.2 This approach considers both the content and instructional process of a curriculum, and is innovative in that it is a method of teaching as well as a philosophy of learning.3 Central to the approach is the use of problems, which are posed to students to initiate learning.

The approach was conceived at Canada’s McMaster University in 1969, and thereafter adopted at medical schools in Africa either fully or in a hybrid format. Costs for the successful implementation of PBL include library and resource development, refurbishment of small meeting rooms, laboratories for skills training, appointment of educationists and retraining of staff and facilitators. The costly implementation of PBL in the resource-constrained African setting left many clinical teachers wondering whether the reform would be worthwhile.4 In addition, despite decades of experience with PBL, the debate surrounding its benefits or superiority over traditional methods continues unabated.5-8

SA has a major shortage of doctors in the public sector and faces concomitant epidemics of HIV and tuberculosis.9 Due to high patient loads and limited supervision from specialists, it is important for students to be clinically competent on graduation. During their internship they will have to manage illnesses independently and often for prolonged periods under challenging circumstances.

There is evidence that students experience PBL environments as more nurturing and enjoyable, and some have argued that this positive learning environment is reason enough for PBL’s adoption.10

Psychiatry is one of the six major clinical disciplines at NRMSM, and the faculty-wide curriculum revision presented an opportunity to test the effect of the reform on students’ clinical competence in this discipline. We hypothesised that the revision should affect students’ performance positively in the final psychiatry clinical examinations, because the PBL curriculum provides students with (i) earlier clinical exposure to psychiatric patients than LB education and (ii) psychiatric cases that allow the application of basic science knowledge. It also offers (iii) systematic and sequential exposure to clinical presentations, which introduce foundational core principles before more complex presentations; (iv) active learning; and (v) enhancement of communication skills.

Curriculum comparison studies that focus on psychiatric knowledge and clinical skills are limited. A study conducted by McParland et al. 11 at a British medical school reported that the students from their PBL curriculum achieved statistically significant higher clinical and knowledge-based examination scores than students graduating from the LB curriculum. To date, no published studies have compared the knowledge and clinical skills of graduating students following the revision, in any of the six major clinical disciplines at our school. This study therefore compared the impact of the curricular revision on the psychiatric knowledge and clinical skills of undergraduate final-year medical students in the MB ChB programme at the NRMSM.

Methods

This quantitative assessment audit examined the extent to which the curricular reform affected the assessment outcomes of 10 cohorts of final-year medical students. The psychiatry examination results of five cohorts of students who graduated from the LB curriculum (2001 - 2005) were compared with those of five cohorts who graduated from the PBL curriculum (2005 - 2009) at the NRMSM. The use of archived student assessment data was approved by the Humanities and Social Sciences Research Ethics Committee of the University of KwaZulu-Natal.

Educational background

Both curricula covered the same content, and students rotated through blocks of equal duration to exactly the same teaching hospitals with the same exit outcomes expected from graduates. There were, however, concerns that the shorter PBL programme would hamper students’ ability to achieve clinical competence, and this was the main reason why the examination format remained unchanged. At the end of their degree, the final exit examination for the discipline required students to demonstrate their clinical competence, and consisted of a long case assessment, a written examination which included short questions based on two case vignettes, and an oral examination (viva).

The LB curriculum

Under the LB curriculum, MB ChBVI students spent three years learning pre-clinical outcomes, followed by three years focusing on clinical outcomes.

Students were introduced to the discipline of psychiatry only in their fourth year, and the undergraduate training programme in psychiatry aimed to instill students with the necessary disciplinary content knowledge and clinical skills to identify and treat psychiatric disorders prevalent in the South African primary healthcare and general hospital setting. The teaching method employed a number of didactic lectures. Clinical teaching and training in psychiatry continued through a series of clinical methods modules, offered as a four-week rotation to fifth-year students and as a six-week rotation in their final year. The exit examination occurred at the end of the six-week rotation in the final year.

The PBL curriculum

The PBL curriculum was five years long (MB ChBV) and has been described in detail elsewhere.12 The NRMSM adheres to Barrows’13 core characteristics for PBL curricula: the programme is student-centred and employs a recognisable sequence of events, where students work on a problem while meeting in small groups under the guidance of a facilitator. The authentic problem serves as a tool to direct students’ learning, as they gather new information through self-directed learning strategies and share it with the group in subsequent sessions.2 Learning was supported by lectures, skills training and clinical exposure, making this a hybrid rather than a pure PBL curriculum.

Psychiatry was introduced in a neuroscience module in the second year of study. Students were thus exposed to disciplinary content much earlier in the PBL curriculum than the LB curriculum. After the second-year module, students were expected to understand the diagnostic classification and clinical presentation of psychiatric disorders, and to be able to perform a mental state examination. As well as basic science, psychiatry was also integrated with other clinical disciplines, e.g. childhood psychiatric disorders were introduced in the ‘fever’ theme, which focused on growth and development and delirium. There was therefore a major shift from a discipline-based model to a more problem-based model. In keeping with the way in which PBL has been implemented globally, our model emphasised PBL to a greater extent in the pre-clinical than in the clinical years.

Assessing outcomes

The long case examination

In the final year of study, a traditional long case examination assesses students’ clinical skills. Students are given 90 minutes to take a history, conduct a mental state and physical examination, formulate a differential diagnosis and present a management plan for an untrained patient they have not previously encountered. The examination is uninterrupted and not directly observed. Students then present their findings to two examiners in a structured format. A final mark is based on the consensus of both the examiners.

The case vignette

A 2-hour pen-and-paper examination is used to assess students’ diagnostic reasoning skills. They are required to answer short questions with reference to two clinical scenarios. The questions related to each case are marked independently

The oral examination ( viva )

For 15 minutes, two examiners assess students’ knowledge on any topic covered in the clinical methods course in psychiatry. The oral examination is primarily a test of factual recall, although students are also expected to answer questions based on simulated case scenarios, demonstrating their ability to apply their knowledge.

Statistical analysis

Curriculum type was the independent variable in this study, and the dependent variables were the results of the long case examination, the case vignette and the viva. Gender, age and matriculation entrance points were the confounding variables.

Mann-Whitney U tests were performed on each of the outcome performance measures and p-values were calculated to show the differences between curricula. The level of significance was p<0.05. Stepwise multiple linear regression was used to control for possible confounding in the final year.

Results

Demographic characteristics

The current study analysed the psychiatry examination records of 771 students in the final year of the LB curriculum from 2000 to 2005, and compared them with those of 936 students in the final year of the PBL curriculum from 2005 to 2009.

Although there were more female students than males in both curricula, the proportion of female students graduating from the PBL curriculum was greater (Table 1). The mean age for final-year students was higher in the LB curriculum, because it was six years long while the PBL was only five years long.

The results of the multiple linear regression (Table 2 ) showed that students in the PBL curriculum performed significantly better in the problem-solving, case vignette-based examination than LB students (p=0.02). After adjusting for the confounding variables, there were no significant differences in the mean marks for the long case and oral examination between the two groups.





Effect of gender, age and matriculation entrance points on the examination marks

Generally, females, younger students and those who entered medical school with higher matriculation entrance points performed better in both curricula than males, older students and those with lower entrance scores.

Discussion

As assessed by their performance in the long case and oral examination, the psychiatric clinical skills of five cohorts of students emerging from a PBL curriculum were no different from those of five cohorts of students emerging from an LB curriculum. However, the students from the PBL cohort fared better when their ability to solve written case scenarios was assessed. This is hardly surprising, because written case scenarios are the primary trigger for learning in a PBL curriculum. Previous studies examining the effects of PBL on students’ problem-solving ability were unequivocally positive.3 , 7 However, although the PBL students were successful problem solvers, this was not generalisable to their clinical reasoning skills as applied to real patients in the long case examination. Clinical reasoning is a complex task and involves much more than the recall of previously learnt information.3

Our results are in keeping with several other studies which failed to show significant differences in medical competence following the introduction of PBL curricula.5 Hecker and Violato,14 utilising United States Medical Licensing Examination performance data from 116 medical schools, showed that various curricular reforms had little effect on students’ performance in the examination.

In keeping with the international literature, previous studies showed that our PBL graduates had positive perceptions of their own clinical skills, the PBL learning environment and respect for diversity within the student population, and were generally satisfied with the approach.15 , 16 These studies were retrospective evaluations of student and faculty perceptions regarding the curricular revision. Data were collected through surveys or interviews soliciting information from current and former students of the PBL curriculum. The impact of curriculum reform was examined mainly through an evaluation of students’ own perceptions of their clinical competencies. Also, none of the studies at our medical school reported on the satisfaction or perceptions of students enrolled in the old LB curriculum.

Notably, our findings support many earlier studies which found that PBL graduates did not receive lower ratings on assessments of clinical competency.5-7 The positive attributes of PBL were therefore not attained at the expense of clinical competence.

Additionally, the shorter PBL curriculum allowed the medical school to produce an additional 200 graduates compared with what was possible with lecture-based learning (LBL). SA has a low physician-to-population ratio of only 36 doctors per 100 000 in the public sector and there is a need to increase the annual number of medical graduates.9

By the time the curriculum was restructured at the NRMSM, it was globally recognised that teaching and learning in small groups requires more time, effort and resources. SA has a shortage of both resources and teaching staff. Clinical teaching staff at SA medical schools are mainly public sector specialists, but the number of specialists in this sector has declined.9 Clinical teaching staff have large case loads and are salaried employees of the Department of Health, for whom patient care is a priority. Clinical teachers also face considerable pressure to publish. The NRMSM Department of Psychiatry has no clinical teachers employed by the university or Department of Education. Innovative pedagogical approaches demand extra time and effort from clinicians, and this must be worthwhile.

Because this study was not an opinion poll and did not rely on student or staff surveys, its findings are not plagued by response fatigue or responder bias. It is also based on a large number of students graduating from one medical school. Furthermore, while many clinical rotations were revised during the nine-year period this study covered, psychiatry was unaffected by the revisions. The method of teaching psychiatry differed significantly between the two curricula but the assessment methods remain unchanged. Identical examination practices make the comparison more robust because there is a perception at the school that the examinations for PBL curricula were more lenient than those for LBL.

However, the study is limited by reporting on the clinical competency in only one discipline within the medical school, and these results may not reflect the overall clinical competencies of the students in the curriculum reform cohort. Comparative education research based on examination performance is also limited by assessment methods.7

Conclusion

Regardless of curriculm, the two cohorts of students displayed the same level of clinical skills in psychiatry.

This study did find that PBL had a positive effect on students’ ability to solve problems based on written case scenarios, but no striking difference in their knowledge or clinical skills. We therefore conclude that although doctors produced under both curricula possessed similar clinical psychiatric skills, it is not possible to make definitive statements regarding the success or failure of the revision. The expected outcomes of the revision were not clearly specified at the outset, so whether it was worthwhile or not is debatable.

We advocate an outcome-based approach for future curriculum innovations in order to provide a framework for the evaluation of such revisions.

This study’s contribution to global medical educational discourse is the finding that students’ graduating from the PBL curriculum at the NRMSM did not have improved psychiatric clinical skills, but were certainly not worse off than their LBL counterparts. Students trod an alternative path from novice to doctor, but although the road was shorter the outcomes were similar. Our study also emphasises that curricular evaluations are more meaningful when informed by well-defined predetermined outcomes.

References
    1. Seggie JL. MB ChB curriculum modernisation in South Africa – growing doctors for Africa. African Journal of Health Professions Education 2010;1(2):8-14.

    1. Seggie JL. MB ChB curriculum modernisation in South Africa – growing doctors for Africa. African Journal of Health Professions Education 2010;1(2):8-14.

    2. Maudsley G. Do we all mean the same thing by ‘problem-based learning’? A review of the concepts and a formulation of the ground rules. Acad Med 1999;74(2):178-185. [http://dx.doi.org/10.1097%2F00001888-199902000-00016]

    2. Maudsley G. Do we all mean the same thing by ‘problem-based learning’? A review of the concepts and a formulation of the ground rules. Acad Med 1999;74(2):178-185. [http://dx.doi.org/10.1097%2F00001888-199902000-00016]

    3. Patel VL, Groen GJ, Norman GR. Effects of conventional and problem-based medical curricula on problem solving. Acad Med 1991;66(7):380-389. [http://dx.doi.org/10.1097%2F00001888-199107000-00002]

    3. Patel VL, Groen GJ, Norman GR. Effects of conventional and problem-based medical curricula on problem solving. Acad Med 1991;66(7):380-389. [http://dx.doi.org/10.1097%2F00001888-199107000-00002]

    4. Mufunda J, Chatora R, Ndambakuwa Y, Samkange C, Sigola L,Vengesa P. Challenges in training the ideal doctor for Africa: Lessons learned from Zimbabwe. Med Teach 2007;29(9):878-881. [http://dx.doi.org/10.1080/01421590701813007]

    4. Mufunda J, Chatora R, Ndambakuwa Y, Samkange C, Sigola L,Vengesa P. Challenges in training the ideal doctor for Africa: Lessons learned from Zimbabwe. Med Teach 2007;29(9):878-881. [http://dx.doi.org/10.1080/01421590701813007]

    5. Colliver JA. Effectiveness of problem-based learning curricula: Research and theory. Acad Med 2000;75(3):259-266. [http://dx.doi.org/10.1097%2F00001888-200003000-00017]

    5. Colliver JA. Effectiveness of problem-based learning curricula: Research and theory. Acad Med 2000;75(3):259-266. [http://dx.doi.org/10.1097%2F00001888-200003000-00017]

    6. Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: A meta-analysis. Learning and Instruction 2003;13(5):533-568. [http://dx.doi.org/10.1016%2FS0959-4752%2802%2900025-7]

    6. Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: A meta-analysis. Learning and Instruction 2003;13(5):533-568. [http://dx.doi.org/10.1016%2FS0959-4752%2802%2900025-7]

    7. Gijbels D, Dochy F, Van den Bossche P, Segers M. Effects of problem-based learning: A meta-analysis from the angle of assessment. Rev Educ Res 2005;75(1):27-61. [http://dx.doi.org/10.3102%2F00346543075001027]

    7. Gijbels D, Dochy F, Van den Bossche P, Segers M. Effects of problem-based learning: A meta-analysis from the angle of assessment. Rev Educ Res 2005;75(1):27-61. [http://dx.doi.org/10.3102%2F00346543075001027]

    8. Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract 2009;18(1):1-9. [http://dx.doi.org/10.1159/00016303]

    8. Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract 2009;18(1):1-9. [http://dx.doi.org/10.1159/00016303]

    9. Strachan B, Zabow T, Van der Spuy ZM. More doctors and dentists are needed in South Africa. S Afr Med J 2011;101(8):522-528. [http://dx.doi.org/10.1080%2F10246029.2011.630808]

    9. Strachan B, Zabow T, Van der Spuy ZM. More doctors and dentists are needed in South Africa. S Afr Med J 2011;101(8):522-528. [http://dx.doi.org/10.1080%2F10246029.2011.630808]

    10. Albanese M. Problem-based learning: Why curricula are likely to show little effect on knowledge and clinical skills. Medical Education 2000;34(9):729-738. [http://dx.doi.org/10.1046%2Fj.1365-2923.2000.00753.x]

    10. Albanese M. Problem-based learning: Why curricula are likely to show little effect on knowledge and clinical skills. Medical Education 2000;34(9):729-738. [http://dx.doi.org/10.1046%2Fj.1365-2923.2000.00753.x]

    11. McParland M, Noble ML, Livingston G. The effectiveness of problem based learning compared to traditional teaching in undergraduate psychiatry. Med Educ 2004;38(8):859-867. [http://dx.doi.org/10.1111/j.1365-2929.2004.01818.x]

    11. McParland M, Noble ML, Livingston G. The effectiveness of problem based learning compared to traditional teaching in undergraduate psychiatry. Med Educ 2004;38(8):859-867. [http://dx.doi.org/10.1111/j.1365-2929.2004.01818.x]

    12. Van Wyk J, Madiba T. Problem-based learning at the Nelson R Mandela School of Medicine. East and Central African Journal of Surgery 2006;11:3-9.

    12. Van Wyk J, Madiba T. Problem-based learning at the Nelson R Mandela School of Medicine. East and Central African Journal of Surgery 2006;11:3-9.

    13. Barrows HS. A taxonomy of problem-based learning methods. Med Educ 1986;20(6):481-486 [http://dx.doi/10.1111/j.1365-2923.1986.tb01386.x]

    13. Barrows HS. A taxonomy of problem-based learning methods. Med Educ 1986;20(6):481-486 [http://dx.doi/10.1111/j.1365-2923.1986.tb01386.x]

    14. Hecker K, Violato C. Medical school curricula: Do curricular approaches affect competence in medicine? Fam Med 2009;41(6):420-426.

    14. Hecker K, Violato C. Medical school curricula: Do curricular approaches affect competence in medicine? Fam Med 2009;41(6):420-426.

    15. Singaram VS, Dolmans DH, Lachman N, van der Vleuten CPM. Perceptions of problem-based learning (PBL) group effectiveness in a socially-culturally diverse medical student population. Education Health (Abingdon) 2008;21(2):1-9.

    15. Singaram VS, Dolmans DH, Lachman N, van der Vleuten CPM. Perceptions of problem-based learning (PBL) group effectiveness in a socially-culturally diverse medical student population. Education Health (Abingdon) 2008;21(2):1-9.

    16. Reddy S. Experiences of Clinical Practice in a Problem-Based Learning Medical Curriculum and Subsequent Clinical Environments. PhD Thesis submitted to the University of KwaZulu-Natal School of Education; 2010.

    16. Reddy S. Experiences of Clinical Practice in a Problem-Based Learning Medical Curriculum and Subsequent Clinical Environments. PhD Thesis submitted to the University of KwaZulu-Natal School of Education; 2010.



Submitted: 15 February 2013
Published: 30 November 2013




African Online Scientific Information Systems (Pty) Ltd t/a AOSIS
Reg No: 2002/002017/07
RSA Tel: 086 1000 381
International Tel: +27 21 975 2602
15 Oxford Street, Durbanville, Cape Town, 7550, South Africa
publishing(AT)aosis.co.za replace (AT) with @

All articles published in this journal are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license, unless otherwise stated.
Website design & content: ©2018 AOSIS (Pty) Ltd. All rights reserved. No unauthorised duplication allowed.
By continuing to use this website, you agree to our Privacy Policy.

Subscribe to our newsletter

Get specific, domain-collection newsletters detailing the latest CPD courses, scholarly research and call-for-papers in your field.

Subscribe

South African Journal of Psychiatry    |    ISSN: 1608-9685 (PRINT)    |    ISSN: 2078-6786 (ONLINE)