Research Article - (2014) Volume 2, Issue 1

Designing, Implementing, and Evaluating a Tutor Guide for Problem Based Learning Phase II Class Tutors at the Faculty of Medicine, Suez Canal University

Marwa Ahmed Abd El-Aziz El Naggar1*, Fathi Abdel Hamid Maklady2, Adel Morshedy Hamam3 and Aziza Sayed Omar4
1Medical Education Department, Faculty of Medicine, Dubai Medical College for Girls, UAE
2Cardiology, Faculty of Medicine, Suez Canal University, Egypt
3Orthopedics, Faculty of Medicine, Suez Canal University, Egypt
4Rheumatology and Rehabilitation, Faculty of Medicine, Suez Canal University, Egypt
*Corresponding Author: Marwa Ahmed Abd El-Aziz El Naggar, Medical Education Department, Faculty of Medicine, Dubai Medical College for Girls, UAE Email:

Abstract

Background: The Faculty of Medicine, Suez Canal University (FOM-SCU) was established in 1978 to become the first Problem-Based (PBL) school in the Middle East Tutor plays a major role in facilitating PBL sessions, Tutor should be well prepared and have the sufficient knowledge and facilitation skills to effectively facilitate PBL sessions.
Aim: The aim of this study was to design, implement, and evaluate a tutor guide for PBL class tutors in phase II at (FOM-SCU) the in order to help tutors facilitate PBL sessions in a competent manner and increase effectiveness of the PBL sessions.
Materials and methods: A quasi-experimental, Pre-program/post-program non-equivalent comparison group design was applied in this study. The target population was randomly assigned to intervention and control groups, a total number of 28 tutors in each group. The study passed through three stages: preparatory, design and implementation and evaluation. The data was collected by: needs assessment questionnaire, tutors self-satisfaction questionnaire, student satisfaction questionnaire administered before and after the implementation, questionnaire to assess tutors’ and students’ satisfaction with the developed tutor guide and “what if” mini cases.
Results: Needs assessment results shows an urgent need for a tutor guide for some educational blocks. Results also showed improvement of the intervention group performance. The average score for the overall performance of the tutor was (7.67 ± 1.20) in the intervention group compared with the control group (6.54 ± 2.02).
Conclusion: The study concluded that implementing a tutor guide increases tutors’ self satisfaction with their performance and enhances students’ satisfaction with tutor performance.

Keywords: Tutor guide; Tutor training; PBL; What if …mini cases; Concept mapping.

Introduction

The Faculty of Medicine, Suez Canal University (FOM-SCU) was established in 1978 to become the first Problem-Based, community oriented/based and student-centered school in the Middle East. The main objective of its establishment was to meet the health needs of the four Suez Canal Governorates and Sinai [1].

The tutorial process begins when an authentic problem is presented to a small group of students. In addition to authentic, problems should be complex and ill-structured [2]. Once students are presented with the problem, they follow an analysis process of determining what they, collectively, know about the problem and what they need to know to solve the problem. Students are then expected to individually use resources they discover for themselves, to acquire the knowledge required for solving the problem [3].

Tutor should be well prepared in the subject so that he /she have sufficient knowledge to distinguish between sense and nonsense, between main points and side-issues Best tutor is the subject-matter expert who understands the course and the curriculum and who has the appropriate group facilitation skills. The second choice would be a medically qualified member of staff who is not an expert in the area but who understand the course and the curriculum and has the appropriate group facilitation skills. The third choice would be someone who has an understanding of the curriculum and the appropriate group facilitation skills but who does not have medical understanding or knowledge. There are, however, many examples of where non-expert and nonmedically qualified facilitators function effectively. What are essential; however, are group facilitation skills and appropriate personal qualities [4].

The FOM-SCU- based health intelligence unit called Problem Formulation Research Lab played a key leadership and coordinating role in the curriculum update process. This unit was establishes in 1990. The lab’s activities include research both into health problems and into format options for problems that appear in the curriculum, analysis and evaluation of the problems used in existing curricula, and training of the faculty in problem formulation and design [1]. Since the establishment of the Faculty, educational problems were associated with tutor guide, but with time members of problem formulation committee lost their motivation to design a tutor guide to be associated with the educational problems, despite that it is widely used in all schools of medicine implementing PBL as a strategy for learning. In order for the tutor to understand the course and the curriculum, he/she needs a tutor guide. This guide should contain guidelines for teachers who facilitate the PBL tutorial sessions. To fulfill the tutor role adequately, the tutor should have some knowledge about the discipline, sufficient understanding of the subject in question, and know in what direction the discussion in the group should be headed. Because the block is multidisciplinary, a tutor will be an expert for particular problems and non-expert or semi-expert for other problems included in the single block. The tutor will only have sufficient experience for some parts of the problems. This means that tutors should be provided with background information about the problems. This should enable the tutor to ask appropriate questions to stimulate the discussion in the group. A problem guide will helps tutors become “case experts” in both the content and process of the case. Therefore this study will be carried out to draw a road map toward introducing a well designed and tested prototype of tutor guide to help in reform in the system of problems formulation by Problem Formulation Committee at the FOM-SCU.

Problem based learning (PBL)

Problem based learning (PBL) can be defined as “a method of learning in which the learners first encounter a problem, followed by a systematic, student-centered enquiry process [5]. PBL is considered as a pedagogical strategy for posing significant, contextualized, real world situations, and providing resources, guidance, and instruction to learners as they develop content knowledge and problem-solving skills [6].

According to Barrows [2] the educational objectives of PBL are to develop knowledge that is better retained, usable in a clinical context and integrated from many disciplines.

• To provide a learning method that is; student-centered, motivating for the students and relevant to a career in medicine.

• To develop clinical reasoning and problem-solving skills.

• To promote interpersonal skills and the ability to work as a team member.

• To develop independent, self-directed critical thinking and learning skills.

• To encourage sensitivity to the entire patient’s needs.

Class tutors in problem-based learning: The tutor is a university educator who leads a task-oriented group to successfully achieve the objectives of a teaching program. In doing this, the tutor has to fulfill several responsibilities and is accountable to the teaching program for the satisfactory completion of them. PBL sessions are more as professional strategy meetings than teaching sessions [7].

The function of the tutor in PBL differs considerably from that of the tutor in conventional tutorials in which the tutor assumes a comparatively didactic role. A major feature of PBL is that learning is student-centered in that students take responsibility for identifying and addressing their own learning needs; tutors are required to facilitate this rather than adopt the position of content expert. Facilitation requires understanding of the learning process and primarily involves monitoring of student learning and promotion of effective group function. The student-centered learning approach of PBL means that for tutors, content knowledge should be subordinate to proficiency in group facilitation [4]. These responsibilities require abilities and skills relevant to the principles and practice of problem-based learning, group dynamics, the assessment of student learning, the use of learning resources and managerial skills. The tutor in a problem based tutorial group should be to facilitate the learning of students rather than to convey knowledge [8].

The role of the tutor is very different from the usual teacher’s role. Rather than being a content expert who provides the facts, the tutor is a facilitator, responsible for guiding students to identify the key issues in each case and to find ways to learn those areas in appropriate breadth and depth. The tutor facilitates and guides learning without contributing directly to the solution of the problem or being the primary source of information. The tutorial process is facilitated when students feel free to be themselves and express their uncertainties about case-related subjects and the group process. The tutor listens carefully, responds, and uses questions to explore and stimulate depth and breadth of knowledge, develop reasoning ability, enhance communication skills, adopt professional behavior and attitudes, and develop skills for self and peer assessment [9].

Characteristics of a good class tutor: The characteristics of good tutor can be viewed in three domains knowledge, skills and attitude. The terms of knowledge the good tutor should know the goals of the curriculum, the learning objectives of the module that he is tutoring in, the available learning resources, principles of assessment, and group dynamics. His set of skills should include facilitating learning, problem solving, critical thinking, group dynamics or conflict resolution and assessment of the students individually and as group; in order to be successful, the tutor should have correct attitudes. He should be comfortable with the PBL philosophy and adopt a positive attitude toward PBL as a teaching method. He should shift his mind from being sag on center stage to be guide on the side [10].

Each tutor comes to the case with a different mix of clinical and scientific expertise. Tutor guide has dividing into two distinct parts: a section on process, or strategies for approaching the case, and a section on content, both clinical and scientific. The process component helps most if it describes concrete scenarios of past or potential approaches to the initial and subsequent phases of the problem. Tutor guides constitute one of the best forms of continuing education available to faculty. Individual teachers find that writing not only the Guide, but also the entire case typically proves to be an informative and creative professional experience [11].

Trigger material of PBL: The essential features are that the trigger material is relevant, interesting and provokes discussion. A written problem is the most common format for trigger material. It can be just a few sentences or a longer piece of text-for example, a full and detailed case history [12].

An educational problem is a situation in which the student is confronted with a task or difficulty and where he/she does not know an immediate answer and cannot find it by means of an automized range of actions.

Role of problems in PBL

• Problems are pivotal in PBL, it drive students’ learning activities.

• The quality of the problems affects the quality of the small group process, which in turn affects students’ educational achievements [13].

Construct a tutor guide for each case: The development of a PBL tutor case guide is an important part of the case writer’s role. Case writers need to be mindful that the tutors in a PBL programme can have a wide range of science and clinical backgrounds and with varying degrees of experience in PBL facilitation, and so the guide needs to be written addressing these factors. A tutor guide should include the following elements:

• The objectives that should be addressed through student learning on the case. These objectives will also help the tutor frame their facilitation of the case. Case objectives (behavioral/ mental health, biological, as well as professional attitudes, values, and ethics.

• Background information and explanatory notes on the focus of the case in relation to others completed or to be done and within the overall block or module. These background notes should include some explanations of the clinical elements and scientific basis of the problem as this will be important for nonexpert tutors in this discipline [14].

• Associated learning activities which will be covered separately during the case that may support student learning.

• For each trigger of the case, some additional explanatory notes can be provided on key terms which will be important.

• Sample facilitating questions can be provided to help tutors either within the tutorial for each session or trigger. These may or may not be used by the tutor in the tutorial but provide a guide as to the areas that should be addressed. It is important that these questions are not seen as a prescriptive requirement.

• Key readings for the tutor can be provided or a list of readings given so that tutors can seek out both the science and clinical information they may need in preparation for the case. This support material is important as the knowledge and experience of tutors will vary.

• Type of the problem (explanation, discussion, strategy. application......)

• The educational objective of the block (which block objectives are addressed by the problem, how the problem is related to other problems in the block or to the problems of others blocks) [4].

• Concepts used in the problem (difficult words or techniques that students are not expected to be familiar with) and glossary of specific terminology.

• What problem is addressed?

• What aspects to be dealt with in analyzing the problem (questions to stimulate the discussion). A list of possible questions.

• A hypothesis scheme.

• How can the problem been discussed following self-study.

• Related educational activities for the week.

• The limits of the problem.

• Suggestions for tutoring the case and the “What if...” minicases.

• Description of physical exam and laboratory test results/ findings.

• Tutor support information (articles and other information to inform tutors on the content of the case).

• What literature is to be studied for the problem?

• The case itself [15].

The guide aims to help tutor to do in-depth guiding of students in using the seven-step method and the proper use of different techniques for analyzing problems. Participants in the study will be able to attain a very satisfactory level of performance in facilitating students in problem-based learning [16]. The case guide will helps tutors become “case experts” in both the content and process of the case. The guide will provided at the Tutor Meeting preceding use of the case. This guide will help PBL tutor who are not content experts to facilitate the PBL sessions in a competent manner. Many PBL tutors will find themselves learning from these guides [17].

“What if” ...mini cases: Each case should be accompanied by two or three brief “What if...” scenarios that will require students to apply learning across contexts. These alternative clinical vignettes may be designed to emphasize any of the following differences in patient presentation:

a. Epidemiological differences (i.e., changes from the original patient presentation in either innate or acquired risk factors due to different ethnicity, age, sex/gender orientation, lifestyle choices, etc.)

b. Symptomatic differences (i.e., changes from the original patient presentation in signs or symptoms, which may be chosen to lead to “the same diagnosis” demonstrating the variability of individual patient presentations, or may lead to identification of a problem involving a different aspect of the same organ system or even a different related organ system)

c. Differences leading to an “unrelated” organ system (may be either epidemiological and/or symptomatic and chosen to emphasize the “integrative” skills expected of the student, by leading to hypothesis generation involving mechanisms that may be relatively remote from the system under consideration) [15].

Concept mapping: Concept mapping is a tool which can represent knowledge structure by illustrating the relationships between relevant concepts within a given subject domain. During construction of concept maps, students draw concepts related to a certain domain and link these concepts in a hierarchically organized knowledge framework. Concept mapping is based on the constructivism theory of learning, which indicates that learning is an active process where students actively construct their knowledge. By relating and integrating new knowledge with already existing knowledge structure, learners develop deeper and richer understanding and better use of knowledge [18].

Conceptual development and clinical reasoning: Woods et al. [12] suggest that causal connections between basic science concepts and categories of disease contribute to the construction of “illness scripts”. Illness scripts are the individual theoretical frameworks of disease that clinicians construct. They are the causal theories that provide the relationships between biomedical concepts (i.e., features associated with acquisition of the illness), a description of what is not functioning (i.e., the “fault”), the signs and symptoms that arise due to the fault (i.e., the “consequences”), and are built up through extensive contact with multiple patients with similar symptoms [19]. Bransford et al. [20] argue that when students are provided with the conceptual building blocks for such frameworks and when they are encouraged to explicitly construct and articulate connections between these concepts, knowledge transfer is enhanced.

Methods

The study was conducted in the problem-based curriculum of FOM-SCU, in the academic years 2010-2011, 2011-2012 and 2012- 2013. A comprehensive sample was taken for needs assessment of targeted population Phase II (2nd and 3rd year) class tutors, total number of 55 tutors.

A simple random sample from Phase II class tutors for implementation of the newly developed tutor guide, and includes 56 tutors (Intervention 28, Control group 28) Phase II PBL class tutors academic year (2011-2012) and (2012-2013).

A simple random sample from Phase II students instructed by PBL class tutors included in the study; to obtain students’ ratings, and include 140 students (Intervention 70, Control group 70).

Informal discussion was used as a tool for collecting qualitative data about the need for tutor guide. It was conducted for both the Dean of the Faculty of Medicine Suez Canal University and Vice Dean of Education and Students Affairs.

This study passed through three stages:

Stage 1: “Preparatory”

Aiming for excellence in tutoring and considering the importance of tutors for student learning; a guide for tutors was designed for the selected agreed upon problems. This guide was designed after the conduction of needs assessment of phase II PBL class tutors; determining their needs for a tutor guide. First needs assessment questionnaire was designed and distributed to comprehensive sample of Phase II PBL class tutor in the academic year (2010-2011), in order to determine their needs for a tutor guide and identify the problems they may face in moderating the sessions, A section for suggestions and free comments was included in this needs assessment questionnaire. These suggestions were of great benefit in constructing the tutor guide.

The following were the steps of the first phase:

1. Literature review for the component of the ideal tutor guide;

2. Designing the needs assessment questionnaire (self administered questionnaire was designed by the researcher)

3. Validated the questionnaire by medical education experts and the study supervisors

4. Determined needs assessment of targeted population (Phase II class tutors)

5. Determined Phase II blocks that needed an urgent tutor guide.

Stage 2: “Design”

1. Designed a tutor guide template for the selected problems.

2. Designed a tutor guide for selected Phase II blocks; year 2 block Tumors, year 3 blocks CNS (Central Nervous system) the most difficult blocks in discussion and block GIT (Gastro Intestinal Track).

3. Participants for this study were randomly assigned to experimental groups (intervention and control groups). The intervention group received a tutor guide and the control group received no intervention.

Stage 3: “Implementation and Evaluation”

4. Intervention group received tutor guide of the problem, one week before the brain storming sessions.

5. Distribution of “what if” mini cases were done in the debriefing sessions.

6. Tutors’ self assessment was assessed before the intervention for both intervention and control group one month before the intervention and three months after the intervention Dolmans D & Ginns P [21] validated questionnaire were used for tutors self assessment with minor modifications.

7. Student satisfaction about the performance of PBL class tutors was assessed one month before the intervention for both the intervention and control groups and three months after the intervention; this reflected the impact of the tutor guide on the performance of the PBL class tutors.

8. Program evaluation of the designed intervention was toke place according to Kirkpatrick’s [22] evaluation of educational intervention; Kirkpatrick level one (response) was evaluated by assessing tutors satisfaction with the designed tutor guide. Also Kirkpatrick’s level three; this include change in behavior of tutors (to what extent did tutors change their behavior back in the PBL sessions as a result of the intervention) was evaluated from tutor point of view (Tutor Self Assessment) and from students point of view students evaluate the performance of their class tutors.

Results

The results will be divided in the following sections (according to the steps done in this study):

Section I: Needs assessment:

Section II: Tutors’ and students’ satisfaction with tutor guide:

(a) Class Tutors’ satisfaction of the developed tutor guide.

(b) Class Tutors’ satisfaction of “what if” mini cases.

Section III: Tutors performance evaluation:

(a) Tutor self assessment pre-post intervention for both intervention and control groups.

(b) Students’ evaluation of tutor performance pre-post intervention for both intervention and control groups.

Table 1 shows that the items the tutors preferred to be included in the tutor guide, most of them preferred the inclusion of the steps of problem solving, explanation to difficult terms and concepts, educational objectives, diagnoses of the case, differential diagnoses of the case , explanation to investigation and laboratory tests , “What if... mini cases” , management plan and learning schedule of the lectures, seminars and laboratory in the designed tutor guide, while 42.5% only preferred the inclusion of the relation of the problem with other problems in the educational block in the tutor guide.

Items tutors prefer to be included in the tutor guide Preferred Not preferred Total
Frequency Percent Frequency Percent Frequency Percent
• Type of the problem 31 77.5% 9 22.5% 40 100
• Steps of problem solving 32 80% 8 20% 40 100
• Explanation to difficult terms and concepts 28 70% 12 30% 40 100
• Educational objectives 32 80% 8 20% 40 100
• Stimulating questions 34 77.5% 6 22.5% 40 100
• Diagnoses of the case 26 65% 14 35% 40 100
• Differential Diagnoses 35 87.5% 5 12.5% 40 100
• Explanation to investigation and laboratory tests 31 77.5% 9 22.5% 40 100
• What if… mini cases 31 77.5% 9 22.5% 40 100
• Management Plan 25 62.5% 15 37.5% 40 100
• Relation of the problem with other problems in the educational block 17 42.5% 23 57.5% 40 100
• Learning schedule of the lectures, seminars and laboratory 25 62.5% 15 37.5% 40 100

Table 1: Percent of items tutors’ prefer to be included in the tutor guide.

Concerning the need to reform the educational problems, Figure 1 shows that 65% of study population suggested reformulation of the educational problems.

intellectual-property-percentage-distribution-responses

Figure 1: Percentage distribution of the responses of the study group in terms of their opinions about the need for reform of the educational problems.

Concerning blocks that need urgent tutor guide, from the class tutors’ point of view the second year educational blocks from the tutors’ point of view regarding their urgency for tutor guide the highest priority was given to the block titled tumors (54.8%) and the lowest was to block titled Gastro Intestinal Tract 0%. As In terms of the third year blocks, ranking of the third year blocks from the tutors’ point of view regarding the urgency for tutor guide the highest priority was given to the block titled Central Nervous System (71.4%) and the lowest was to a block titled Locomotors System 0%.

Section I: Needs assessment

Needs assessment of target population Phase II (2nd and 3rd year) class tutors, total number of 55 tutors.

Section II: Tutors’ satisfaction with the tutor guide

Class tutors’ satisfaction with the developed tutor guide: As In terms of tutors’ satisfaction with the structure of the guide, more than 57% of the tutors agreed that the content was adequate and that the tutor guide was well organized, and that printing was of good quality, 67% of the tutors agreed that the illustrations helped them better understand the content, and 78.5% of the tutors agreed that the topics were arranged in a meaningful manner.

As In terms of Phase II class tutors satisfaction with the components of the tutor guide, more than 46% of the tutors strongly agreed that difficult terms provided them with helpful information, while 42.8% of the tutors were neutral towards the difficult techniques (not expected that tutor will be familiar with, more than 57.3% of the tutors agreed that intended learning outcomes of the problem, how to check out students’ information, suggestions for tutoring the case, hypotheses scheme of the problem (Differential Diagnoses), and related educational activities of the week (weekly schedule) provided them with helpful information to moderate the session effectively. More than 67.8% of the tutors agreed that concept map of the problem and description of physical exam and laboratory test results/findings provided them with helpful information. 42.8% of them strongly agreed that questions tutor may ask and its answers provided them with helpful questioning techniques.

More than 57% of Phase II class tutors preferred to have a tutor guide in all blocks, and 50 % of them strongly agreed to use the same tutor guide in the subsequent years.

Class tutors’ satisfaction about “what if” mini cases (level 1): What if” mini cases were constructed with each problem from 2 to 3 case associated with every problem, concerning tutor opinions of the developed “what if” mini cases, more than 57% of the tutors agreed that cases were challenging, well written, 50% of them agreed that the cases were stimulating to students to use previous knowledge, 35.7% of them strongly disagreed that the cases were difficult to discuss, 42.9% strongly agreed that the cases were relevant to the problem, and 67.9% agreed that cases brought up new ideas that enriched the discussion. As In terms of students’ opinions it, more than 40% of the students strongly agreed that cases were challenging, 57% of them said the cases were well written, 62.9% of them agreed that the cases were stimulating to use previous knowledge, 34.3% of them strongly agreed that the cases were difficult to discuss, 65.7% strongly agreed that the cases were relevant to the problem, and 62.9% agreed that cases brought up new ideas that enriched the discussion.

As In terms of comparing the opinions of the tutors and students about the developed “what if” mini cases, Table 1 shows no statistically significance differences between tutors’ and students’ opinions.

Section III: Tutors Performance Evaluation:

Class tutors self-assessment: Concerning tutors self assessment of their overall performance during facilitating PBL sessions, Table 2 shows that there is a statistically a significant difference between the control and intervention groups in post self evaluation regarding tutors overall self assessment of their performance as class tutors.

The average score for the overall performance of the tutor p-value
CONTROL GROUP INTERVENTION GROUP (t-test)
(n=28) (n=28)  
Pre Post Pre Post  
Mean ±SD Mean ±SD Mean ±SD Mean ±SD  
6.22 6.54 6.9 7.67 0.02352*
±2.81 ±2.02 ±2.12 ±1.20

Table 2: Comparison between tutors’ self assessment average score for the overall performance of the intervention and control groups of the tutor after the introduction of tutor guide (n =28 per group).

Phase II Students evaluation of tutors’ performance: Evaluation of tutors’ performance during facilitating the PBL sessions was one of the tools to evaluate the outcomes of introducing a tutor guide, students were asked to evaluate their tutors’ performance before and after the implementation of tutor guide (Kirkpatrick level 3).

Results shows that there is a statistically significant difference between control and intervention groups concerning tutors role in clarifying difficult terminologies, facilitating problem identification, problem analysis, helping students generating hypothesis and teaching students how to construct a (concept map) for the problem.

The duration of brain storming sessions increase with implementing the tutor guide mean of the duration of brain storming session was (Mean ± SD 69.40 ± 35.36) minutes; there is a statistically significant difference between control and intervention groups concerning duration of brain storming session. There was a statistically significant difference between control and intervention groups concerning tutors’ questioning techniques; tutors in the intervention group asked more non directive, open-ended questions that stimulated the discussion.

As In terms of students overall satisfaction with their tutors performance during facilitating PBL sessions Table 2 shows that there is statistically significant difference between control and intervention groups in the post intervention mean scores.

Discussion

Because the educational blocks in PBL schools are integrated; block is multidisciplinary, a tutor will be an expert in a particular problems and non-expert or semi-expert in other problems included in the single block. The tutor will only have sufficient experience for some parts of the problems. This means that tutors should be provided with background information about the problems. This should enable the tutor to ask appropriate questions to stimulate the discussion in the group. A problem tutor guide will helps tutors become “case experts” in both the content and process of the case.

The preparation of this study started with needs assessment of targeted population, The Continuing Medical Education (CME) literature suggests that learning is more likely to lead to changes in practice when needs assessments are conducted, when education is linked to practice, and when personal incentives drive learning [23]. Teachers should determine their needs through self-reflection, informed by feedback from students, colleagues, mentors, educational consultants and administrators [24].

Phase II class tutors were selected, because students in this phase are still novice, they arrive to begin a PBL curriculum with little prior knowledge, and into a self-directed learning environment which is unstructured, the novice student with little prior knowledge on which to build a scaffolding for new knowledge needs guidance and relies on the competent tutor, students will turn to their tutors for help and direction. The development of a PBL case guide is an important part of the case writer’s role Case writers need to be mindful that the tutors in a PBL program can have a wide range of science and clinical backgrounds and with varying degrees of experience in PBL facilitation, and so the guide needs to be written addressing these factors [25].

The other part of the questionnaire was concerned with construction of the tutor guide and items tutors’ prefer to be included in the tutor guide and their suggestions for the designed tutor guide. This helped in designing the template for the tutor guide, in the study held by Young L and Papinczak T [26] PBL Tutor Manual was provided to all tutors upon commencement of tutor training, this manual summarized and collated main points like dealing with difficult PBL groups, using wikis, enhancing metacognition, development of clinical reasoning skills, and effective engagement of the “millennium” generation.

David et al. [27] said that that tutor guide should include some guidelines on conducting PBL tutorial groups, the nature of the guidance will depend on the nature and amount of training that is given to all tutors, it should include the intended learning outcomes, suggested learning resources. The notes may give specific comments on item in the problem. They also emphasized that if tutor is a member of module planning, has helped to prepare the problems, is familiar with the learning objectives for each case, and is an expert in the topic, then tutor guide may not be crucial. However on most occasions the tutor will not be in this position, in which case these type of note to tutor are essential. They emphasized that tutor guide should not be given to students, the use of tutor guide by students would undermine the whole process of PBL, for this reason we stressed on the tutors that tutor guide is for tutor use only (It is Not Intended to be copied for Student Use in Tutorials).

54.80% of needs assessment study population select the block tumors in year 2, and 71.4% of them select block Central Nervous System (CNS) in year 3 as the most difficult blocks needed urgent tutor guide. This prioritization helped in focusing on solving tutor problems in moderating and facilitating PBL sessions in these blocks, this helped in making tutors more motivated to use the designed tutor guide. Another block was also selected; block “Gastro Intestinal Tract” (GIT). This arrangement of the blocks matched with my experience as a Phase II class tutor for more than 8 years. The selected blocks were the hardest to be facilitated in my previous experience as class tutor. This result is similar with the study held by Baroffio et al [24]; there were considerable differences in tutors’ perception of their level of preparedness across the teaching units. While in some teaching units none of the tutors reported any difficulties, in some others up to 27% of the tutors felt insufficiently prepared in guiding students’ learning of the problem content, and up to 42% had difficulties in handling student and group functioning. Most importantly, in some teaching units these difficulties were raised by more than 90% of the tutors in their written comments. Many tutors also indicated a need to further define, with the tutors in their teaching unit, the level of depth to which the problem-learning objectives should be achieved.

Multiple measures were applied in this study to evaluate the effectiveness of the tutor guide. Evaluation was also principally guided by four level models of Kirkpatrick’s [22].

Both tutors and students agreed that tutors’ questioning techniques were enhanced, 42.8% of the tutors strongly agreed that questions tutor may ask and its answers helped them during tutoring the cases, as selected samples of the questions were included in the tutor guide.

Results of students’ rating of tutors’ performance in the area of questioning techniques, showed statistically significance difference between control and intervention groups concerning tutors’ questioning techniques; tutors in the intervention group asked more non directive questions; control group post students rating mean was (3.8 ± 1.09) compared with intervention group post rating mean was (4.97 ± 0.17) with (p-value 0.000), also open-ended questions that stimulated the discussion p-value was (0.000).

The majority of the tutors agreed that with tutor guide they became more motivated to moderate PBL sessions; they emphasized the importance of the tutor guide to help them to facilitate students’ creating learning objectives, and distinguish between main points and side-issues during discussing the problem, knowing in what direction the discussion should be heading, and that tutor guide helped them to become “case experts” in both the content and process of the case. According to Margetson [28], the tutors facilitate PBL sessions by “questioning, probing, encouraging critical reflection, suggesting and challenging in helpful ways, but only where necessary”. Most new tutors in problem based learning are challenged by the “where necessary” (deciding when and how) part of intervention, we think that tutor training and tutor guide helped them in this part.

Both the tutors and students assessed the educational effectiveness of “what if” mini cases, they agreed that the cases were challenging, well written, stimulating to them to use previous knowledge, relevant to the problem, and brought up new ideas that enriched the discussion, and it were not difficult to discuses. There was no statistically significance difference between tutors’ and students’ opinions regarding ‘what if” mini cases. In a study held by Tosteson et al. [29] Mini cases were designed in a course entitled ‘Human Nervous System and Behaviour’. This 10-week block course, conceived as part of Harvard Medical School’s ‘New Pathway’ curricular reform, designed mini cases were used as evaluation instruments for the mid-term and final examination, respectively, in our study “what if” mini cases has not been involved in any formative or summative assessment, but we think it can effectively be used. Eva [30] emphasized the importance of learning with cases; it allows learners at any level to build up a mental database of cases to enhance expertise in clinical reasoning.

As In terms of level 3 in Kirkpatrick’s [22] model, Dolmanss et al. [21] standardized, validated and reliable tutor self assessment questionnaire was distributed to the Phase II class tutors in both control and intervention groups, for obtaining rating of their self assessment.

Results of the questionnaire showed improvement of intervention group tutors’ performance in the area of constructive active learning, self- directed learning, collaborative learning, intra-personal behavior as tutor and problem content knowledge. Improvement was in the item related to tutors helping students more to search for explanations during discussion, summarize what they had learnt in their own words, participate actively in the PBL sessions, generate clear learning objectives by themselves, search for various resources by themselves, and create a concept map for the problem. There were a statistically significance differences between both groups in post self assessment evaluation (p≤0.05 in all items).

The average score for the overall performance of the tutor was (7.67 ± 1.20) in the intervention group compared with control group (6.54 ± 2.02), this was parallel with the results of the study held by Dolmanss et al [21].

As In terms of level 3 in Kirkpatrick’s [22] model, behavior outcomes address either the extent to which knowledge and skills gained in training are applied on the job or result in exceptional job-related performance. Evaluation of tutors’ performance during facilitating the PBL sessions was one of the tools to evaluate the outcomes of implementing a tutor guide, students were asked to evaluate their tutors’ performance before and after the implementation of tutor guide, for both intervention and control group pre-post intervention [30].

Analysis of students’ results revealed improvement of tutor performance in tutoring blocks CNS, GIT and Tumors, as David et al [27], observed that tutors’ performance varies across teaching units and by his study, showing that tutors’ needs are different for each teaching unit.

Results of tutor self assessment and students rating of tutor performance shows that both results are nearly equal for the intervention group, tutors performance mean scores from students point of views was (78.14 ± 12.64), tutor overall self assessment mean score (75.78 ±12.30). We found that tutor self assessment and students rating are matching. In a study held by Irby et al [31], they found that self-reposts and observable behavior matched.

Studies investigating the related factors to tutor performance have found that a tutor’s performance is also dependent on the quality of the cases, structure of PBL courses, and link with students’ level of prior knowledge, and the functioning of tutorial groups [32].

Finally, students’ ratings revealed that tutors improved their ability in facilitating student participation, and tutors assessed themselves as being better facilitators of small-group functioning after the workshop, and after having a tutor guide.

Conclusion

The study concluded that tutor guide were effective in improving tutor facilitation skills, and increase educational effectiveness of the PBL sessions.

Limitations

i. Although we designed a tutor guide for problems, we did not reformulate the problems; 65% of the needs assessment sample population said that they need reformulation of the educational problems, but we designed from 2-3 what if mini cases for each problem. Medical educationist task force team was assigned to reformulate FOM-SCU educational problems starting from year one. We already finished reformulation of Phase I problems, and the work is ongoing.

ii. We designed a tutor guide for only 3 blocks out of 12 in Phase II, not for all the Phases and all the blocks.

iii. Lack of using an trained standardized internal audit from quality assurance unit to measure and assess the real performance of the tutors on sessions (on-life performance) of the tutors with tutor guide (Kirkpatrick’s level 3), and the effectiveness of the PBL sessions with using a tutor guide.

iv. Furthermore, level 4 in Kirkpatrick’s model results was not assessed in the present study. The impact of the program on the students’ progress their clinical reasoning, critical thinking and problem-solving skills, and achievement scores, as well as the impact of implementing the tutor guide on the institution itself in abrader sense, are the worth to be evaluated as well, other outcome indicators should be used as a basis for assessment.

v. We want to design another version of the tutor guide more concise and short for busy tutors.

vi. We did not provide feedback to individual tutors following students’ evaluation; tutor did not received constructive feedback about their performance in PBL sessions either oral or written feedback.

vii. Lack of comparing study results with the results of a same study used the same methodology, due to paucity of research on the effect of having a tutor guide on tutor performance (Table 3).

Students’ overall evaluation of  tutors’ performance CONTROL GROUP INTERVENTION GROUP p-value
(n=70) (n=70) (t-test)
Pre Post Pre Post  
Mean ±SD Mean ±SD Mean ±SD Mean ±SD  
2.49 2.86 2.47 3.38 0.02734*
±0.7 ± 1.15 ±0.61 ±1.12

Table 3: Comparison between students’ in the intervention and control groups opinions about overall evaluation of tutors’ performance.

References

Citation: El-Aziz El Naggar MAA, Maklady FAH, Hamam AM, Omar AS (2013) Designing, Implementing, and Evaluating a Tutor Guide for Problem Based Learning Phase II Class Tutors at the Faculty of Medicine, Suez Canal University. Intel Prop Rights 2: 106.

Copyright: ©2013 El-Aziz El Naggar MAA, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.