Evaluation Plan
The program chosen for evaluation is near-peer teaching of nursing students concerning their understanding of Quality Improvement (QI) efforts and their preparation to implement them in their future practice (McGeorge et al., 2020). The students are expected to gain immense value from the experience because they are to be taught not by senior tutors or instructors but rather by individuals with similar levels of experience and preparation. To measure the outcomes of the program and its influence on the target group, Anderson’s Value of Learning model will be used. Its objective is to focus evaluation on the interplay and the alignment of the goals of a learning program and strategic goals of an organization within which the program is carried out. Only once the goals align with each other is it possible to evaluate the success of a learning program in meeting its goals.
The Anderson model of learning evaluation represents a unique three-stage learning evaluation cycle designed to be applied at a wide scale. While other methods of training evaluation focus on specific interventions, the model rather considers the alignment of training goals with the strategic goals of an organization. As a relatively new framework to the training evaluation industry, the model can be distinguished for focusing on two areas with which organizations often struggle, such as conducting evaluations and gathering evidence (Alsalamah & Callinan, 2021). Therefore, in the context of the training program in question, the model is intended to align the dedication of an educational facility to teaching students about QI implementation with the improvement of QI knowledge and skills through the near-peer program.
There are three stages of the Anderson model of learning evaluation that help determine whether the implemented program has been beneficial for an organization. The first step is determining the current alignment of training against the existing strategic priorities of an institution. The second step is using assessment methods to evaluate the contribution of the learning process. The third step is establishing the most relevant approaches for an organization based on the outcomes of the evaluation (Deller, 2020). An appropriate approach for any organization depends on stakeholders’ goals and values. Anderson’s model suggests four measurement categories, such as the emphasis on short-term outcomes, the emphasis on long-term outcomes, the trust of the senior management in learning contribution, as well as the requirement of an organization for learning metrics.
Based on the framework of Anderson’s Value of Learning model, several evaluation questions can be developed to help identify the outcomes of the learning program that entailed near-peer tutoring for QI training at a medical education facility. The questions are the following:
- What is the influence of the near-peer program on the preparation of medical students for quality improvement efforts in their future careers?
- What is the influence of the near-peer program on the effectiveness of junior tutors when educating medical students on quality improvement?
- What is the impact of the near-peer program on the effectiveness of the medical school in preparing its students for future quality improvement procedures?
- What is the impact of a near-peer program on improving the approach of the medical school toward preparing students for their future work in quality improvement?
The sampling technique to be used for the purpose of the evaluation is simple random sampling, which entails that every member of a subset population has an equal probability of being chosen. Because the sample of the students and junior tutors participating in the near-peer program is not large, it is essential to ensure the highest level of representativeness. All students’ names will be coded, and the codes will be entered into a randomization software that will help select the sample. Since there are only eight junior tutors, all of them should be surveyed; if some of them refuse to participate, this will be noted in the report of the evaluation results.
The data for evaluating the outcomes of the near-peer tutoring program will be collected with the help of anonymous questionnaires that are quick and easy to implement. It is necessary to seek formal and structured feedback from both students and junior teachers, with questionnaires being conducted before and after the training (McGeorge et al., 2020). The questionnaires can be distributed electronically and include short summaries as to how students’ and tutors’ data will be used for evaluation (Minto et al., 2017). The data necessary to collect is concerned with the reflections on the usefulness and the content of workshops. Moreover, questions that concern students’ confidence will incorporate categorical Likert scales. Besides, there will be an area in questionnaire forms where students can write their suggestions for improvement or mode comprehensive accounts of their experience. Tutors will be given short online surveys several weeks after the completion of workshops. The answers to yes/no survey questions will be filtered by cross-tabulation, while the Likert scales will be analyzed for significance using the Mann-Whitney U test. The null hypothesis of the test is that there is no increase in students’ confidence implementing QI following the near-peer teaching session.
Throughout the process of evaluation, it is imperative to consider several ethical issues that may impact participants. Important ethical considerations include informed consent, voluntary participation, confidentiality and anonymity, and the assessment of only relevant components. Informed consent is important to take into account in the evaluation process due to the need to ensure that the individuals participating are fully informed of the process that is taking place (Manti & Licari, 2018). Both students and junior tutors will be made aware of the purpose of the project, what steps are involved, and whether there could be any adverse impacts on their participation. This is necessary because of the importance for the participants to make informed decisions regarding their involvement in the evaluation.
Once informed consent is obtained, voluntary participation is considered to make sure that individuals are free to participate in project evaluation free of pressure or coercion. Participants should have the freedom to withdraw their involvement in the evaluation at any point without negative repercussions for their future services or relationships with the researchers involved. Confidentiality entails that any identifying information that concerns participants is not available in free access to anyone except program coordinators or researchers (Manti & Licari, 2018). It also ensures that such information is excluded from any reports or republished documents that discuss the findings of the evaluation. Finally, the access to only relevant components of a program entails the use of information that only bears relevance to the evaluation. Because of this, the evaluation should be as simple as possible to facilitate the emphasis on its goals and intentions and the ways in which the collected and analyzed data will be used.
Recommendations
When considering the findings of the evaluation, it should be noted that the surveys offered quantitative data while the quality of free-text responses involving recommendations and accounts of tutoring experiences was insufficient for performing thematic analysis. Therefore, it is possible to present data from individual feedback forms for providing s snapshot of student opinion on the workshop. Based on the student feedback, there are several highly important positive factors that came out from the near-peer teaching program. Specifically, students showed a good attitude toward the interactive teaching style, the ability for them to carry out their own projects without strict supervision, as well as the combination of group-based and didactic teaching activities.
Overall, the attitude of students toward being taught by junior doctors was positive. It was indicated that the teaching approaches that senior quality improvement experts took were often too traditional and daunting (McGeorge et al., 2020). Because junior doctors are closer to medical students in terms of experience and knowledge, there was an overall greater appreciation of quality improvement strategies among students. Near-peer tutors are usually more relatable in their teaching and approachable. Because of this, students view them as more reliable and valuable sources of information on QI. The fresh accounts of the everyday clinical practice that junior doctors experience offer the first-hand experience that is invaluable to medical students.
When it comes to the perceptions of junior doctors, they are also likely to show a positive attitude toward the experience. Due to the need for them to develop leadership and teaching skills, the near-peer tutoring may facilitate improved knowledge of effective educational practices. The focus on QI as a component of clinical practice is helpful for the tutors because it enables them to reflect further on how they can approach the issue themselves. Following tutoring sessions, they are more likely to engage in QI initiatives themselves or develop them in the future. Because of this, it could be a positive practice for junior doctors to engage in QI teaching (Doran et al., 2018). Besides, the closeness between junior doctors and medical students creates an environment of collaboration within which everyone involved is encouraged to offer their suggestions as to how the QI process should be incorporated into learning.
Because the project can positively contribute to the understanding of QI and help embed the topic into the medical school curriculum, it can be implemented at different institutions to boost the effectiveness of preparing students for QI. The interactivity and the direct application of QI principles during the near-teaching program are the principles that allow for the effective integration of a topic into practice. Besides, the preference for near-peer teaching, as opposed to traditional teaching approaches, shows that more educational institutions should introduce the methodology when it comes to the learning of new subjects and topics that are often overlooked.
For junior doctors, the experience tutoring can be highly valuable because of the opportunity to improve their leadership alongside clinical competencies. In particular, the experience with near-peer tutoring can allow foundation doctors to develop valuable skills for delivering teaching and mentoring to doctors and nurses-to-be. As long as there is an ongoing supply of motivated junior doctors working at teaching hospitals and medical schools across the US and the UK, the approach can be made sustainable and get embedded into the teaching practice. Moreover, senior staff involvement is necessary for establishing high expectations for educational standards and developing materials for informing tutors and helping educate medical students. Thus, the near-peer tutoring model can offer abundant teaching opportunities for junior doctors while also easing the pressures on faculty members who are already overburdened.
To implement the recommendation of introducing near-peer tutoring for QI at medical schools and teaching hospitals, it is necessary first to assess the state of QI preparation at institutions to develop a plan of action that is suitable for each setting. Then, it is recommended to evaluate the availability of junior doctors that are willing to participate in the program. Thus, it is necessary to facilitate the alignment between the goals of each organization and the resources available to them to ensure the successful implementation of the program (Romero, 2018). It is expected to have a variance in organizational goals and the levels of preparation for near-peer teaching, which is why it is imperative to implement assessments first.
Due to the significant gap in the attention given to QI efforts across facilities, it is expected that there may be a variance in the preparation of institutions for teaching QI efforts in medical practice. The pilot project implemented at Imperial College Healthcare NHS Trust entailed a single-center setting, and it may not work for other locations. Nevertheless, considering the low costs and the high impact of the near-peer model, there is an excellent potential of implementing it internationally. Future work should be concerned with directly comparing the effects of undergraduate QI teaching by junior doctors and the senior teaching faculty. The extended assessment of near-peer programs for teaching can also be helpful for determining whether self-reported improvements in QI knowledge and implementation can be sustained over time. Specifically, it may be useful to introduce objective markers of educational facilities’ engagement with QI. For instance, one can measure the completion of postgraduate QI projects in clinical settings and the available incentives at facilities to add weight to the near-peer approach.
Students represent a group of individuals that have the time, space, and motivation to participate in quality improvement projects within their studies and future practice. Due to this, medical students are obligated to provide their graduates with the necessary knowledge and competencies in quality improvement. They can encourage student engagement in QI through involving projects in future clinical placements and facilitating incentives for participating. Thus, near-peer teaching can be placed as a mid-way point in the curricula of medical schools.
The findings and recommendations should be openly and transparently shared with the relevant stakeholders that have the capacity of introducing the near-peer program at institutions that they lead. While engaging students in QI efforts may be challenging due to the lack of resources and competing educational priorities, it is imperative that curricula developers and senior staff have access to the information illustrating the advantages of the near-peer program, as illustrated by the example of the pilot study at Imperial College Healthcare NHS Trust. With the benefits available for students, junior doctors, and senior staff, the near-peer teaching program can be used as a model for further undergraduate training at medical education institutions.
References
Alsalamah, A., & Callinan, C. (2021). Adaptation of Kirkpatrick’s four-level model of training criteria to evaluate training programmes for head teachers. Education Sciences, 11, 116. Web.
Deller, J. (2020). Anderson model of learning evaluation: The comprehensive guide. Web.
Doran, N. J., Bethune, R., Watson, J., Finucane, K., & Carson-Stevens, A. (2018). Empowering junior doctors: a qualitative study of a QI programme in South West England. Postgraduate Medical Journal, 94(1116), 571-577. Web.
Manti, S., & Licari, A. (2018). How to obtain informed consent for research. Breathe (Sheffield, England), 14(2), 145-152. Web.
McGeorge, E., Coughlan, C., Fawcett, M., & Klaber, R. (2020). Quality improvement education for medical students: A near-peer pilot study. BMC Medical Education, 128. Web.
Minto, C., Vriz, G., Martinato, M., & Gregori, D. (2017). Electronic questionnaires design and implementation. The Open Nursing Journal, 11(Suppl. 1-3), 157-202. Web.
Romero, E. F. (2018). Thus, it is necessary to facilitate the alignment between the goals of each organization and the resources available to them to ensure the successful implementation of the program. Revista EAN, 2018, 15-31. Web.