English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Similars in
SciELO 
Permalink
INTRODUCTION
Medical doctors serving either primary care, rural or underserved areas do not only need to be clinically competent, but they also need to have health management competencies from early on in their medical careers 1. Studies suggest that medical students are underprepared in their health services management competencies, constituting a public health challenge 2,3,4. A study conducted in the USA in 2009 5) of over 58 000 medical students shown that medical students have inadequate instructional time in courses related to health systems: health economics, health policy, health management, and medical practice management. This is consistent with the report by the Can-MEDS Roles (Communicator, Collaborator, Leader, Health Promoter, Scholar and Practitioner) from the Royal College of Physicians and Surgeons in Canada, which identified that medical students possess only a rudimentary level of management related competencies 6.
The consequences of inadequate training in management are of the same magnitude as those of inadequate clinical training. Many junior doctors will oversee the first level of care in rural and remote areas and will need essential managerial competencies 7. Decision-making is considered essential for medical leadership by the National Health Service (NHS) in England. Managerial decision-making is defined as choosing between alternatives to maximize the benefits for the organization 8.
Among the causes of inadequate training and interest in learning health services management-related courses amongst medical students are the use of traditional teaching methods, theoretical and non-participatory settings, as well as limited-time allocation 5. Such techniques emphasize management-related courses' importance and discourage them from even considering management training in their future professional careers. More intensive curricula in medical economics, health systems, health management, and medical practice management would reduce the perception of insufficient training 9.
In this context, virtual simulated training would constitute an innovative alternative in the learning of medical students in courses related to health services 10. Indeed, simulation is a student-centered learning and assessment technique that uses simulated tools and methods in controlled environments, in accordance with the training objectives in the health sciences, allowing them to identify, pose, clarify, and solve health problems of varying complexity 11. It recreates a situation that the student could face in their clinical or community practice, allowing them to experience the process as if it were a real event, to practice, learn and evaluate, to subsequently address the situation in a better and safer way 12. Simulation is materialized through a role-play scenario. A set of activities around a simulation space, with defined objectives and competences to be achieved through the implementation of the methodology 13. It includes pre-briefing, familiarization with the simulator, theoretical information, scenario briefing, simulated scenario, debriefing and closing 14-16. The scenario is the basic unit of simulation activities.
This study aimed to determine the knowledge improvement in managerial decision-making amongst medical students at the Universidad Nacional Mayor de San Marcos (UNMSM) in Lima, Peru. The results of this research will not only strengthen the competencies in health services management of medical students but will also enhance the pedagogical skills of their teachers and will integrate a new methodological tool for teaching courses related to health services management.
METHODS
Study design
The study was a before-after crosssectional study design with an intervention that included virtual simulations of tele-management. Medical students were assessed on the level of knowledge in managerial decision-making using pre-and post-test assessments. Students' performance in applying the steps of the rational managerial decision-making process and their perception of satisfaction with the virtual simulation methodology were also assessed.
Setting, population and sampling
The population consisted of 84 sixth year medical students, just before the beginning of their final year of medical internship and completion of their training, in November and December 2020 at UNMSM in Lima, Peru. As part of their training, medical students have a family medicine course (externado) which is oriented towards learning comprehensive care for individuals, families and communities in clinical medicine, surgery, obstetrics and gynecology, pediatrics, emergencies, and public health. To be eligible, a participant had to be a final year medical student, enrolled in the above course, and had to provide informed written consent.
Data collection
The 79 participants were organized into 16 groups with an average of five students per group. The rotation of four practice groups per week was scheduled with two dates per week on Mondays from 15:00 to 18:00 and Thursdays from 17:00 to 20:00. The educational intervention took place between the 23rd November and 17th December 2020.
A virtual COVID-19 pandemic outreach course was offered to potential participants prior to the study. One of its themes was the virtual simulation of remote management of management decision-making in primary care related to the incorrect completion of the referral form of an infant with a diagnosis of conductive hearing loss due to impacted cerumen. The students took on the role of heads of primary care establishments who had to make a managerial decision in a simulated case of an error by a rural service doctor, a health professional prepared for this role, in the registration of required forms in the primary care information system. The study was carried out in accordance with the Directive for the development of academic activities in the Simulation and Learning Area of the Faculty of Medicine 13.
The variables studied were the virtual simulation of tele-management related to the poor completion of the referral form in primary care, the level of knowledge, performance in managerial decision-making, and the perception of satisfaction in relation to the virtual simulation methodology. The instruments used for the evaluation of the virtual simulation of a tele-management related to the poor completion of the referral form in primary care was a virtual non-technical simulation scenario validated by the team of teachers and the specialists of the Educational Innovation Unit. These included the following simulation tools:
Template for designing virtual simulation scenarios for managerial decision-making at the first level of care.
Script for the role of the standardized managerial decision-making at the first level of care.
Student's guide to the activities in the virtual simulation scenario.
Student's guide to the activities in the virtual simulation scenario for managerial decision-making at the first level of care.
Template for the role of the standardized manager for managerial decision-making at the first level of care.
The level of student knowledge of the rational managerial decision-making process was measured using an online questionnaire which specifically assessed knowledge of the rational managerial decision-making process and scores on a scale of 0-20 points. Similarly, the performance measurement of the application of the rational summarized decision making process during the simulation used a performance evaluation with a continuous scale of 0 - 20. The perception of student satisfaction in relation to the simulation methodology was measured through the application of a 5-point Likert scale (1= total dissatisfaction; 5= total satisfaction), regarding to content, methodology, teaching staff and overall assessment of the simulation.
Simulation process
The educational intervention of virtual simulation in tele-management had four phases with individual and group educational interventions (Figure 1):
Phase 1 - Pre-briefing (30 minutes): in this phase, participants were introduced to the case in eight sequential steps as detailed below.
Welcome: Simulation and e-learning area welcomes the medical student as a competent professional, with a person-centered approach and a culture of continuous improvement.
Confidentiality agreements and commitments.
Learning outcome: the student learns the learning outcome related to the application of the protocol for managerial decision-making in the case of incorrect completion of the referral form in primary care, using tele-management.
Motivation: application of the safe environment.
Familiarization with the simulator: incident or event for the generation of learning is the managerial decision making of a head of a health center in a role simulated by the student, in the face of the wrong registration of the referral form of a child with conductive hypoacusis due to impacted cerumen diagnosis in primary care, which generates the non-reimbursement of payment by health insurance. The simulation ends when the student takes the managerial decision in the face of the poor registration of the referral form.
New management concepts were developed in the framework of the health system reform and health emergence due to COVID-19. This includes an explanation of the Integrated Health Network (RIS); audit of the correct registration of the referral form of an infant with a diagnosis of conductive hearing loss due to impacted cerumen; the relationship between incorrectly completed referral forms and lack of reimbursement by the comprehensive health insurance; and registration of the Tele-management Application Form with agreements and commitments and management decision making in case of incorrect completion of the referral form.
Scenario report: information on the familiarization with the simulator is expanded with comments on the reality of the scenario; the presentation of the problematic situation of decision making in the face of the poor recording of the reference sheet; the roles of the student (head of the health institution), teacher and standardized manager (rural doctor) are explained; doubts are resolved; it is emphasized that the activity in the scenario culminates in decision-making.
Technological guidelines: explanation of human resources (role of the tele-manager), technological resources (electronic devices) and infrastructure (physical environment with internet connection) for the implementation of tele-management are regulated by National Directives.
Figure 1 Moments and organization of the pedagogical intervention based on the simulation of virtual tele management simulation.
Phase 2 - Simulation: The students migrate to a video conference room, where they encounter the simulation of decision making between the head of the health institution in the face of a wrong record of the referral form. They had to evaluate whether the six phases of the management decision-making methodology have been fulfilled: a) Identification and choice of the problem b) analysis of the problem, c) generation of possible solutions, d) choice and planning of the solution, e) implementation of the solution, f) evaluation of the solution 17.
The standardized manager was trained and used a validated script for their standardized role in managerial decision-making at the first level of care which included the following: a) summary of the case, b) preconsiderations such as the interaction of two characters with either the video camera on or off with microphones muted in the latter instance; c) characteristics with the specification of the emotion to be simulated (anxiety, worry, anger among others), d) text for the standardized role, e) gestures and feelings and f) external aids to reorient the student's learning achievements; g) the primary care referral forms completed correctly and incorrectly, the application forms for tele-management filled correctly and not completed.
On the other hand, the students previously received a synchronous virtual training and accessed the contents of the virtual classroom. It is made up of the following components: 1) learning outcome; 2) previous activities such as the review of the instruments that were validated by lecturers and experts 3) Final activities: completion of the post-test and the satisfaction survey of the simulation practice; as well as the fulfilment of the commitment undertaken for the application of the management decisionmaking protocol by means of remote management. Finally, 5) access to bibliographic material for further learning.
Phase 3 - Debriefing Plus/Delta: Key simulation element. Its preparation should be designed by the instructors. This phase involved participatory and group learning. Its phases included the following: a) reactions-emotion closure: comprised the participant's self-assessment. b) Description: included the playback of the simulation video; the orientation of the analysis and understanding of the description of the experience lived, c) analysis and comprehension: the behaviors of the students were identified; inviting them to express their points of view regarding the procedure or process observed. Finally, in the fourth phase, a posttest of the student's level of knowledge of the rational process of managerial decision-making was carried out using Google Forms, lasting 15 minutes.
Data analysis
Analyses were performed with the use of STATA version 17.0 (STATA Corp, College Station, Texas, USA). Numerical data were explored for normality using the Shapirowilk test. Since numerical data were not normally distributed, they are summarized using the median and interquartile range (IQR). The Wilcoxon signed rank test was used to demonstrate the difference between the medians of pre- and posttest scores. This test was used to test the hypothesis of equality of median scores before and after the test. Categorical variables are summarized using frequencies, proportions, and graphs. The level of significance was p-value < 0.05.
RESULTS
84 study participants were recruited. However, five were excluded, one of them did not consent and four did not undertake the pretest. As a result, only 79 students were considered for the study. The demographic characteristics and some performance indicators of participants are summarized in Table 1. Of the 79 participants, 48 (60.8%) were female, the median age was 25 years (IQR = 23 - 26), 64.6% (51/79) participated in the face-to-face workshop in 2019, 54.4% (43/79) participated in the 2020 tele-training and 76.0% (60/79) had reviewed the virtual classes previously. The medians of the pre-test and post-test score were both 20 with similar IQRs (18 - 20). There was a median satisfaction level of 3.8 (IQR = 3.5 - 4.2) and the performance median score was 17 (IQR=16.5 - 18).
Assessment of the level of knowledge in managerial decisionmaking.
Table 2 shows that the median posttest ranks of participants were statistically higher than the median pre-tests rank scores, and this was statistically significant (p-value = 0.004); this tendency was maintained when they were evaluated according to sex with statistically higher post-test scores for males (p-value = 0.05) and females (p-value = 0.03) respectively. Other statistically significant post-test scores included participation in face-to-face workshops in the previous year (p-value = 0.01) and the prior review of virtual classroom material (p-value = 0.02). However, there was no statistically significant difference between the median pre-test and post-test scores of those who participated in the tele-training workshop in 2020 (p-value = 0.09).
DISCUSSION
This study gathers lecturers' and students' efforts in developing a virtual decision-making simulation for the first time as part of the medical curricula. It represents a challenge compared to traditional teaching methods, a change of attitude of teachers and students, and application of this methodology 18.
Compared to traditional teaching methods, virtual teleconferencing promotes an intrinsic change in the management of classes, so much from the students as from the teachers 19,20. Decision-making is one of the critical aspects of these students' professional training 21. The new methodology must be appropriately implemented, and familiarization with the platform should be the norm before its use. Therefore, an acclimatization period is needed, with all pertinent information before and during the training, providing the student with the necessary tools to achieve appropriate learning 22.It is possible that the differences in the pre and post-test scores when they previously reviewed the virtual material was a consequence of the proximity between the simulated remote management session and that the students undertook the review immediately before performing the pre-test. In addition, the medians in both cases improved after the simulated activity, confirming its better impact on the development of capacities.
As a first experience from the teaching and learning sides, there were some limitations. First, the sample size does not allow a better evaluation according to different categories. In addition, it does not allow an adequate assessment of the variables distribution. However, the analysis of the entire sample and with the use of non-parametric tests makes it possible to find the significant differences that explain the contribution of the simulated remote management. This allows these findings to be used as a baseline for future studies. Second, failure to consider a comparison between the baseline method of teaching, and the virtual teaching changes brought on by the COVID-19 pandemic. Third, it was not easy to arrange meetings with all the staff and students due to restrictions and spread of COVID-19 cases. However, this did not have a negative impact as other alternatives were devised to communicate the message. Furthermore, virtual simulation has a lengthy background in clinical sciences teaching in Peru, opposite to health management education which does not have previous experiences. Nevertheless, this was countered with the development of a customized template.
The management courses in the medical curriculum are not enough in daily teaching, not only in terms of hours dedicated but also in how it is delivered. Therefore, freshly graduated students do not have the competencies to handle health management in their work. In this regard, a Simulation is a methodological tool that undergraduate medical lecturers should include in the courses they develop. That is primarily because the participation of teachers and students is dynamic. It is a way of learning that is difficult to forget because of what one sees, hears, and feels. It is therefore difficult to erase from one's memory, and the experiences gathered will help students when making day-to-day decisions as future medical doctors. Teachers need to be trained in this tool, as it is a different way of teaching from what is traditionally done. It does not require a large budget, but it does require a change of attitude on both teachers and students.
In conclusion, simulation enhanced the student learning process by a "real experience" in the managerial decision-making process. In addition, the students welcomed this methodology because it was dynamic, and its approach was real so that the students recognized the impact of managerial skills in their work. The impact of using simulation methodology in teaching management was related to the innovative use for its virtual use and its application to teaching management and focused on primary care services. Moreover, the implementation of the virtual simulation was at a low cost and highly replicable in different scenarios such as pandemic and other courses of health management not only in undergraduate but also graduate programs and trainee of health workers.
