Type of study

A retrospective cohort study was carried out. The target population was all patients under 18 who underwent heart surgery with ECC admitted between 2000 and 2010 to the Post-Operative Cardiovascular Unit (UPOCV) of the Instituto Nacional de Salud del Niño (INSN) in Lima, Peru. Patients who did not use ECC were excluded.

The UPOCV team of intensivists and an INSN pediatric infectious disease specialist made the diagnosis of infection. The diagnostic criteria and definitions established by STS for sepsis, mediastinitis and endocarditis were used ²¹.

Procedure

Independent variables

Risk factors for postoperative pediatric cardiac infections were formally sought in the literature ^{1 , 2 , 7 , 9 - 13 , 17 - 22}. The search was conducted on PubMed and Scopus through October 2018. Preoperative variables studied included age, weight, sex, provenance, prematurity, acute malnutrition, length of preoperative stay, previous cardiac surgery, type of heart disease, Risk Adjustment for Congenital Heart Surgery score (RACHS-1), previous mechanical ventilation, pneumonia, pulmonary tuberculosis, tracheostomy, congenital malformations, genetic disorders, cardiac and non-cardiac malformations, hepatitis, sepsis and endocarditis, renal failure, dialysis, cardiogenic shock, arrhythmias, pulmonary hypertension, cyanotic heart disease, Ross-modified functional class, hypothyroidism, history of epilepsy and neurological damage ^{7 , 11 - 13 , 17 , 21}. The intraoperative variables studied were non-elective surgery (versus elective surgery), intraoperative diagnosis, type of surgery, time of ECC, circulatory arrest, ultrafiltration during ECC, use of corticosteroid, deep hypothermia, and open chest.

Data source

The sources of information were the patient’s medical records and medical history. The data contained the same information indicated in the STS file for congenital heart surgery database. Data collection was performed by one of the researchers (EWS). The information collected was exported to the SPSS version 22 program (IBM Corp., Armonk, NY, USA) for statistical analysis.

Development of the model

We used The Chi-square test and Fisher’s exact test to determine differences in population distribution between patients with and without MPIs. Subsequently, we carried out a logistic and binary regression model and selected those with a p < 0.200 ²³. The association of each variable with the primary outcome was reported using odds ratio (OR) and a 95% confidence interval (95% CI).

Internal validation of the prediction model

The model was validated internally using the bootstrapping technique with 200 repetitions. Each repetition created models in samples identical in size to the study sample size, using random sampling with replacement of individuals. The internal validation provided a correction factor to adjust the model coefficients, assuming that the initial model coefficients were inflated for predictions in the new groups.

Evaluation of model performance

The predictive model was evaluated for discriminatory performance and clinical utility. Discrimination refers to how useful the model is in differentiating between patients with presence and absence of MPI. Discrimination is quantified with the c-statistic, which is equivalent to the area under the ROC curve. The c-statistic has values ranging from 0.5 to 1, and the closer it is to 1, the better the model. Clinical utility was evaluated with benefit analysis ²⁴; this method evaluates the benefit of correctly identifying a patient with a high risk of MPIs (true positives) versus mislabeling them as probable MPIs (false positives) in a range of reasonable MPI risk thresholds where the physician feels comfortable deciding (for example, to administer antibiotics for an MPI). A threshold of 20% was considered reasonable, with a range of 5% to 40%.

Finally, the calibration of an internally validated model was evaluated with a calibration graph describing the relationship between the expected and observed MPI risks. Restricted cubic splines were used to visualize the relationship. The calibration of the model was also evaluated using the Hosmer-Lemeshow test.

Score and nomogram

We presented a model as scores for clinical use. For each variable we assigned a score that was proportional to the strength of association with the primary outcome of MPI. Therefore, the variables of the final prediction model with higher OR had higher scores. The final score was obtained by the sum of the scores of each variable in the final model. The scores were created by multiplying the variable’s coefficient β by 10. We also created A nomogram, which allows the direct calculation of the probability of the primary outcome. In the nomogram, each variable has a different scale that is proportional to the weight of its coefficients ^{20 , 24 - 27}. The equivalence of MPI scores and risks with preoperative variables was also calculated. We carried out all these analyses with the R 3.3.2 program (www.r-project.org).

Validation of STS model

Besides discrimination and internal calibration, the comparison of infection rates was also used for model validation. Our model and STS model were compared using the MPI rates of six heart surgeries comparable to our study (atrial septal defect, ventricular septal defect, atrial ventricular canal, tetralogy of Fallot, Glenn, and double outlet right ventricle) through Fisher’s exact test.

Ethical considerations

The Ethics Committees of Universidad Peruana de Ciencias Aplicadas (CEI/515-03-15) and INSN (Letter No. 01265-CEI- INSN-2015) approved the realization of the study.

Demographic characteristics

A total of 1,025 patients were analyzed, 59 (5.8%) of them had at least one episode of MPI (4.8% sepsis, 1.0% mediastinitis, 0% endocarditis). Three patients had more than one type of infection. Ninety-five percent of the operated patients had postoperative sepsis. There were only ten cases of mediastinitis.

Patients with MPI were younger (2.95 vs. 5.42 years; p < 0.001), and 62.7% were men (Table 1). The average weight of patients with MPI was significantly lower than the rest of the patients (10.9 kg vs. 16.8 kg; p < 0.001) (Table 2). The length of preoperative stay was longer in patients with MPI (median of 40 vs. 15 days; p < 0.001). Most patients with MPI were diagnosed with cyanotic heart disease (69.5%; p < 0.001). In addition, patients with MPI had a higher percentage on the RACHS-1 3-4 scale (54% vs. 27%; p < .001). A similar result was found for Ross-modified functional class IV (higher in patients with MPI, 37% vs. 15% in the total population, p < 0.001). On the other hand, more patients with MPI had previous mechanical ventilation (11.9% vs 0.9%; p < 0.001). There was a greater number of patients with a history of previous infection resolved at the time of surgery in the cases with MPI (7.5% vs 32%; p < 0.001). There were no differences in congenital anomalies, malformations, and malnutrition status.

We found that patients undergoing non-elective surgery had more MPIs (79.7% vs 44.8%; p < 0.001) (Table 2). There was no difference between the cases of MPI for the year of surgery, the longest time of extracorporeal circulation, deep hypothermia, open chest, and valve prosthesis.

The total mortality rate at 30 days was 13.7% and significantly higher in the group of patients with MPI (mortality 37/59 [63%] vs. 125/966 [13%]; p < 0.001). The duration of postoperative mechanical ventilation (301.6 hours vs. 34.3 hours; p < 0.001), postoperative intensive care unit stay (20.9 days vs. 5.1 days; p < 0.001), and postoperative stay (42.2 days vs. 15.1 days; p < 0.001) were greater in patients with PMI. Nine deaths with MPI had a history of sepsis and two of endocarditis. Excluding these patients, mortality in patients with MPI remained high (58%) and significantly different from patients without MPI (p < 0.001). Of the 32 patients, two died of sepsis.

Bivariate and multivariate regression model

In the bivariate analysis of preoperative variables, a strong association was found between MPI and younger age, lower weight, longer preoperative stay, type of heart disease, cyanotic heart disease, higher RACHS-1 scale and Ross-modified functional class, and previous mechanical ventilation (Table 1 and 2). The non-elective (vs. elective) surgeries also had a statistical association with MPI (Table 2).

We created a preoperative model and a combined model (preoperative and intraoperative). In the first model, shown in Table 3, the variables associated with PMI were prolonged preoperative hospital stay (p = 0.053), diagnosis of cyanotic heart disease (p = 0.020), RACHS-1 (p = 0.013), and Ross-modified functional class IV (p = 0.009). In the second model, the associated variables were cyanotic heart disease (p = 0.020), RACHS-1 (p = 0.020), Ross-modified functional class IV (p = 0.010), and non-elective surgical condition (p = 0.020).

Performance of the preoperative model

Internal validation of the model produced a shrinkage factor of 0.84. The discriminatory performance of the model was good, with an adjusted c-statistic of 0.80 (95% CI 0.74-0.86). The combined preoperative and intraoperative model showed no significant improvement in discrimination compared to the preoperative model (P = 0.600), which is shown in the supplemental material (Figure S1). The net benefit analysis showed that both models had a similar performance for MPI ranging from 0% to 40%, and their use is better than not using a predictive model (supplemental material, Figure S2). The calibration of both models was good as expected in an internally validated model.

The preoperative model was presented as scores and a nomogram. The scores per variable included in the final model are shown in Table 4, and the equivalence of MPI scores and risks are shown in Table 5. Scores between 12 and 29 correspond to MPI probabilities of 5% to 95%, respectively. A nomogram is shown in Figure 1. For example, a 5-year-old girl with 20 kg, 30 days of preoperative stay, with cyanotic heart disease and a RACHS-1 scale of 3 will have a 19-point and 30% risk of MPI.

Figure 1 Nomogram for the preoperative model. 

Validation of the Society of Thoracic Surgeons model

The STS model for MPI was validated with our own database. The discriminatory performance was moderate (c-statistical 0.78; 95% CI 0.71 to 0.84) and the calibration was not good (Hosmer-Lemeshow test was 18.2; P = 0.020). The STS model underestimated the risk of MPI, particularly for the high risks observed in our population. Fisher’s exact test was used to compare both models regarding the MPI rate (supplemental material, Table S1). Significant differences were found in two surgeries: atrial septal defect and double outlet right ventricle (p = 0.010 and p = 0.030, respectively).