A machine learning model for predicting severe mycoplasma pneumoniae pneumonia in school-aged children

Abstract

Objective

To develop an interpretable machine learning (ML) model for predicting severe Mycoplasma pneumoniae pneumonia (SMPP) in order to provide reliable factors for predicting the clinical type of the disease.

Methods

We collected clinical data from 483 school-aged children with M. pneumoniae pneumonia (MPP) who were hospitalized at the Children's Hospital of Soochow University between September 2021 and June 2024. Difference analysis and univariate logistic regression were employed to identify predictors for training features in ML. Eight ML algorithms were used to build models based on the selected features, and their effectiveness was validated. The area under the curve (AUC), accuracy, five-fold cross-validation, and decision curve analysis (DCA) were utilized to evaluate model performance. Finally, the best-performing ML model was selected, and the Shapley Additive Explanations (SHAP) method was applied to rank the importance of clinical features and interpret the final model.

Results

After feature selection, 30 variables remained. We constructed eight ML models and assessed their effectiveness, finding that the CatBoost model exhibited the best predictive performance, with an AUC of 0.934 and an accuracy of 0.9175. DCA was used to compare the clinical benefits of the models, revealing that the CatBoost model provided greater net benefits than the other ML models within the threshold probability range of 34% to 75%. Additionally, we applied the SHAP method to interpret the CatBoost model, and the SHAP diagram was used to visually show the influence of predictor variables on the outcome. The results identified the top six risk factors as the number of days with fever, D-dimer, platelet count (PLT), C-reactive protein (CRP), lactate dehydrogenase (LDH), and the neutrophil-to-lymphocyte ratio (NLR).

Conclusions

The interpretable CatBoost model can help physicians accurately identify school-aged children with SMPP. This early identification facilitates better treatment options and timely prevention of complications. Furthermore, the SHAP algorithm enhances the model's transparency and increases its trustworthiness in practical applications.

Background

Mycoplasma is the most common pathogen that causes community-acquired pneumonia (CAP) in hospitalized children aged five years and older. Since September 2023, there has been a significant increase in the number of M. pneumonia cases among children in different regions of China [1]. In respiratory wards, more than 50% of hospitalized children have been diagnosed with MPP, particularly in the 6–11 age group, where mycoplasma is the primary cause [2, 3]. Although most children with M. pneumonia experience mild symptoms and have a good prognosis, there has been a notable rise in mycoplasma resistance in recent years. It is estimated that over 80% of pneumonia cases in China involve strains of mycoplasma that are resistant to macrolide antibiotics, contributing to an increase in the number of children diagnosed with SMPP [4, 5]. SMPP can lead to more serious inflammatory responses and complications, including necrotizing pneumonia, pulmonary embolism, and obstructive bronchiolitis. These complications pose a significant risk to children's health [6]. Therefore, using existing clinical data to predict the occurrence of SMPP in school-aged children is crucial for clinical practice.

As the incidence of SMPP grows, more studies are focusing on its early prediction. Current predictive models mainly rely on traditional logistic regression methods, which often struggle with data imbalance. Furthermore, these algorithms may not meet the precision needed for modern diagnostics and treatment. Currently, there is no comprehensive model that systematically integrates clinical features and effectively quantifies the early predictive capability of SMPP in school-aged children. With advancements in artificial intelligence, ML has been widely applied in various medical fields due to its ability to handle large datasets and its strong predictive power [7, 8]. Additionally, the SHAP algorithm was used to address the "black box" problem associated with machine learning algorithms. This approach clarifies the predictions made by the optimal model, ranks the importance of the predictors, and provides interpretable outputs [9, 10]. This study aims to integrate multiple clinical and laboratory datasets to develop a ML model that assists frontline clinicians in the early prediction of SMPP in school-aged children, thereby facilitating early diagnosis and targeted treatment strategies.

Materials and methods

Ethics approval

This study was approved by the Ethics Committee of Children's Hospital of Soochow University (2024 CS098). Written informed consent was obtained from the guardians of all participants.

Study population and data collection

A total of 562 school-age children with MPP who were hospitalized at the Children's Hospital of Soochow University from September 2021 to June 2024 were enrolled in this study. After applying the inclusion and exclusion criteria and excluding extreme or abnormal values, 483 samples were ultimately included. Collect clinical information from the pediatric records within the electronic medical record system. Document the patient's age, sex, clinical symptoms, pulmonary complications, fever days, length of hospitalization, the number of fiberoptic bronchoscopy procedures performed, and their results. Clinical symptoms were recorded within two hours of admission. Additionally, record the allergy history, including eczema, allergic rhinitis, allergic sinusitis, and urticaria. Include laboratory test results obtained within 24 h of admission, such as complete blood count, CRP, comprehensive biochemical panel, fluid immunology, cardiac troponins, coagulation profile, and lymphocyte subpopulation analysis. Finally, document the results of the initial chest imaging conducted after admission.

SMPP diagnoses and inclusion and exclusion criteria

Diagnostic criteria

Diagnosis is based on the"Guidelines for the Diagnosis and Treatment of Mycoplasma Pneumonia in Children (2023 Edition)" [11]. MPP is defined by the presence of one or both of the following: (1) a single serum Mycoplasma pneumonia (MP) antibody titer of ≥ 1:160 (PA method) or a fourfold or greater increase in paired serum MP antibody titers during the illness; (2) Positive MP DNA or RNA. SMPP refers to MPP accompanied by one of the following conditions: poor general condition, significantly increased respiratory rate (RR > 70 or > 50 breaths/min), cyanosis, respiratory distress, involvement of ≥ 2/3 of a single lung lobe or multiple lung lobes, pleural effusion, pulse oxygen saturation ≤ 0.93, or extrapulmonary complications. NSMPP refers to children who do not meet the diagnostic criteria for SMPP.

Inclusion criteria

1) Age ≥ 6 years and ≤ 12 years; 2) Presence of respiratory symptoms and/or fever (> 37.3℃); 3) Chest imaging confirming pneumonia; 4) Positive MP-DNA/RNA in nasopharyngeal secretions or BALF, or a fourfold or greater increase in paired serum MP antibody titers.

Exclusion criteria

1) Age < 6 years or > 12 years; 2) Incomplete clinical data; 3) Presence of other respiratory diseases, such as bronchial asthma, bronchopulmonary dysplasia, tuberculosis, primary ciliary dyskinesia, cystic fibrosis, bronchial foreign body, lung tumors, and non-infectious interstitial lung disease; 4) Presence of immunodeficiency, congenital heart disease, or hereditary neurological disorders.

Predictors screening

By comparing the demographic characteristics, clinical features, laboratory tests, bronchoscopy findings, and chest imaging results of the SMPP group and the NSMPP group, we selected variables with statistically significant differences (P < 0.05). Subsequently, we employed univariate logistic regression to filter out variables with P < 0.1.

Model development

This study randomly divided all enrolled children into a training cohort (N = 387) and a testing cohort (N = 96) in an 80:20 ratio. Based on the type of problem and the characteristics of the data, we selected eight machine learning models suitable for classification and prediction during model training. These models include logistic regression (LR), support vector machine (SVM), classification decision tree (CDT), random forest (RF), gradient boosting decision tree (GBDT), light gradient boosting machine (LightGBM), CatBoost, and extreme gradient boosting (XGBoost). Figure 1 illustrates the workflow of this research.

Fig. 1

Work flow chart

Model performance evaluation

The performance of the models was evaluated using AUC, along with additional metrics such as accuracy, sensitivity, specificity, F1 score, and recall. AUC is a performance measure used to evaluate the quality of classifiers, with a higher AUC value indicating better classification performance. Accuracy represents the proportion of correctly identified samples among all samples and is one of the most common metrics for model evaluation. Sensitivity measures the percentage of true positive patients correctly identified as positive; a higher sensitivity indicates a lower rate of missed diagnoses. Specificity is assessed as the percentage of true negative patients correctly identified as negative, with higher specificity reflecting a lower misdiagnosis rate. Recall is an important metric that measures a model's ability to identify all true positive examples in classification tasks; a higher value indicates that the model is more effective at finding all true positives. The F1 score is a weighted average of precision and recall, combining the results of these two metrics. Each model underwent five-fold cross-validation to assess average performance. The ML model with the best overall performance was then selected as the final model, and its applicability was evaluated using DCA.

Model explanation

In addition to evaluating the model, this study uses the SHAP algorithm [12] to rank the importance of predictive factors in the optimal model, providing interpretable outputs. The SHAP algorithm explains predictive models by quantifying the influence of each feature on the model's predictions, highlighting both positive and negative contributions of features. This research uses the SHAP algorithm to clarify the significance of features in the predictive model and to calculate SHAP values, indicating whether the input features are positively or negatively associated with the output. This method addresses the"black box"problem, helping clinicians understand the rationale behind model decisions, which enhances the model's trustworthiness and usability in practical applications.

Statistical analysis

This study utilized SPSS 27 and Python 3.10.0, TensorFlow framework for programming, Pandas library and scikit-learn library for data preprocessing and model training. The included characteristics comprised both numerical and categorical variables. For numerical variables, If the data followed a normal distribution, independent samples t-tests were conducted to compare differences between groups, reported as mean ± standard deviation ($\overline{x }\pm s$). For data that did not conform to a normal distribution, non-parametric tests (Mann–Whitney U test) were employed, with results presented as median and interquartile range (M [P25, P75]). Categorical variables were expressed in terms of frequency and percentage (n [%]), and comparisons were made using chi-square tests. All statistical results were considered significant at P < 0.05.

Results

Patient characteristics

Among the 483 children diagnosed with MPP, 236 were male (48.86%) and 247 were female (51.14%), with an average age of 8.30 ± 1.47 years. Of these, 91children were classified into the SMPP group, while 392 were placed in the NSMPP group. There were no statistically significant differences in gender or age between the two groups (P > 0.10), as shown in Table 1.

Table 1 Baseline characteristics of the patients

The differential analysis of clinical manifestations and laboratory test results between the SMMP and NSMMP patient groups is presented in Table S1 - 2. Compared to the NSMPP group, children with SMPP experienced a longer duration of fever and hospitalization (P < 0.001). Additionally, the incidence of pleural effusion, atelectasis, and lung collapse was significantly higher in SMPP patients (P < 0.001). Furthermore, a greater proportion of SMPP patients underwent electronic bronchoscope compared to NSMPP patients (P < 0.001). Regarding laboratory test results, significant differences were observed in most parameters between SMPP and NSMPP, with the exception of white blood cells (WBC), globulin (GLB), urea, creatinine (CREA), total cholesterol (TCHOL), cholic acid (CG), immunoglobulin A (IgA), immunoglobulin G (IgG), immunoglobulin M (IgM), fibrinogen (Fib), cytotoxic T cells as a percentage of T lymphocytes (CD3 + CD8 + %), natural killer cells (CD3-CD(16 + 56) + %), and high-sensitivity cardiac troponin T (Hs-cTnT) (P < 0.05).

Subsequently, the characteristics that showed significant differences were further analyzed using univariate logistic regression, with the results presented in Table S3. This table includes the odds ratios (OR), confidence intervals, and associated p-values. In summary, the analysis of clinical features and observations demonstrated significant differences in clinical characteristics and laboratory test results between SMPP and NSMPP. Finally, the study identified 30 variables as potential predictive factors for inclusion in the machine learning model.

Model development and performance comparison

All enrolled patients were randomly divided into a training cohort (N = 387) and a testing cohort (N = 96) in an 80:20 ratio. The selected 30 variables were used to train and evaluate eight different ML models. The study used five-fold cross-validation to determine the hyperparameters for each model. First, the original dataset was divided into five mutually exclusive subsets. Next, a hyperparameter space was created for grid search, and each pair of hyperparameter combinations was cross-validated. Finally, the combination with the best average validation performance was selected to determine the hyperparameters for each model. The discriminative performance of each model is presented in Table 2 and Fig. 2. The CatBoost model (AUC = 0.934, ACC = 0.9175) demonstrated the highest predictive performance among the eight models, followed by the RF model (AUC = 0.934, ACC = 0.8792) and the XGBoost model (AUC = 0.927, ACC = 0.8351). To reduce the risk of selection bias in the dataset, five-fold cross-validation was conducted for all models in the test set to obtain the average performance across five predictions. The results, shown in Table S4, indicate that the CatBoost model outperforms the other ML models in terms of average predictive performance.

Table 2 Performance of each model for prediction

Fig. 2

ROC curves and precision-recall curves for 8 models

Additionally, DCA was performed on five machine learning models with an AUC greater than 0.9 in the testing dataset to compare the net benefits of different clinical decision-making methods. DCA assesses the clinical net benefit of predictive models by comparing the intervention strategies suggested by the models against the default strategies of intervening in all patients or not intervening at all. The clinical net benefit is defined as the minimum probability at which further intervention in the disease is justified [13]. Figure 3 illustrates the net benefits at various threshold probabilities. The blue line represents the scenario in which all patients receive the intervention, while the yellow line indicates the scenario where no patients receive the intervention. The strategies provided by any of the five machine learning models outperform the default strategies of either intervening with all patients or not intervening at all. Within the threshold probability range of 34% to 75%, the CatBoost model demonstrates superior net benefits compared to the other machine learning models.

Fig. 3

Decision curve analysis for five models. Net benefits are plotted for different threshold probabilities

Model interpretation using the SHAP algorithm

The SHAP algorithm can be utilized to elucidate the individual predictive variables of the final model, allowing us to ascertain the significance and polarity of each predictor's contribution to the model's predictions. The ranking of variable importance is illustrated in Fig. 4. The number of days with fever exhibits the strongest predictive value across all ranges, followed by D-dimer levels. Furthermore, the horizontal axis (SHAP values) indicates the magnitude and direction of each feature's impact on the prediction outcome. The further a point is from the central line, the greater the influence of that feature on the model's output; positive SHAP values denote a positive impact, while negative SHAP values indicate a negative impact. The color coding reflects whether the variable is high (red) or low (blue) in that particular row of the dataset. It is evident that increases in the number of fever days, D-dimer, CRP, LDH, and NLR have a positive influence, propelling the prediction towards SMPP, whereas increases in PLT and the albumin-to-globulin ratio (A:G) exert a negative influence, steering the prediction towards NSMPP.

Fig. 4

Explanation of predictive modeling for SMPP in school-aged children. Top 20 risk predictors for early prediction of SMPP in school-aged children

Using SHAP values, Fig. 5 illustrates the explanations for two different predictions. Features that increase the predicted probability of SMPP (compared to the average prediction, referred to as the baseline) are displayed in red, while features that decrease the prediction are shown in blue. In Fig. 5a, most parameters for the affected child fall outside the normal range (with moderate to severe pleural effusion and elevated levels of NLR, D-dimer, CRP, natural killer (NK) cells, B cells, and LDH), leading the model to predict an increased risk of SMPP for this child. Conversely, Fig. 5b depicts a child without moderate to severe pleural effusion, with most parameters within the normal range (including NLR, D-dimer, NK, B cells, PLT, creatine kinase (CK), prealbumin (PA), alanine aminotransferase (ALT), etc.), resulting in the model predicting a lower risk of SMPP for the second child.

Fig. 5

Inference process of the model with (a) SMPP and (b) NSMPP

Discussion

We developed a machine learning model for the early diagnosis of SMPP, based on children with MP infection admitted to the Children's Hospital of Soochow University. Additionally, we employed the SHAP algorithm to interpret the optimal machine learning model, illustrating the impact of predictive features on the predicted outcomes.

In previous studies predicting pediatric SMPP, researchers have constructed models using classical logistic regression. The nomogram model achieved an AUC value of 0.777 for predicting pediatric SMPP [14].The predictive features included immunoglobulin M, eosinophil percentage, eosinophil count, hemoglobin, erythrocyte sedimentation rate, and prealbumin. Similarly, another model also developed a nomogram model for predicting pediatric SMPP, reporting AUC values of 0.867 and 0.840 for the training and validation sets, respectively [15]. With the advancement of machine learning, CatBoost has emerged as a superior method for constructing medical predictive models. CatBoost [16, 17] is a gradient boosting framework based on decision trees. Compared to traditional decision tree models, CatBoost excels in handling categorical variables. It calculates target statistics for categorical features to reduce distribution bias in training and testing data, thereby improving model accuracy and generalization performance. Furthermore, CatBoost provides excellent predictive results without the need for extensive parameter tuning, which reduces the risk of overfitting and facilitates the development of more robust models. Researchers have employed six machine learning models to predict the severity of acute pancreatitis, finding that the CatBoost model demonstrated optimal performance, with an average AUC score of 0.81 ± 0.033 and an accuracy of 89.1% [18]. In another study [19], six machine learning models were used to predict survival outcomes in patients with recurrent cervical cancer, and the results similarly indicated that the CatBoost model outperformed all other models, achieving an AUC of 0.99 and an accuracy of 95.6%. This study also utilized eight machine learning models to predict SMPP in school-aged children, with the CatBoost model achieving an AUC value of 0.991 and an accuracy of 0.964. Compared to other machine learning models, the CatBoost model exhibited superior accuracy. In clinical applications, the feasibility of the CatBoost model is greater than that of traditional logistic regression, as distinguishing between SMPP and NSMPP based on clinical data requires high accuracy, sensitivity, and specificity.

The strengths of this study are as follows: First, the clinical features included were multidimensional, encompassing both clinical presentations and laboratory tests. These data are commonly available in most care settings, enhancing the generalizability of the prediction models. Second, we utilized machine learning algorithms to train the models, which are better suited for handling complex data and nonlinear relationships compared to traditional logistic regression. In clinical practice, the occurrence of disease typically results from a combination of factors, making machine learning algorithms more appropriate for the early prediction of severe Mycoplasma pneumoniae pneumonia (SMPP) and capable of providing accurate predictions. Furthermore, transparency and interpretability of predictive models are crucial in real-world clinical settings. However, machine learning models often lack explanations for their outputs. In this study, we employed explainable artificial intelligence tools (SHAP) to address the issue of interpretability. Consequently, the model we constructed provides interpretable outputs while effectively capturing complex data, offering valuable support for the early clinical identification of SMPP.

In this study, we provided a comprehensive interpretation of the model using SHAP summary plots. We identified several clinical factors that may be associated with the development of SMPP in school-age children. These factors include the number of days with fever, D-dimer levels, PLT, CRP, LDH, and the NLR. Numerous studies have confirmed that a longer duration of fever, along with elevated levels of D-dimer, CRP, LDH, and NLR, is associated with an increased likelihood of developing SMPP [20,21,22,23]. However, there is still a lack of research on PLT in relation to SMPP. The findings of this study indicate that lower PLT levels are associated with a higher risk of developing SMPP. PLT levels in the pediatric SMPP group are significantly lower than those in the mild illness group and the healthy control group, which aligns with the results of this study and confirms the importance of PLT in the early prediction of SMPP [24]. PLT refers to the number of platelets in a unit volume of peripheral blood, reflecting the dynamics of platelet production and apoptosis in the bloodstream. In recent years, a growing body of evidence has shown that platelets are not only essential components in mediating hemostasis and thrombosis but also play an active role in the immune response to microorganisms and foreign substances [25,26,27]. When the body is infected by pathogens, the interaction between platelets and neutrophils triggers an immune response that facilitates the progression of inflammation. During bacterial infections, platelets actively mediate the host response through interactions with circulating leukocytes. Additionally, platelets can interact with various viruses, and thrombocytopenia often affects the severity and prognosis of viral infections. Research has shown that the antiplatelet agent aspirin is beneficial in the treatment of influenza [28,29,30]. However, there is currently a lack of studies examining the interactions between platelets and mycoplasma infections. This study has demonstrated the clinical value of platelets in predicting SMPP, highlighting the need for further foundational research to explore the role of platelets following mycoplasma infection. Such research could inform clinical strategies for the prevention and treatment of SMPP from multiple perspectives.

Certainly, this research has several limitations. First, the model was developed retrospectively using data from a single center, which introduces inherent biases in the data collection process. Second, the model validation was conducted solely through internal validation methods. To enhance the model's generalizability, it may be necessary to utilize additional external datasets for validation. Third, the proposed model lacks prospective validation, which will be a focus of future research efforts.

Conclusions

The CatBoost model shows promise as a tool for the early identification of SMPP in school-age children. To further validate the findings of this study, multi-center validation and large-scale prospective studies are recommended.