Home
JournalsCollections
For Authors For Reviewers For Editorial Board Members
Article Processing Charges Open Access
Ethics Advertising Policy
Editorial Policy Resource Center
Company Information Contact Us
OPEN ACCESS

Development and Validation of a New Prognostic Model for Predicting Survival Outcomes in Patients with Acute-on-chronic Liver Failure

  • Wende Li1,#,
  • Wanshu Liu2,#,
  • Yihui Rong1,#,
  • Dongze Li2,
  • Bing Zhu2,
  • Shaobo Yang1,
  • Shidong Sun1,
  • Shaoli You2,* ,
  • Yu Chen3,*  and
  • Jun Li4,* 
Journal of Clinical and Translational Hepatology   2024;12(10):834-844

doi: 10.14218/JCTH.2024.00316

Received:

Revised:

Accepted:

Published online:

 Author information

Citation: Li W, Liu W, Rong Y, Li D, Zhu B, Yang S, et al. Development and Validation of a New Prognostic Model for Predicting Survival Outcomes in Patients with Acute-on-chronic Liver Failure. J Clin Transl Hepatol. 2024;12(10):834-844. doi: 10.14218/JCTH.2024.00316.

Abstract

Background and Aims

Early determination of prognosis in patients with acute-on-chronic liver failure (ACLF) is crucial for optimizing treatment options and liver allocation. This study aimed to identify risk factors associated with ACLF and to develop new prognostic models that accurately predict patient outcomes.

Methods

We retrospectively selected 1,952 hospitalized patients diagnosed with ACLF between January 2010 and June 2018. This cohort was used to develop new prognostic scores, which were subsequently validated in external groups.

Results

The study included 1,386 ACLF patients and identified six independent predictors of 28-day mortality through multivariate analysis (all p < 0.05). The new score, based on a multivariate regression model, demonstrated superior predictive accuracy for both 28-day and 90-day mortalities, with Areas under the ROC curves of 0.863 and 0.853, respectively (all p < 0.05). This score can be used to stratify the risk of mortality among ACLF patients with ACLF, showing a significant difference in survival between patients categorized by the cut-off value (log-rank (Mantel–Cox) χ2 = 487.574 and 606.441, p = 0.000). Additionally, the new model exhibited good robustness in two external cohorts.

Conclusions

This study presents a refined prognostic model, the Model for end-stage liver disease-complication score, which accurately predicts short-term mortality in ACLF patients. This model offers a new perspective and tool for improved clinical decision-making and short-term prognostic assessment in ACLF patients.

Graphical Abstract

Keywords

Acute-on-chronic liver failure, ACLF, Outcome prediction, Mortality, Prognosis, Model

Introduction

Acute-on-chronic liver failure (ACLF) is a complex clinical syndrome with a high short-term mortality rate of 50–90%.1,2 It is characterized by clinical complications such as ascites, hepatic encephalopathy, infection, and portal hypertensive hemorrhage.3 Research has revealed that most patients with ACLF have clinically apparent triggers of systemic inflammation, including proven bacterial infection and gastrointestinal hemorrhage with shock.4 Despite intensive care and extracorporeal artificial liver support systems, some ACLF patients at high risk of rapid clinical deterioration may require emergency liver transplantation (LT) to improve their chances of survival.5,6 Due to the rapid progression of ACLF, LT faces practical challenges such as liver shortages and high costs. Therefore, accurate early prognosis of ACLF patients is critical for optimizing treatment options and liver allocation.

Several models have been proposed to predict outcomes for these patients. For example, the Chronic Liver Failure Consortium (hereinafter referred to as CLIF-C) ACLF model and the North American Consortium for the Study of End-stage Liver Disease model, validated by European and American consortia on ACLF, respectively, reflect terminal events in ACLF. These models are primarily suitable for high-risk ACLF patients in the intensive care unit, but their sensitivity to specific complications (such as infection) is inadequate.7,8 The Asian-Pacific Association for the Study of the Liver (hereinafter referred to as APASL)-ACLF Research Consortium model is effective for predicting short-term mortality in ACLF patients9 and is superior to scales such as the Sequential Organ Failure Assessment (hereinafter referred to as SOFA), CLIF-C-SOFA, and Acute Physiology and Chronic Health Evaluation scores in assessing ACLF severity.10 However, its specificity for predicting 30-day mortality in ACLF patients is only 57%.11 Given these limitations, there is no consensus on which model is best suited for different populations.

The model for end-stage liver disease (MELD) is a well-recognized scale used for liver transplant allocation. The MELD model is based on serum creatinine, total serum bilirubin, and the international normalized ratio (INR). However, it does not account for all complications related to portal hypertension, such as esophageal variceal bleeding, spontaneous bacterial peritonitis, ascites, and hepatic encephalopathy.12 Additionally, the MELD-sodium score lacks specificity in predicting outcomes for ACLF patients.13 A team from Stanford University School of Medicine and Mayo Clinic further optimized the MELD model to create MELD 3.0, which includes albumin.14 However, this model does not yet address the impact of ACLF-related complications on prognosis. The albumin-bilirubin (ALBI) score is a new model for assessing liver dysfunction severity, based on albumin and bilirubin levels. Although the ALBI score has prognostic value in various contexts, it may be limited in predicting complications related to factors such as multiple organ failure.15 Few large cohort studies have compared the prognostic efficacy of these new scores in ACLF patients.

Common complications affecting ACLF prognosis include spontaneous bacterial peritonitis (SBP), acute kidney injury (AKI), hepatic encephalopathy (HE), variceal bleeding, pneumonia, and hepatic hydrothorax.16–20 Understanding these and other risk factors could help identify ACLF patients with a poor prognosis due to complications. However, there is limited information about the impact of various complications on ACLF in large cohorts. This study aimed to identify the risk factors associated with ACLF and develop new prognostic models to accurately predict patient outcomes.

Methods

Study population

We retrospectively collected data from patients admitted to the Fifth Medical Center of the Chinese PLA General Hospital between January 2010 and June 2018. As this was a retrospective study, we could not obtain direct consent from the participants. To maximize privacy protection, we ensured that data were anonymized throughout the study. This study was approved by the Ethics Committee of the 302 Hospital of PLA, the predecessor of the Fifth Medical Center of the Chinese PLA General Hospital.

Patient inclusion criteria

We selected patients who were diagnosed with ACLF and admitted to the Fifth Medical Center of the Chinese PLA General Hospital between January 2010 and June 2018. According to the standards proposed by the Asia–Pacific Association for the Study of the Liver in 2019, patients diagnosed with ACLF met the following conditions: (1) serum total bilirubin ≥5 mg/dL; (2) INR ≥1.5; (3) preexisting chronic liver disease; and (4) complications, including ascites or HE. The exclusion criteria were as follows: (1) preexisting cancer at diagnosis; (2) age younger than 13 years; (3) scheduled liver transplantation; (4) missing data or hospital stay < three days; and (5) diabetes mellitus, chronic kidney disease, or chronic thyroid disease. Although diabetes mellitus is relatively common in the ACLF population, its presence may obscure the relationship between interventional measures and the natural history of liver failure. To minimize the confounding effects of diabetes on study outcomes and to enhance the precision of clinical management and prognostic accuracy within the ACLF cohort, patients with diabetes mellitus were excluded from this investigation.

All patients were followed up for at least three months after discharge or until death. Data on patient outcome events, such as liver transplantation or death, were collected through outpatient medical records or telephone follow-up.

Diagnostic criteria for complications

HE was confirmed according to the West Haven criteria. The presence of ascites was confirmed via ultrasonography and assessed by the need for treatment with diuretics or paracentesis. Variceal bleeding was defined as bleeding requiring endoscopic intervention. AKI was defined according to internationally accepted criteria as an increase in serum creatinine ≥0.3 mg/dL from baseline.21,22

Data collection and score calculation

Information on sex, age, albumin, total bilirubin, creatinine, total cholesterol, leukocyte count, hemoglobin, platelet count, INR, and ammonia on admission, as well as treatment data after admission, were collected from medical records. These data were used to calculate the MELD, MELD 3.0, and ALBI scores as follows:

  • MELD score = 3.78 × loge (bilirubin)+ 11.2 × loge (INR) + 9.57 × loge (creatinine) + 6.43.12

  • MELD 3.0 score = 1.33 (if female) + 4.56 × loge (bilirubin) + 0.82 × (137−Na) − 0.24 × (137−Na) × loge (bilirubin) + 9.09 × loge (INR) + 11.14 × loge (creatinine) + 1.85 × (3.5–albumin) − 1.83 × (3.5–albumin) × loge(creatinine) + 6.14

  • ALBI score = 0.66 × log10(bilirubin) - 0.085 × (albumin).

  • The ALBI score was subsequently classified into three grades: Grade 1: ≤−2.6; Grade 2: >−2.6, ≤−1.39; and Grade 3: >−1.39.23

Outcome measures

The primary outcome was mortality at 28 days, and the secondary outcome was mortality at 90 days. Patient survival status was confirmed after discharge through telephone interviews and/or by analyzing medical records. Each patient was followed for three months or until their date of death.

Development and validation of a new prognostic score model

All patients with ACLF were assigned to a training set for the development of a new prognostic model. Initially, univariate analysis was conducted to examine how each factor of interest influenced short-term mortality (28-day). Subsequently, multivariate logistic regression analysis was used to identify independent factors associated with 28-day LT-free mortality (Table 1). We integrated all the parameters identified through multivariate logistic regression into a new ACLF scoring model. We then evaluated and compared the performance of this new ACLF model, referred to as the MELD-complication score for simplicity in the following text. An external test set containing data from The First Affiliated Hospital, Zhejiang University School of Medicine (Hangzhou set) and Beijing Youan Hospital, Capital Medical University (Youan set) was used for validation. All data required to calculate the ACLF score were measured and are presented in Table 1 (test set). To assess the generalizability of the new model, we collected patient data from the aforementioned hospitals in Hangzhou and Beijing, namely, The First Affiliated Hospital, Zhejiang University School of Medicine (Hangzhou set) and Beijing Youan Hospital, Capital Medical University (Youan set), as external test sets to verify the predictive performance of the new model in different populations. Both hospitals, being of the same level in first-tier cities, ensured homogeneity in treatment across the external test sets.

Table 1

Logistic regression analysis of factors associated with the risk of 28-day mortality

ParametersUnivariable Analysis OR (95% CI)p-valueMultivariable Analysis OR (95% CI)p-value
Age1.432 (1.131–1.812)0.0031.420 (1.050–1.922)0.023
Aetiology of the disease1.299 (0.949–1.779)0.103
ACLF subtype1.976 (1.379–2.830)0.0001.976 (1.396–2.797)0.000
Variceal bleeding2.605 (1.871–3.628)0.000
Hepatic encephalopathy5.220 (4.066–6.703)0.0002.281 (1.689–3.080)0.000
Hydrothorax1.271 (0.997–1.620)0.053
Pulmonary infection3.394 (2.579–4.466)0.0001.413 (1.002–1.994)0.049
SBP0.660 (0.477–0.913)0.0120.646 (0.473–0.883)0.006
AKI3.869 (3.031–4.938)0.000
Albumin1.383 (1.096–1.745)0.006
Total bilirubin1.636 (1.297–2.063)0.000
INR3.709 (2.899–4.744)0.0002.806 (2.072–3.800)0.000
Platelets1.094 (0.868–1.379)0.445
Creatinine5.305 (4.133–6.809)0.000
Urea3.382 (2.629–4.352)0.000
Sodium1.652 (1.266–2.155)0.000
Total cholesterol1.955 (1.546–2.473)0.000
Ammonia3.219 (2.528–4.099)0.000
Haemoglobin1.195 (0.864–1.653)0.282
WBC2.086 (1.607–2.708)0.000
Neutrophil count7.832 (6.054–10.131)0.0002.923 (2.119–4.034)0.000
MELD score11.939 (8.927–15.969)0.0004.270 (3.026–6.025)0.000
MELD-3.0 score3.640 (2.851–4.649)0.000
ALBI score3.000 (2.364–3.808)0.000

Statistical analysis

All statistical analyses were performed using SPSS software version 22.0 (IBM Corp, Armonk, NY, USA). For continuous variables, the Kolmogorov–Smirnov test assessed the normality of the data. Continuous variables are expressed as means ± standard deviations or medians (interquartile ranges) depending on normality and were compared between groups using a t test or the Mann–Whitney U test, respectively. Categorical variables are expressed as percentages and were compared between groups using the chi-square test or Fisher’s exact test. Survival curves for 28-day and 90-day mortality were plotted based on the number of complications at baseline, and survival differences between groups with different numbers of complications were assessed using the log-rank test. Logistic regression models identified independent risk factors for mortality, with results presented as odds ratios (ORs) and 95% confidence intervals (CIs). The Hosmer–Lemeshow goodness-of-fit test assessed the calibration ability of the prediction model. The predictive ability of each score was valuated via receiver operating characteristic (ROC) analysis. Areas under the ROC curves (AUROCs) and their 95% CIs for the MELD, MELD 3.0, ALBI scores, and the MELD complication score in predicting 28-day and 90-day survival rates of ACLF patients were calculated and compared using the DeLong test. Critical values were determined by maximizing the Youden index, and sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. Cumulative survival at 28 and 90 days was analyzed using the Kaplan–Meier method to compare survival times between groups above and below the cut-off value. The predictive performance of the new model was validated using an external test set, and ROC curves compared the predictive power of different prognostic models for 28-day and 90-day mortality in ACLF patients. Finally, AUROCs for the different models were calculated and compared using the DeLong test. A two-tailed p-value ≤ 0.05 was considered statistically significant.

Result

Study subjects

Of the 1,952 patients initially identified for inclusion in this study, the majority were excluded for various reasons. Ultimately, 1,386 patients were enrolled in the study (detailed information is presented in Fig. 1). Patients who underwent LT within 90 days of hospital admission were also excluded. Clinical characteristics of the enrolled patients: The mean age of the entire cohort was 46.68 ± 9.95 years, and the majority of patients were male. All included patients were followed up for at least three months or until death. The patients were divided into survival and nonsurvival groups based on 90-day follow-up outcomes. The baseline clinical characteristics and laboratory test results of the patients were compared between the survival and nonsurvival groups, as summarized in Table 2. Viral hepatitis B was the most common etiology in both groups, followed by alcohol-related liver disease and other etiologies, including autoimmune hepatitis, drug-induced chronic liver disease, and nonalcoholic steatohepatitis.

Flow chart of patient enrollment.
Fig. 1  Flow chart of patient enrollment.

ACLF, acute-on-chronic liver failure; HCC, hepatocellular carcinoma.

Table 2

Comparison of clinical characteristics between the survival and nonsurvival groups

Survival group (n = 797)Nonsurvival group (n = 589)p-value
Age (years), mean (SD)45.77 ± 9.6647.92 ± 9.494.1170.000a
Male, n (%)682 (85.57)488 (81.61)1.9030.168c
Aetiology of the disease9.4610.009c
  HBV, n (%)672 (84.31)471 (79.96)
  Alcohol, n (%)76 (9.54)55 (9.34)
  Others, n (%)49 (6.15)63 (10.70)
ACLF subtype84.2170.000c
  Subtype A, n (%)353 (44.29)122 (20.71)
  Subtype B, n (%)369 (46.30)397 (67.40)
  Subtype C, n (%)75 (9.41)70 (11.88)
Complications
  Ascites, n (%)596 (74.78)479 (81.32)14.9810.002c
  Variceal bleeding, n (%)69 (8.66)95 (16.13)18.1240.000c
  Hepatic encephalopathy138.2460.000c
    No, n (%)564 (70.76)234 (39.73)
    Grade I, n (%)190 (23.85)262 (44.48)
    Grade II, n (%)33 (4.14)70 (11.88)
    Grade III-IV, n (%)10 (1.25)23 (3.90)
  Hydrothorax, n (%)233 (29.23)211 (35.82)11.9460.001c
  Pulmonary infection, n (%)104 (13.05)168 (28.52)51.6590.000c
  SBP, n (%)237 (29.74)212 (35.99)11.4470.001c
  AKI, n (%)248 (31.12)366 (62.14)132.1010.000c
Number of complications
  0, n (%)51 (6.40)6 (1.02)127.1580.000c
  1, n (%)344 (43.16)126 (21.39)
  2, n (%)280 (35.13)259 (43.97)
  ≥3, n (%)122 (15.31)198 (33.62)
Laboratory data
  Albumin (g/L), mean (SD)29.78 ± 5.3228.67 ± 5.763.6930.000a
  Total bilirubin (µmol/L), mean (SD)202.25 ± 172.42383.21 ± 162,6919.7820.000a
  INR, mean (SD)1.95 ± 0.422.18 ± 0.6510.3440.000a
  Sodium (mmol/L), mean (SD)133.87 ± 5.63132.17 ± 6.145.3480.000a
  Total cholesterol (mmol/L), mean (SD)1.72 ± 0.951.46 ± 1.124.6280.000a
  Ammonia (µmol/L), mean (SD)94.51 ± 43.07121.61 ± 59.655.5820.000a
  Haemoglobin (g/L), mean (SD)120.47 ± 24.25117.39 ± 27.382.2110.027a
  WBC (109/L), median (IQR)6.37 (4.59, 8.82)7.88 (5.51, 11.26)6.8780.000b
  Neutrophil count (109/L), median (IQR)4.15 (1.52,4.77)5.71 (3.56,8.94)7.2400.000b
  Platelets (109/L), mean (SD)94.30 ± 55.1589.95 ± 57.961.4190.156a
  Neutrophil/PLT ratio, median (IQR)0.04 (0.02–0.08)0.15 (0.08–0.30)18.2110.000b
  Creatinine (µmol/L), median (IQR)87.00 (77.00,110.00)93.00 (80.00,120.00)5.6420.000b
  Urea (mmol/L), median (IQR)6.05 (4.50, 9.42)10.20 (6.10, 16.50)11.0850.000b
  MELD score, median (IQR)18.32 (14.05,25.21)32.60 (26.06,41.29)21.0680.000b
  MELD 3.0 score, median (IQR)26.80 (24.70,29.30)30.10 (27.20,33.32)11.6620.000b
  ALBI score, median (IQR)−1.12 (−1.42, −0.79)−0.76 (−1.04, −0.51)13.9180.000b
ALBI grade81.4950.000b
  Grade 11 (0.13)0 (0.00)
  Grade 2224 (28.11)51 (8.66)
  Grade 3572 (71.76)538 (91.34)

We aimed to identify parameters that could differentiate survivors from nonsurvivors to develop a new prognostic score model for patients with ACLF. The nonsurvival group exhibited notable increases in several parameters, including INR, total serum bilirubin, ammonia, white blood cell count, serum blood urea nitrogen, serum creatinine, MELD score, MELD 3.0 score, and ALBI score, compared to the survival group. Specifically, the mean ALBI scores of the survival and nonsurvival groups were −1.12 (−1.42, −0.79) and −0.76 (−1.04, −0.51), respectively (Z = 13.918, p = 0.000). The survival group had higher serum albumin and total cholesterol levels than the nonsurvival group. Additionally, the incidence of hepatic complications in the nonsurvival group was greater than in the survival group (all p < 0.005). These complications included ascites, HE, AKI, variceal bleeding, SBP, pulmonary infection, and hepatic hydrothorax.

Survival analysis was conducted for 28-day and 90-day mortality according to the number of complications at baseline (Fig. 2). The survival curve showed that as the number of complications increased, the survival rate of patients decreased significantly. The proportion of patients with three or more complications in the nonsurvival group was significantly greater than in the survival group for both 28-day and 90-day mortality rates [log-rank (Mantel–Cox) χ2 = 143.265 and 211.153, respectively, p = 0.000]. Thus, we can conclude that the number of complications is a critical predictor of short-term prognosis for patients with ACLF, with an increase in complications significantly raising the risk of death.

Survival curves for mortality.
Fig. 2  Survival curves for mortality.

Survival curves for 28-day (A) and 90-day (B) mortality according to the number of complications at baseline.

Univariate and multivariate analyses of factors associated with 28-day mortality in ACLF patients

Clinical data and laboratory indicators collected at admission were used to select the most significant risk factors. We conducted univariate and multivariate analyses to identify the risk factors associated with 28-day mortality. Univariate analysis revealed that age, INR, serum creatinine, the MELD score, the ALBI score, HE, variceal bleeding, SBP, and pulmonary infection were associated with 28-day mortality in patients with ACLF. These characteristics were then included in multivariate logistic regression (Table 1); however, we excluded the ammonia level to avoid collinearity. The results of the multivariate analysis revealed that HE, pulmonary infection, and SBP were independent risk factors for short-term mortality and were indeed the most important complications for short-term mortality (ORs = 2.281, 1.413, and 0.646, respectively; all p < 0.05). Other well-known risk factors, including age, the ACLF subtype, INR, neutrophil count, and MELD score, were also found to be independent risk factors for short-term mortality (ORs = 1.420, 1.976, 2.806, 2.923, and 4.270, respectively; all p < 0.05).

Development of the MELD-complication score

The MELD score is currently recognized as the scoring model for liver transplant allocation. However, because it does not sufficiently consider the impact of clinical complications on prognosis, independent predictors of short-term mortality were determined through multivariate analysis, and a multivariate regression model incorporating the MELD score was developed. As shown by the results of the multivariate analysis, age, the ACLF subtype, INR, MELD score, HE grade, SBP, and pulmonary infection (PI) were found to be independent predictors of 28-day mortality. The formula for this new score, named the MELD-complication score, is as follows: MELD-complication score = −3.585 + 1.452 × (log10(MELD score)) + 0.825 × HE grade + 1.073 × log10(neutrophil count (109/L)) + 1.032 × INR + 0.681 × ACLF subtype (liver cirrhosis = 1; non- liver cirrhosis = 0) + 0.346 × age(≥45, y = 1; <45, y = 0) + 0.346 × PI (PI = 1; non-PI = 0) - 0.437 × SBP (SBP = 1; non-SBP = 0). The results of the Hosmer–Lemeshow goodness-of-fit test (chi-square = 9.215, p = 0.324) indicated that the predicted values of the model and the actual observed values were not significantly different and that the prediction model had good calibration ability.

Assessment of the performance of the ALBI score, MELD score, MELD 3.0 score, and MELD-complication score in predicting 28-day mortality

The AUROCs of the three existing prognostic scores (ALBI, MELD, and MELD 3.0) and the novel MELD-complication score were compared to predict the primary outcome. For predicting 28-day mortality, the AUROCs (95% CIs) were as follows: ALBI score, 0.675 (0.645–0.705); MELD score, 0.694 (0.662–0.725); MELD 3.0 score, 0.697 (0.666–0.728); and MELD-complication score, 0.863 (0.843–0.882) (Fig. 3A, C). These results indicate that the MELD-complication score exhibited the highest predictive strength for 28-day mortality, outperforming the ALBI score and the two other MELD scores.

ROC curves and survival analysis for ACLF mortality prediction in the training cohort.
Fig. 3  ROC curves and survival analysis for ACLF mortality prediction in the training cohort.

Survival curves for 28-day (A) and 90-day (B) mortality according to the number of complications at baseline. (C) Cut-off values, sensitivity, and specificity of the prognostic scores according to ROC analysis. Cumulative survival at 28 days (D) and 90 days (E) according to the Kaplan–Meier method. ALBI, albumin-bilirubin; ACLF, acute-on-chronic liver failure; MELD, Model for end-stage liver disease; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operating characteristic.

We also assessed the performance of the ALBI score, MELD score, MELD 3.0 score, and MELD-complication score in predicting 90-day mortality via ROC analysis. The AUROCs (95% CIs) of the four scores were 0.678 (0.650–0.707) for ALBI, 0.648 (0.617–0.678) for MELD, 0.662 (0.633–0.692) for MELD 3.0, and 0.853 (0.832–0.873) for MELD-complication, (Fig. 3B, C). These results indicate that the MELD-complication score also has predictive ability for 90-day mortality compared to the ALBI score and the other two MELD scores.

Cut-off values, sensitivity, and specificity of the prognostic scores in predicting ACLF patient mortality

In addition to the AUROC, additional accuracy metrics were calculated for the scores, as shown in Figure 3C. The ALBI score, MELD score, and MELD 3.0 score demonstrated sensitivities of 62.90%, 62.08%, and 64.01%, respectively, while the MELD-complication score exhibited a sensitivity of 77.89% and a specificity of 77.81%. The critical value of 2.58 for the MELD-complication score, determined using the maximum Youden index method, showed that above this threshold, the 28-day mortality rate was 60.42%, whereas below this threshold, the mortality rate was 11.07%. Survival rates for 28-day and 90-day periods were also analyzed using the Kaplan–Meier method according to this cut-off value. The results revealed a significant difference in survival between patients dichotomized by this threshold (log-rank (Mantel–Cox) χ2 = 487.574 and 606.441, p = 0.000) (Fig. 3D–E).

Model performance and validation

The new model was validated in two independent external cohorts: 292 patients from Beijing Youan Hospital, Capital Medical University (Youan cohort), and 242 patients from The First Affiliated Hospital, Zhejiang University School of Medicine (Hangzhou cohort).

Statistical analysis revealed significant differences between the training set and external test set in terms of age, sex, etiology, ACLF subtype, incidence of complications, and multiple laboratory indicators (all p-values < 0.05) (Table 3). These differences highlight the heterogeneity in the ACLF patient population across different medical institutions, and further validate the universality and stability of the new model.

Table 3

Comparison of clinical characteristics between the training and test sets

Training set (n = 1,386)Youan set (n = 292)Hangzhou set (n = 242)p-value
Age (years), mean (SD)46.68 ± 9.6548.12 ± 11.0154.32 ± 12.6556.9400.000a
Male, n (%)1,170 (85.57)239 (81.61)17915.9050.000b
Aetiology of disease7.4970.023c
  HBV, n (%)1,143 (82.47)202 (69.18)165 (68.18)
  Alcohol, n (%)131 (9.45)70 (23.97)39 (16.12)
  Other, n (%)112 (8.08)20 (6.85)38 (15.70)
ACLF subtype8.5320.031c
  Subtype A, n (%)475 (34.27)71 (24.32)79 (32.64)
  Subtype B, n (%)766 (55.27)140 (47.94)115 (47.52)
  Subtype C, n (%)145 (10.46)81 (27.74)48 (19.83)
Complications
  Ascites, n (%)1,055 (76.12)230 (78.77)134 (55.37)286.8120.000c
  Variceal bleeding, n (%)164 (11.83)31 (10.62)21 (8.68)2.1920.334c
Hepatic encephalopathy123.4750.000c
  No, n (%)798 (57.58)186 (63.70)197 (81.40)
   Grade I, n (%)492 (35.51)80 (27.40)13 (5.37)
  Grade II, n (%)103 (7.43)20 (6.85)13 (5.37)
  Grade III-IV, n (%)33 (2.38)6 (2.05)19 (7.85)
  SBP, n (%)449 (32.40)181 (61.99)68 (28.10)99.4290.000c
  Pulmonary infection, n (%)272 (19.62)54 (18.49)22 (9.09)15.6050.000c
  AKI, n (%)614 (44.30)36 (12.33)55 (22.73)129.4290.000c
Laboratory data
  Albumin (g/L), mean (SD)29.31 ± 5.5329.80 ± 5.3129.91 ± 4.561.9290.146a
  Total bilirubin (µmol/L), Median (IQR)235.40 (107.98,433.20)333.30 (204.40,473.83)308.45 (235.68,411.60)42.0920.000b
  INR, mean (SD)2.08 ± 0.602.23 ± 0.772.10 ± 0.646.8670.001a
  ALT, median (IQR)192.00 (78.00,338.00)118.00 (45.00,456.50)132.50 (54.25,423.75)148.6960.000b
  AST, median (IQR)100.00 (169.00,352.50)151.00 (84.00,321.50)147.50 (80.00,322.25)4.6030.100b
  Sodium (mmol/L), mean (SD)133.16 ± 5.91135.95 ± 7.61134.21 ± 6.3382.8770.000
  Ammonia (µmol/L), mean (SD)106.18 ± 52.6085.17 ± 49.30/36.4840.000a
  WBC (109/L), median (IQR)5.59 (3.57, 9.82)7.12 (5.21, 10.14)7.00 (5.00,10.00)16.6130.000b
  Neutrophil count (109/L), median (IQR)5.15 (1.70,7.30)5.34 (3.91,7.61)5.00 (3.00,7.75)68.3320.000b
  Platelets (109/L), median (IQR)81.00 (52.00,120.00)101.00 (58.50,145.00)89.50 (55.00,137.50)21.5210.000b
  Creatinine (µmol/L), median (IQR)87.00 (72.00,123.00)63.00 (53.00,77.50)66.00 (56.00,93.75)179.7450.000b
  MELD score, median (IQR)25.46 ± 5.5536.92 ± 2.7722.41 ± 5.99654.5680.000a
  MELD 3.0 score, median (IQR)28.10 ± 5.4135.22 ± 3.5540.14 ± 3.00753.9850.000a
  ALBIs, median (IQR)−0.99 (−1.34, −0.69)−0.85 (−1.18, −0.58)−0.91 (−1.17, −0.66)8.9950.011b
  28-day mortality (%)414 (29.87)67 (22.95)76 (31.40)6.3870.041c

Using ROC analysis, the MELD-complication score outperformed other models in predicting 28-day and 90-day mortality rates in both external test cohorts (Fig. 4). The DeLong test confirmed statistical superiority with p < 0.05 (Table 4). In the Youan cohort, the AUROCs for 28-day mortality were 0.696 for ALBI, 0.770 for MELD, 0.757 for MELD 3.0, and 0.820 for MELD-complication. For 90-day mortality, AUROCs were 0.693, 0.771, 0.759, and 0.800, respectively. In the Hangzhou cohort, the corresponding 28-day AUROCs were 0.602 for ALBI, 0.662 for MELD, 0.646 for MELD 3.0, and 0.774 for MELD-complication (Fig. 4A–B, Table 4). The 90-day AUROCs were 0.595, 0.661, 0.619, and 0.724, respectively, all indicating the MELD-complication score’s predictive advantage (Fig. 4C–D, Table 4).

ROC curves of different prognostic models in predicting 28-day and 90-day mortality in patients with ACLF.
Fig. 4  ROC curves of different prognostic models in predicting 28-day and 90-day mortality in patients with ACLF.

(A–B) ROC curves for the prediction of (A) 28-day and (B) 90-day mortality in the Youan cohort. (C–D) ROC curves for the prediction of (C) 28-day and (D) 90-day mortality in the Hangzhou cohort. ALBI, albumin-bilirubin; ACLF, acute-on-chronic liver failure; MELD, Model for end-stage liver disease; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operating characteristic.

Table 4

AUROCs of the prognostic scores in predicting 28-day and 90-day mortality of patients with ACLF in the external test sets

28-day mortality
90-day mortality
AUROC (95% CI)p-valueAUROC (95% CI)p-value
Youan
  ALBI score0.696 (0.623–0.770)0.0000.693 (0.626–0.759)0.000
  MELD score0.770 (0.704–0.836)0.0000.771 (0.710–0.832)0.000
  MELD 3.0 score0.757 (0.690–0.824)0.0000.759 (0.699–0.819)0.000
  MELD-complication score0.820 (0.762–0.879)0.0000.800 (0.744–0.857)0.000
Hangzhou
  ALBI score0.602 (0.530–0.674)0.0070.595 (0.522–0.667)0.012
  MELD score0.662 (0.593–0.731)0.0000.661 (0.592–0.730)0.000
  MELD 3.0 score0.646 (0.574–0.719)0.0000.619 (0.546–0.693)0.002
  MELD-complication score0.774 (0.714–0.833)0.0000.724 (0.659–0.788)0.000

Discussion

This retrospective study aimed to identify mortality risk factors in patients with ACLF and to develop a prognostic model, termed the MELD-complication score. By analyzing extensive clinical data, we identified age, ACLF subtype, INR, MELD score, HE grade, SBP, and white blood cell count as independent predictors of 28-day mortality. Our survival analysis showed that higher numbers of complications were significantly associated with lower survival rates. The non-survivor group had a notably greater proportion of patients with three or more complications [log-rank (Mantel–Cox) χ2 = 143.265 and 211.153, respectively, p = 0.000], highlighting the critical relationship between complications and prognosis.

The MELD-complication score integrates laboratory values and essential clinical complications, thereby enhancing the prediction of short-term outcomes for ACLF patients. It serves as a valuable decision-support tool for clinicians managing these patients and prioritizing liver transplants.

We compared the prognostic value of the new MELD-complication score with the ALBI, MELD, and MELD 3.0 scores in ACLF patients. The ALBI score, recently developed to assess the prognosis of hepatocellular carcinoma, detects small liver function changes more objectively than the Child–Pugh or MELD scores and predicts survival in non-malignant liver disease.15 In an HBV-ACLF study, the ACLF group had a significantly higher ALBI score compared to healthy controls and CHB groups, correlating positively with MELD and Child–Pugh scores.24 However, another study found the ALBI score ineffective in predicting in-hospital mortality in cirrhotic ACLF patients (AUROC: 0.53, 95% CI: 0.42–0.63, p = 0.69).25 Sun et al. found no association between ALBI or ALBI grade and rehospitalization in ACLF patients.26 Our multivariate analysis revealed that the ALBI score was not an independent risk factor for 28-day mortality; its AUROC values for predicting 28-day mortality and 90-day mortality were 0.675 (0.645–0.705) and 0.678 (0.650–0.707), respectively. In this cohort of ACLF patients, the majority were classified as ALBI grade 3 (80.09%) or ALBI grade 2 (19.84%), with only a minority having grade 1 (0.07%), which is consistent with the results of a previous study.26 In summary, while the ALBI score has certain advantages in evaluating liver function, its efficacy in predicting the prognosis of ACLF patients remains insufficient. The MELD 3.0 score, though achieving better accuracy in predicting mortality in liver diseases, was not superior to the MELD score in predicting short-term survival in ACLF patients (AUROCs: 0.697 and 0.662 for 28-day and 90-day mortality, respectively). This suggests the need for more accurate and effective tools to evaluate ACLF prognosis.

Through comparative analysis, we expanded our understanding of ACLF’s clinical characteristics and developed the MELD-complication score for short-term prognosis. Key risk factors include HE, pulmonary infection, SBP, INR, and neutrophil count.27–30 Additionally, infections are now considered a major trigger for encephalopathy.31 This information suggests that bacterial infection is a crucial component of ACLF and that systemic inflammatory response syndrome is the most important factor for assessing the severity of liver failure.32,33 Some propose that intense systemic inflammation may lead to organ failure through various mechanisms, including neutrophil activation and tissue damage.34 Despite the lack of a universal ACLF definition for ACLF, HE is recognized as a significant form of organ failure and a key marker of ACLF. Acute systemic inflammation stimulates intense amino acid catabolism, potentially contributing to HE, which in turn predicts ACLF patient mortality.35 By analyzing the pathophysiological characteristics of ACLF, we developed a new scoring system that integrates the MELD score with important ACLF complications (HE, SBP, and pulmonary infection), INR, and neutrophil count. This system reflects the impact of systemic inflammation on the prognosis of ACLF patients. Regarding clinical application, a prognostic model should be evaluated for its utility in a homogeneous patient cohort in terms of discrimination, calibration ability, and predictive/diagnostic power.11

The results of the Hosmer–Lemeshow goodness-of-fit test suggested that the prediction model had good calibration ability. The clinical features selected for inclusion indicate that the new scoring model, which incorporates ACLF complications and the MELD score, is effective for accurately assessing the prognosis of patients with ACLF according to the APASL criteria. The AUROCs revealed that the MELD-complication score, is more accurate in assessing LT-free short-term mortality compared to generic scoring models, including the MELD score, MELD 3.0 score, and ALBI score. The MELD-complication score can be used to stratify mortality risk among ACLF patients, effectively identify high-risk individuals, and demonstrate robustness in two external cohorts, with distinct ACLF etiologies.

Finally, we validated the MELD-complication score in two independent external cohorts from Beijing Youan Hospital, Capital Medical University, and the First Affiliated Hospital of Zhejiang University. All three hospitals are of the same level, and homogeneous in medical standards. By comparing the clinical characteristics of the test set and the training set, we found some differences in patient age, etiology distribution, and incidence of complications. However, despite these differences, the MELD-complication score accurately predicted patient prognosis. The AUROCs for predicting 28-day mortality were 0.820 and 0.774 (all p < 0.05), and for predicting 90-day mortality were 0.800 and 0.724 (all p < 0.05). This finding indicates that the new model maintains accuracy and stability across various patient groups and medical environments.

This study has certain limitations. The retrospective design may introduce biases, highlighting the need for future prospective studies for validation. Additionally, the relatively small sample size in external cohorts suggests a need for enhanced model robustness. Future research should focus on larger cohorts and long-term prognostic assessments to adapt the model as ACLF treatment strategies evolve.

Conclusions

This retrospective study has identified key risk factors associated with the prognosis of patients with ACLF and developed a novel prognostic scoring model, the MELD-complication score. This new model, which integrates the MELD score with significant complications and indicators of systemic inflammatory response, demonstrated superior predictive accuracy for short-term mortality compared to traditional scores such as MELD, MELD 3.0, and ALBI. The MELD-complication score offers improved predictions of short-term prognosis for ACLF patients, potentially aiding in patient management and liver transplant prioritization. Future validation across diverse patient populations is warranted to ensure the model’s generalizability and reliability.

Declarations

Acknowledgement

We express our profound gratitude to all participants in the study, including the patients and their families, as well as the dedicated researchers.

Ethical statement

This study was approved by the Ethics Committee of the 302 Hospital of PLA, the predecessor of the Fifth Medical Center of the Chinese PLA General Hospital, in accordance with the Declaration of Helsinki (2013005D).

Data sharing statement

All data associated with this study are included in the paper.

Funding

None to declare.

Conflict of interest

JL has been an Editorial Board Member of Journal of Clinical and Translational Hepatology since 2020. The other authors have no conflict of interests related to this publication.

Authors’ contributions

Collecting data, performing the statistical analysis, drafting the final manuscript (WL, WL, YR), helping to collect data and perform statistical analysis (DL, BZ, SY, SS), conceptualizing and designing the study, collecting data, and revisinge the final manuscript (SY, YC, JL). All authors were involved in the critical revision and approved the final version of the manuscript.

References

  1. Katoonizadeh A, Laleman W, Verslype C, Wilmer A, Maleux G, Roskams T, et al. Early features of acute-on-chronic alcoholic liver failure: a prospective cohort study. Gut 2010;59(11):1561-1569 View Article PubMed/NCBI
  2. Jalan R, Perricone G, Moreau R, Arroyo V, Williams R. Acute-on-Chronic Liver Failure: A New Disease or an Old One Hiding in Plain Sight?. Clin Liver Dis (Hoboken) 2020;15(Suppl 1):S45-S51 View Article PubMed/NCBI
  3. Gülcicegi DE, Goeser T, Kasper P. Prognostic assessment of liver cirrhosis and its complications: current concepts and future perspectives. Front Med (Lausanne) 2023;10:1268102 View Article PubMed/NCBI
  4. Trebicka J, Fernandez J, Papp M, Caraceni P, Laleman W, Gambino C, et al. PREDICT identifies precipitating events associated with the clinical course of acutely decompensated cirrhosis. J Hepatol 2021;74(5):1097-1108 View Article PubMed/NCBI
  5. Kjaergard LL, Liu J, Als-Nielsen B, Gluud C. Artificial and bioartificial support systems for acute and acute-on-chronic liver failure: a systematic review. JAMA 2003;289(2):217-222 View Article PubMed/NCBI
  6. European Association for the Study of the Liver. EASL Clinical Practice Guidelines for the management of patients with decompensated cirrhosis. J Hepatol 2018;69(2):406-460 View Article PubMed/NCBI
  7. Karvellas CJ, Garcia-Lopez E, Fernandez J, Saliba F, Sy E, Jalan R, et al. Dynamic Prognostication in Critically Ill Cirrhotic Patients With Multiorgan Failure in ICUs in Europe and North America: A Multicenter Analysis. Crit Care Med 2018;46(11):1783-1791 View Article PubMed/NCBI
  8. Rosenblatt R, Shen N, Tafesh Z, Cohen-Mekelburg S, Crawford CV, Kumar S, et al. The North American Consortium for the Study of End-Stage Liver Disease-Acute-on-Chronic Liver Failure Score Accurately Predicts Survival: An External Validation Using a National Cohort. Liver Transpl 2020;26(2):187-195 View Article PubMed/NCBI
  9. Sarin SK, Choudhury A, Sharma MK, Maiwall R, Al Mahtab M, Rahman S, et al. Correction to: Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific association for the study of the liver (APASL): an update. Hepatol Int 2019;13(6):826-828 View Article PubMed/NCBI
  10. Choudhury A, Jindal A, Maiwall R, Sharma MK, Sharma BC, Pamecha V, et al. Liver failure determines the outcome in patients of acute-on-chronic liver failure (ACLF): comparison of APASL ACLF research consortium (AARC) and CLIF-SOFA models. Hepatol Int 2017;11(5):461-471 View Article PubMed/NCBI
  11. Verma N, Dhiman RK, Singh V, Duseja A, Taneja S, Choudhury A, et al. Comparative accuracy of prognostic models for short-term mortality in acute-on-chronic liver failure patients: CAP-ACLF. Hepatol Int 2021;15(3):753-765 View Article PubMed/NCBI
  12. Wiesner R, Edwards E, Freeman R, Harper A, Kim R, Kamath P, et al. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology 2003;124(1):91-96 View Article PubMed/NCBI
  13. Wlodzimirow KA, Eslami S, Abu-Hanna A, Nieuwoudt M, Chamuleau RA. A systematic review on prognostic indicators of acute on chronic liver failure and their predictive value for mortality. Liver Int 2013;33(1):40-52 View Article PubMed/NCBI
  14. Kim WR, Mannalithara A, Heimbach JK, Kamath PS, Asrani SK, Biggins SW, et al. MELD 3.0: The Model for End-Stage Liver Disease Updated for the Modern Era. Gastroenterology 2021;161(6):1887-1895.e4 View Article PubMed/NCBI
  15. Toyoda H, Johnson PJ. The ALBI score: From liver function in patients with HCC to a general measure of liver function. JHEP Rep 2022;4(10):100557 View Article PubMed/NCBI
  16. Weng WZ, Chen JF, Peng XH, Huang M, Zhang J, Xiong J, et al. Risk factors for underlying comorbidities and complications in patients with hepatitis B virus-related acute-on-chronic liver failure. Epidemiol Infect 2022;150:e147 View Article PubMed/NCBI
  17. Guo HY, Zhang ZG, Zhang YY, Mei X, Liu Y, Wang JF, et al. Risks and predicting factors of bleeding complications in hepatitis B virus-related acute-on-chronic liver failure. Turk J Gastroenterol 2020;31(9):620-625 View Article PubMed/NCBI
  18. Zang H, Liu F, Liu H, You S, Zhu B, Wan Z, et al. Incidence, risk factors and outcomes of acute kidney injury (AKI) in patients with acute-on-chronic liver failure (ACLF) of underlying cirrhosis. Hepatol Int 2016;10(5):807-818 View Article PubMed/NCBI
  19. O’Leary JG, Rajender Reddy K, Tandon P, Biggins SW, Wong F, Kamath PS, et al. Increased Risk of ACLF and Inpatient Mortality in Hospitalized Patients with Cirrhosis and Hepatic Hydrothorax. Dig Dis Sci 2021;66(10):3612-3618 View Article PubMed/NCBI
  20. Napoleone L, Solé C, Juanola A, Ma AT, Carol M, Pérez-Guasch M, et al. Patterns of kidney dysfunction in acute-on-chronic liver failure: Relationship with kidney and patients’ outcome. Hepatol Commun 2022;6(8):2121-2131 View Article PubMed/NCBI
  21. Angeli P, Ginès P, Wong F, Bernardi M, Boyer TD, Gerbes A, et al. Diagnosis and management of acute kidney injury in patients with cirrhosis: revised consensus recommendations of the International Club of Ascites. J Hepatol 2015;62(4):968-974 View Article PubMed/NCBI
  22. Ginès P, Krag A, Abraldes JG, Solà E, Fabrellas N, Kamath PS. Liver cirrhosis. Lancet 2021;398(10308):1359-1376 View Article PubMed/NCBI
  23. Johnson PJ, Berhane S, Kagebayashi C, Satomura S, Teng M, Reeves HL, et al. Assessment of liver function in patients with hepatocellular carcinoma: a new evidence-based approach-the ALBI grade. J Clin Oncol 2015;33(6):550-558 View Article PubMed/NCBI
  24. Chen B, Lin S. Albumin-bilirubin (ALBI) score at admission predicts possible outcomes in patients with acute-on-chronic liver failure. Medicine (Baltimore) 2017;96(24):e7142 View Article PubMed/NCBI
  25. Peng Y, Qi X, Tang S, Deng H, Li J, Ning Z, et al. Child-Pugh, MELD, and ALBI scores for predicting the in-hospital mortality in cirrhotic patients with acute-on-chronic liver failure. Expert Rev Gastroenterol Hepatol 2016;10(8):971-980 View Article PubMed/NCBI
  26. Sun Y, Li Z, Liao G, Xia M, Xu X, Cai S, et al. aMAP Score as a Predictor for Long-Term Outcomes in Patients with HBV-Related Acute-on-Chronic Liver Failure. Int J Gen Med 2022;15:407-415 View Article PubMed/NCBI
  27. Arroyo V, Moreau R, Jalan R. Acute-on-Chronic Liver Failure. N Engl J Med 2020;382(22):2137-2145 View Article PubMed/NCBI
  28. Arroyo V, Moreau R, Kamath PS, Jalan R, Ginès P, Nevens F, et al. Acute-on-chronic liver failure in cirrhosis. Nat Rev Dis Primers 2016;2:16041 View Article PubMed/NCBI
  29. Moreau R, Jalan R, Gines P, Pavesi M, Angeli P, Cordoba J, et al. Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis. Gastroenterology 2013;144(7):1426-1437.e9 View Article PubMed/NCBI
  30. Weiss E, de la Grange P, Defaye M, Lozano JJ, Aguilar F, Hegde P, et al. Characterization of Blood Immune Cells in Patients With Decompensated Cirrhosis Including ACLF. Front Immunol 2021;11:619039 View Article PubMed/NCBI
  31. Badal BD, Bajaj JS. Hepatic Encephalopathy in Acute-on-Chronic Liver Failure. Clin Liver Dis 2023;27(3):691-702 View Article PubMed/NCBI
  32. Wong F, Piano S, Singh V, Bartoletti M, Maiwall R, Alessandria C, et al. Clinical features and evolution of bacterial infection-related acute-on-chronic liver failure. J Hepatol 2021;74(2):330-339 View Article PubMed/NCBI
  33. Mezzano G, Juanola A, Cardenas A, Mezey E, Hamilton JP, Pose E, et al. Global burden of disease: acute-on-chronic liver failure, a systematic review and meta-analysis. Gut 2022;71(1):148-155 View Article PubMed/NCBI
  34. Clària J, Arroyo V, Moreau R. Roles of systemic inflammatory and metabolic responses in the pathophysiology of acute-on-chronic liver failure. JHEP Rep 2023;5(9):100807 View Article PubMed/NCBI
  35. Verma N, Dhiman RK, Choudhury A, Taneja S, Duseja A, Singh V, et al. Dynamic assessments of hepatic encephalopathy and ammonia levels predict mortality in acute-on-chronic liver failure. Hepatol Int 2021;15(4):970-982 View Article PubMed/NCBI