Imbalance-Aware AutoML for Financial Risk Prediction
Abstract
Class imbalance is a persistent challenge in financial and business risk prediction because
the minority class often represents the cases of greatest practical interest, such as fraud,
default, or customer attrition. This study investigates where imbalance should be handled
inside an automated machine-learning (AutoML) pipeline. Seven strategies are compared:
an untreated baseline, SMOTE oversampling, an average-precision training objective,
balanced class weighting, cost-sensitive decision-threshold tuning, a combined objective
and-threshold strategy, and a full combination of SMOTE, objective optimization, and
threshold tuning. The supplied experiment evaluates these strategies on eight datasets
spanning approximately 0.17% to 44.5% minority prevalence, across ten random seeds
and two engines, XGBoost-GPU and FLAML, for a total of 1,120 runs.
The experimental design uses stratified 60/20/20 train-validation-test splits, leakage
aware preprocessing, probability-based evaluation, and an asymmetric cost function in
which a false negative costs five times a false positive. Nine evaluation measures are
computed, with PR-AUC, recall, F1, G-mean, MCC, and total cost emphasized. For
statistical comparison, the study ranks strategies within dataset-seed blocks and applies
the Friedman test, with a critical-difference analysis.
The results show that imbalance handling primarily improves the decision process rather
than the ranking of observations. On XGBoost-GPU, the threshold strategy achieves the
best average rank, followed by class weighting and the combined objective-plus-threshold
strategy. On FLAML, the combined objective-plus-threshold strategy ranks first,
followed by threshold tuning and class weighting. Across both engines, these three
strategies form a consistent top group, whereas the untreated baseline and objective-only
strategy are the weakest. The study therefore supports decision-level cost-sensitive
thresholding as a particularly effective and robust component of imbalance-aware
AutoML, while also showing that the preferred exact strategy can depend on the AutoML
engine.
Collections
- 2026-2030 [51]