Improving the fairness and robustness of deep learning classification models through bias mitigation and post hoc calibration

Purpose: Thegoal of this study was to enhance the fairness and clinical applicability of deep learning models for classifying Alzheimer’s disease from structural MRI scans. To achieve this, we addressed two key challenges: age-related bias and unreliable probability estimates. Specifically, we investigated in-processing debiasing strategies to achieve age-invariant predictions while preserving diagnostic accuracy. We also systematically evaluated post hoc calibration methods to improve the reliability, interpretability, and clinical usefulness of predicted probabilities across diverse patient populations.

Methods: Structural MRI data from three independent cohorts (ADNI, AIBL, and DELCODE) were preprocessed using a standardized pipeline. Age-related bias was addressed using two in-processing strategies, adversarial training (CF-Net) and GLM-based feature residualization (PMDN), implemented in a shared 3D convolutional neural network (CNN) framework. Additionally, post hoc calibration was performed using four regression methods: logistic, probit, isotonic, and Bayesian binning. Weevaluated modelperformance,fairness, and calibration using classification as well as calibration-specific metrics.

Results: PMDNachievedthebestoverallperformance,withhigherbalancedaccuracy(0.71)andimprovedcalibration(Brier ≈0.22), showing substantially reduced variability compared to baseline and CF-Net. PMDN effectively removed age-related bias, achieving near-zero age prediction correlation; CF-Net, however, showed only partial and unstable debiasing. Post hoc calibration further improved probability reliability, with nonparametric methods (isotonic regression and Bayesian binning) yielding the lowest calibration errors (MCE ≈0.21−0.24).

Conclusion: This study shows that, although deep learning models achieve similar classification performance, they differ significantly in terms of calibration, stability, and fairness. PMDN provides more consistent and effective age bias mitigation than the CF-Net approach, particularly in clinically relevant subgroups. Post hoc calibration improves probability reliability further, with nonparametric methods achieving the best calibration.