On the use of shap for measuring variable importance in modelling nonlinear complex systems with multiple input data

Model explanation is an important part of model development for interpreting the reasoning and decisions of complex models, especially black-box artificial intelligence (AI) and machine learning (ML) models, as it enables a better understanding not only of what the developed models can do but also of how they do it. To meet increasingly demanding interpretability requirements, the SHapley Additive exPlanations (SHAP) method is one of the most employed post-hoc model explanation techniques. Despite its utility and popularity, SHAP faces challenges in modelling nonlinear systems with correlated inputs, as it is prone to assigning high importance to strongly correlated variables, resulting in redundant variable rankings and causing inherently important features to be underestimated or omitted from top-priority lists. This study addresses these limitations by conducting a rigorous analysis using both simulation and real-life data.