Loss-attentional physics-informed neural networks

Physics-informed neural networks (PINNs) have emerged as a significant endeavour in recent years to utilize artificial intelligence technology for solving various partial differential equations (PDEs). Nevertheless, the vanilla PINN model structure encounters challenges in accurately approximating solutions at hard-to-fit regions with, for instance, “stiffness” points characterized by fast-paced alterations in timescale. To this end, we introduce a novel model architecture based on PINN, named loss-attentional physics-informed neural networks (LA-PINN), which equips each loss component with an independent loss-attentional network (LAN). Feeding the squared errors (SE) on every training point into LAN as the input, the attentional function is then built by each LAN and provides different weights to diverse point SEs. A point error-based weighting approach that utilizes the adversarial training between multiple networks in the LA-PINN model is proposed to dynamically update weights of SE during every training epoch. Additionally, the weighting mechanism of LA-PINN is analysed and also be validated by performing several numerical experiments. The experimental results indicate that the proposed method displays superior predictive performance compared to the vanilla PINN and holds a swift convergence characteristic. Moreover, it can advance the convergence of those hard-to-fit points by progressively increasing the growth rates of both the weight and the update gradient for point error.