Bayesian Processing of Big Data using Log Homotopy Based Particle Flow Filters

Bayesian recursive estimation using large volumes of data is a challenging research topic. The problem becomes particularly complex for high dimensional non-linear state spaces. Markov chain Monte Carlo (MCMC) based methods have been successfully used to solve such problems. The main issue when employing MCMC is the evaluation of the likelihood function at every iteration, which can become prohibitively expensive to compute. Alternative methods are therefore sought after to overcome this difficulty. One such method is the adaptive sequential MCMC (ASMCMC), where the use of the confidence sampling is proposed as a method to reduce the computational cost. The main idea is to make use of the concentration inequalities to sub-sample the measurements for which the likelihood terms are evaluated. However, ASMCMC methods require appropriate proposal distributions. In this work, we propose a novel ASMCMC framework in which log-homotopy based particle flow filters form adaptive proposals. We show the performance can be significantly enhanced by our proposed algorithm, while still maintaining a comparatively low processing overhead.