Differentiable Learning of Sparse Lossless Fourier Filters

Purpose: The Fourier Transform bounded Kolmogorov Complexity proposed in M. Terry-Jack, J. Softw. Eng. Appl. 15, 359 (2022), and M. Terry-Jack and S. O'Keefe, Physica D 453, 133824 (2023), relies upon finding the minimal sub-set of frequency coefficients that can exactly reconstruct the original image. However, we show that finding this minimal subset of coefficients is a combinatorial optimisation problem that becomes computationally intractable for large images. As such, naive solutions like greedy search are impractical at scale, limiting the feasibility and application of Fourier Transform bounded Kolmogorov Complexity. Methods: We reformulate the hard combinatorial problem as a differentiable optimisation task via the introduction of smooth surrogates for non-differentiable components such as the quantisation step and use a straight-through estimator (STE) to learn sparse binary masks (frequency filters) that allow for exact reconstruction of the original image via gradient-descent. Results: Experiments on Elementary Cellular Automata, QR codes, natural binary images, and random noise demonstrate that sparse Fourier masks capable of bit-exact reconstruction can be learned efficiently at scales that are impractical for a combinatorial or Greedy search. Although the problem admits a linear LASSO reformulation that enables classical sparse-recovery methods, experiments against a FISTA baseline demonstrate the advantages of directly optimising the original nonlinear objective via STE. Conclusion: This work provides the first practical framework for learning loss- less Fourier-domain filters at scale by reframing a hard combinatorial problem as a differentiable learning task.