Huang, Guoxi and Bors, Adrian Gheorghe orcid.org/0000-0001-7838-0021 (2022) BQN: Busy-Quiet Net Enabled by Motion Band-Pass Module for Action Recognition. IEEE Transactions on Image Processing. pp. 4966-4979. ISSN 1057-7149
Abstract
A rich video data representation can be realized by means of spatio-temporal frequency analysis. In this research study we show that a video can be disentangled, following the learning of video characteristics according to their spatiotemporal properties, into two complementary information components, dubbed Busy and Quiet. The Busy information characterizes the boundaries of moving regions, moving objects, or regions of change in movement. Meanwhile, the Quiet information encodes global smooth spatio-temporal structures defined by substantial redundancy. We design a trainable Motion Band-Pass Module (MBPM) for separating Busy and Quiet-defined information, in raw video data. We model a Busy-Quiet Net (BQN) by embedding the MBPM into a two-pathway CNN architecture. The efficiency of BQN is determined by avoiding redundancy in the feature spaces defined by the two pathways. While one pathway processes the Busy features, the other processes Quiet features at lower spatio-temporal resolutions reducing both memory and computational costs. Through experiments we show that the proposed MBPM can be used as a plug-in module in various CNN backbone architectures, significantly boosting their performance. The proposed BQN is shown to outperform many recent video models on Something-Something V1, Kinetics400, UCF101 and HMDB51 datasets. The code for the implementation is available1
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 IEEE. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Funding Information: | Funder Grant number EPSRC EP/V009591/1 |
Depositing User: | Pure (York) |
Date Deposited: | 08 Aug 2022 11:20 |
Last Modified: | 18 Dec 2024 00:22 |
Published Version: | https://doi.org/10.1109/TIP.2022.3189810 |
Status: | Published |
Refereed: | Yes |
Identification Number: | 10.1109/TIP.2022.3189810 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:189725 |