Frequency-aware SVD decomposition and its application to color magnification and motion denoising

Videos are full of dynamic changes along both the spatial and temporal dimensions. Large, jerky short-term motions make it difficult to extract significant changes from videos such as subtle color changes and long-term motions occurring in time-lapse sequences. In this paper, we introduce two singular value decomposition (SVD)-based video decomposition schemes to clearly reveal such changes. The first scheme involves enhancing the visual characteristics of small subtle color changes in the presence of a wide variety of motion patterns by magnifying their pixel intensities. The second scheme removes short-term motions that visually distract attention from the underlying content of video sequences such as time-lapse videos, snowing scene, and maritime surveillance. Both schemes involve the decomposition of videos into spatiotemporal slices in which each slice is further decomposed into several singular components. The low-rank components that primarily represent background and color intensity information are then temporally processed to magnify the magnitude of the signal at the subtle color change target frequency. At the same time, an approach similar to that used in denoising time-lapse sequences is applied to temporally filter the singular components representing sparse information, thereby removing jittery short-term motions while preserving long-term motions, which are represented by both low-rank and unfiltered sparse components. We demonstrate promising color magnification and motion denoising results that can be obtained much faster than results estimated using state-of-the-art techniques. © 2021 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.