SMURF: Continuous Dynamics for
Motion-Deblurring Radiance Fields

Jungho Lee    Dogyoon Lee    Minhyeok Lee    DongHyeong Kim    Sangyoun Lee†
Yonsei University    
† Corresponding author
CVPR 2025 - 2nd Workshop on Neural Fields Beyond Conventional Cameras
arXiv Full Paper Code Datasets
PDF Thumbnail

A blurry image $I_{blur}$ is acquired as the camera moves over the exposure time ($t_{0}\sim t_{\mathcal{N}}$), with images $I_{t}$ captured at each camera pose being composited together. Our approach iteratively estimates warped rays along the sequential camera motion trajectory.

Abstract


Neural radiance fields (NeRF) has attracted considerable attention for their exceptional ability in synthesizing novel views with high fidelity. However, the presence of motion blur, resulting from slight camera movements during extended shutter exposures, poses a significant challenge, potentially compromising the quality of the reconstructed 3D scenes. While recent studies have addressed this issue, they do not consider the continuous dynamics of camera movements during image acquisition, leading to inaccurate scene reconstruction. To effectively handle this issue, we propose sequential motion understanding radiance fields (SMURF), a novel approach that employs neural ordinary differential equations (Neural-ODEs) to model continuous camera motion and leverages the explicit volumetric representation method for robustness to motion-blurred input images. The core idea of the SMURF is continuous motion blurring kernel (CMBK), a unique module designed to model a continuous camera movements for processing blurry inputs. Our model, rigorously evaluated against benchmark datasets, demonstrates state-of-the-art performance both quantitatively and qualitatively.

Method


Architecture

The CMBK encoder $\mathcal{E}_{\theta}$ transforms the embedded 2D pixel location $\mathbf{p}$ of the initial ray and view index into a latent feature. This feature is extended into an IVP with a parameterized derivative function $f_{\phi}$ in the latent space. Then, it is solved by Neural-ODEs along with given time $t$ and a chrono-view embedding function $\mathbf{\Psi}$, obtaining latent features for all warped rays. These features are transformed into changes of the ray through a decoder $\mathcal{D}_{\xi}$, and we get the warped rays by applying them to the initial ray. These rays are rendered into 2D pixel colors through a 3D grid-based method, and a blurry color is acquired by summing up the colors with weights from the decoder.

Architecture

We assume that camera motion encompasses inherent dynamics with a unique solution. The assumed solution is represented by the lighter circles in the left plots of (b). Rather than implementing the inherent dynamics in a simple physical space, we refine them within a latent space with parametric learning. The continuous dynamics of CMBK involve transforming the pixel coordinates corresponding to the ray into latent features via a parameterized encoder $\mathcal{E}_{\theta}$, and a unique numerical solution is obtained by solving the initial value problem on the latent space through a neural ordinary differential equation.

Comparison with State-of-the-Arts


Please click the videos for better view.

BibTeX

@article{lee2024smurf,
      title={SMURF: Continuous Dynamics for Motion-Deblurring Radiance Fields},
      author={Lee, Jungho and Lee, Dogyoon and Lee, Minhyeok and Kim, Donghyung and Lee, Sangyoun},
      journal={arXiv preprint arXiv:2403.07547},
      year={2024}
    }