CoCoGaussian: Leveraging Circle of Confusion for Gaussian Splatting from Defocused Images

3D Gaussian Splatting (3DGS) has attracted significant attention for its high-quality novel view rendering, inspiring research to address real-world challenges. While conventional methods depend on sharp images for accurate scene reconstruction, real-world scenarios are often affected by defocus blur due to finite depth of field, making it essential to account for realistic 3D scene representation. In this study, we propose CoCoGaussian, a Circle of Confusion-aware Gaussian Splatting that enables precise 3D scene representation using only defocused images. CoCoGaussian addresses the challenge of defocus blur by modeling the Circle of Confusion (CoC) through a physically grounded approach based on the principles of photographic defocus. Exploiting 3D Gaussians, we compute the CoC diameter from depth and learnable aperture information, generating multiple Gaussians to precisely capture the CoC shape. Furthermore, we introduce a learnable scaling factor to enhance robustness and provide more flexibility in handling unreliable depth in scenes with reflective or refractive surfaces. Experiments on both synthetic and real-world datasets demonstrate that CoCoGaussian achieves state-of-the-art performance across multiple benchmarks.

In this paper, we propose CoCoGaussian, a framework that leverages physically grounded photographic principles to model defocus blur for 3D scene reconstruction. By incorporating an aperture to compute the Circle of Confusion (CoC) diameter based on depth and learnable aperture information, CoCoGaussian generates 3D Gaussians to form the CoC shape, accurately capturing shallow depth of field effects. To handle challenges such as unreliable depths caused by reflection or refraction, we introduce a learnable scaling factor to adaptively constrain Gaussians within the CoC radius, enhancing flexibility and robustness. This approach not only enables the reconstruction of sharp 3D scenes from defocused images but also allows customizable depth of field and dynamic focus plane adjustments for flexible scene visualization.

Visualization of Aperture parameter and Focus Plane Customization. The top row of images decreases the aperture parameter $K$ from left to right, while the bottom row moves the focus plane $d_{F}$ further from the camera from left to right.