Learning Continuous Implicit Representation for
Near-Periodic Patterns.

NPP-Net handles both global consistency (e.g., window spatial arrangement) and local variations (e.g., window appearance variations) in the NPP scene, thereby producing better completion for the regions behind the tree, lamp, and car.

Abstract

Near-Periodic Patterns (NPP) are ubiquitous in man-made scenes and are composed of tiled motifs with appearance differences caused by lighting, defects, or design elements. A good NPP representation is useful for many applications including image completion, segmentation, and geometric remapping. But representing NPP is challenging because it needs to maintain global consistency (tiled motifs layout) while preserving local variations (appearance differences). Methods trained on general scenes using a large dataset or single-image optimization struggle to satisfy these constraints, while methods that explicitly model periodicity are not robust to periodicity detection errors.

To address these challenges, we learn a neural implicit representation using a coordinate-based MLP with single image optimization. We design an input feature warping module and a periodicity-guided patch loss to handle both global consistency and local variations. To further improve the robustness, we introduce a periodicity proposal module to search and use multiple candidate periodicities in our pipeline. We demonstrate the effectiveness of our method on more than 500 images of building facades, friezes, wallpapers, ground, and Mondrian patterns on single and multi-planar scenes.


Video

NPP-Net

NPP-Net consists of four modules: (1) Periodicity Proposal (green) proposes multiple candidate peroidicities for the input NPP scene. (2) Periodicity-Aware Input Warping (pink) warps input coordinates using detected periodicity. (3) Coordinate-based MLP (blue) maps warped and input coordinate features to an RGB value. (4) Single Image Optimization (yellow) uses pixel loss and periodicity-guided patch loss on a single NPP image.

NPP-Net architecture.

Application 1: NPP Completion

Single-Planar NPP Completion

NPP-Net architecture.

Different Size of the Unknown Mask.

Move the slider to adjust the mask size.


Multi-Planar NPP Completion

Move the slider to show results of different models.

Move the slider to show results of different models.

Application 2: NPP segmentation

Application 3: NPP Remapping


BibTeX

@inproceedings{chen2022learning,
  author = {Bowei Chen and Tiancheng Zhi and Martial Hebert and Srinivasa G Narasimhan},
  title = {Learning Continuous Implicit Representation for Near-Periodic Patterns},
  booktitle = {European Conference on Computer Vision},
  year={2022}
}

Acknowledgements

This work was supported by a gift from Zillow Group, USA, and NSF Grants #CNS-2038612, #IIS-1900821. Special thanks to Sing Bing Kang and Ivaylo Boyadzhiev for their constructive feedback; This website template was borrowed from SunStage. Thanks Yifan!