Deep Learning in 3D Vision and Computer Graphics
Through "Deep Learning in 3D Vision and Computer Graphics", we aim to advance research in computer graphics and 3D vision through the development of novel datasets, models, and algorithms. In SketchyScene, we developed a large-scale dataset of scene sketches for advancing research in sketch understanding and enabling several applications such as image retrieval, sketch colorization, editing, and captioning. In Language-based Colorization, we developed a language-based system for interactive colorization of scene sketches, which is more natural for novice users than scribble-based interfaces. In OmniSyn, we developed a novel pipeline for 360◦ view synthesis between wide-baseline panoramas, which is expected to produce a smoother experience for navigating immersive maps. In PRIF, we developed a new implicit shape representation called Primary Ray-based Implicit Function, enabling efficient shape extraction and differentiable rendering, and achieving successes in various tasks including shape generation, completion, and inverse rendering. In MDIF, we developed Multiresolution Deep Implicit Functions, a hierarchical representation that can recover fine geometry detail while being able to perform global operations such as shape completion, and achieving superior performance in various 3D reconstruction tasks.