I3D 2018 Demo Video

In this paper, we present a novel GPU-driven pipeline for saliency computation and navigation in 360° videos, based upon spherical harmonics (SH). By analyzing the spherical harmonics spectrum of the 360° video, we extract the spectral residual by accumulating the SH coefficients between a low band and a high band. Our model outperforms the classic Itti et al.'s model in timings by over 5 to 13 times and runs at over 60 FPS for 4K videos. Furthermore, we use the interactive computation of saliency to devise a saliency-guided system for virtual cinematography in 360° videos. We formulate a spatiotemporal model to ensure large saliency coverage while reducing the camera movement jitter. We envision that our pipeline will be used in processing, navigating, and rendering 360° videos in real time.

This full paper is still under review

BibTeX

@article{Du2018Spherical,
    title = "Spherical Harmonics for Saliency Computation and Navigation in 360° Videos",
    author = {Ruofei Du and Amitabh Varshney},
    journal = {Poster of the ACM on Computer Graphics and Interactive Techniques (I3D)},
    year = {2018},
    month = {May},
    day = {15 - 18},
    volume = {1},
    number = {5},
    location = {Montreal, Quebec, Canada},
    publisher = {ACM},
    series = {I3D},
}

Citation

Ruofei Du, and Amitabh Varshney. (2018). Spherical Harmonics for Saliency Computation and Navigation in 360° Videos. In Poster Proceedings of the ACM on Computer Graphics and Interactive Techniques (I3D).