I3D 2018 Demo Video

This is Ruofei's archive for the Kernel Foveated Rendering project. Please refer to Meng's websitefor more details.

Foveated rendering coupled with eye-tracking has the potential to dramatically accelerate interactive 3D graphics with minimal loss of perceptual detail. In this paper, we parameterize foveated rendering by embedding polynomial kernel functions in the classic log-polar mapping. Our GPU-driven technique uses closed-form, parameterized foveation that mimics the distribution of photoreceptors in the human retina. We present a simple two-pass kernel foveated rendering (KFR) pipeline that maps well onto modern GPUs. In the first pass, we compute the kernel log-polar transformation and render to a reduced-resolution buffer. In the second pass, we carry out the inverse-log-polar transformation with anti-aliasing to map the reduced-resolution rendering to the full-resolution screen. We have carried out pilot and formal user studies to empirically identify the KFR parameters. We observe a 2.8X-3.2X speedup in rendering on 4K UHD (2160p) displays with minimal perceptual loss of detail. The relevance of eye-tracking-guided kernel foveated rendering can only increase as the anticipated rise of display resolution makes it ever more difficult to resolve the mutually conflicting goals of interactive rendering and perceptual realism.

Selected Comments from I3D'18 Reviewers

R1: The log-polar transformation was previously used in the prior [Antonelli et al. 2015], but this mapping is applied to 3D graphics for the first time. I believe this is a meaningful contribution. Also, user studies are conducted to decide ideal values for the two parameters used in the mapping function.
R2: The log-polar transformation idea is interesting and novel, to my best knowledge. The system is simpler to implement than prior foveated rendering methods that require temporal reprojection.
R3: This paper is very interesting to me as it extends the idea behind NVIDIA's MRS and LMS technologies to use a parameterized log-polar mapping. The proposed mapping is more complicated than previous work, and hence applies primarily to the lighting/shading pass in a deferred renderer. The positive aspects of this work include: The use of relatively few (2) foveation parameters; The use of relatively few (2) foveation parameters; An overall improvement of performance for foveated rendering.

Paper

Proceedings of the ACM on Computer Graphics and Interactive Techniques (I3D), 1(5). (20 pages)

Slides

Presented on ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. Montreal, Quebec, Canada: 15-18 May 2018

Code

Minimum demo on ShaderToy without overdraw and anti-aliasing.

BibTeX

@article{Meng2018Kernel,
    title = "Kernel Foveated Rendering",
    author = {Xiaoxu Meng and Ruofei Du and Matthias Zwicker and Amitabh Varshney},
    journal = {Proceedings of the ACM on Computer Graphics and Interactive Techniques (I3D)},
    year = {2018},
    month = {May},
    day = {15 - 18},
    volume = {1},
    number = {5},
    location = {Montreal, Quebec, Canada},
    publisher = {ACM},
    series = {I3D},
    keywords = {foveated rendering, perception, log-polar mapping, eye-tracking, virtual reality, head-mounted displays},
    doi = {10.1145/3203199},
    pages = {1--20}
}

Citation

Xiaoxu Meng, Ruofei Du, Matthias Zwicker, and Amitabh Varshney. (2018). Kernel Foveated Rendering. In Proceedings of the ACM on Computer Graphics and Interactive Techniques (I3D), 1(5), 1-20. DOI:10.1145/3203199