Interactive Perception & Graphics for XR
2019 - Present
Ruofei Du, Xun Qian, David Li, Zhongyi Zhou, Alex Olwal, and David Kim
Projects
2D justifies existence,
3D validates identification,
4D convinces living,
and these projects fulfill my life.
Despite that “most software today is very much like an Egyptian pyramid with millions of bricks piles on top of each other, with no structural integrity, but just done by brute force and thousands of slaves”, I commit to inventing novel interactive systems that make information more useful and accessible, and help people to live a better physical life.
Research Projects
Augmented Communication for XR
2021 - Present
Ruofei Du, Zhenyi He, Xingyu Bruce Liu, Vladimir Kirilyuk, Xiuxiu Yuan, Peggy Chi, Xiang 'Anthony' Chen, Ken Perlin, and Alex Olwal
Visual Blocks: Visual Prototyping of AI Pipelines
2022 - Present
Ruofei Du, Na Li, Jing Jin, Michelle Carney, Scott Miles, Maria Kleiner, Xiuxiu Yuan, Yinda Zhang, Anuva Kulkarni, Xingyu Bruce Liu, Sergio Orts Escolano, Abhishek Kar, Alex Olwal, Ping Yu, Ram Iyengar, and Adarsh Kowdle
Avatars and Digital Human for Augmented Communication
2016 - Present
Ruofei Du, Ming Chuang, Wayne Chang, Hugues Hoppe, Amitabh Varshney, Feitong Tan, and Yinda Zhang
Improving Accessibility for Extended and Physical Reality
2014 - Present
Dhruv Jain, Tiger Ji, Lee Stearns, Ruofei Du, Uran Oh, Yumeng Wang, Leah Findlater, Rama Chellappa, Yuhang Zhao, Alex Olwal, and Jon E. Froehlich
Deep Learning in 3D Vision and Computer Graphics
2018 - Present
Changqing Zou, Qian Yu, Ruofei Du, Haoran Mo, Yi-Zhe Song, Tao Xiang, Chengying Gao, Baoquan Chen, Hao Zhang, Chengying Gao, and Hongbo Fu
Geollery: Social Media Metaverse in Virtual and Augmented Reality
2014 - 2019
Ruofei Du, David Li, and Amitabh Varshney
Foveated Rendering and Streaming for 3D Graphics
2017 - 2020
Xiaoxu Meng, Ruofei Du, David Li, Adharsh Babu, Camelia Brumar, Matthias Zwicker, and Amitabh Varshney
DepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality
2019 - Present
Ruofei Du, Eric Turner, Maksym Dzitsiuk, Shahram Izadi, Adarsh Kowdle, Konstantine Tsotsos, and David Kim
SketchyScenes: Understanding Scene Sketches
2018 - 2019
Changqing Zou, Qian Yu, Ruofei Du, Haoran Mo, Yi-Zhe Song, Tao Xiang, Chengying Gao, Baoquan Chen, Hao Zhang, Chengying Gao, and Hongbo Fu
360° Video Streaming and Rendering
2018 - 2021
David Li, Ruofei Du, Adharsh Babu, Camelia Brumar, and Amitabh Varshney
CollaboVR: A Reconfigurable Framework for Multi-user to Communicate in Virtual Reality
2019 - 2019
Zhenyi He, Ruofei Du, and Ken Perlin
GazeChat: Enhancing Virtual Conferences with Gaze-aware 3D Photos
2020 - 2021
Zhenyi He, Keru Wang, Br Feng, on Yushan, __ , __Ruofei Du, and Ken Perlin
Digital Human and Video-based Rendering
2016 - 2018
Ruofei Du, Ming Chuang, Wayne Chang, Hugues Hoppe, and Amitabh Varshney
ORCLayout: Adaptive GUI Layout with OR-Constraints
2018 - 2020
Yue Jiang, Ruofei Du, Christof Lutteroth, and Wolfgang Stuerzlinger
HandSight: A Touch-Based Wearable System to Increase Information Accessibility for People with Visual Impairments
2013 - 2015
Lee Stearns, Ruofei Du, Uran Oh, Yumeng Wang, Leah Findlater, Rama Chellappa, and Jon E. Froehlich
VRSurus: Enhancing Interactivity and Tangibility of Puppets in Virtual Reality
2016 - 2017
Ruofei Du and Liang He
Tracking-Tolerant Visual Cryptography
2018 - Present
Ruofei Du, Eric Lee, and Amitabh Varshney
Online Vigilance Analysis Combining Video and Electrooculography Features
2011 - 2013
Ruofei Du, Renjie Liu, Tianxiang Wu, and Bao-Liang Lu
3DVAR: From 3D Reconstruction to Virtual and Augmented Reality
2012 - 2013
Ruofei Du, Zhiwei Li, Rui Cai, Siyuan Fang, Jiawei Gu, Bao-Liang Lu, and Lei Zhang
MeteoVis: Visualizing Meteorological Events in Virtual Reality
2019 - 2020
David Li, Eric Lee, Elijah Schwelling, Mason Quick, Patrick Meyers, Ruofei Du, and Amitabh Varshney
ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard of Hearing Users
2021 - 2021
Dhruv Jain, Khoa Nguyen, Steven Goodman, Rachel Grossman-Kahn, Hung Ngo, Aditya Kusupati, Ruofei Du, Alex Olwal, Leah Findlater, and Jon E. Froehlich
RetroSphere: Self-Contained Passive 3D Controller Tracking for Augmented Reality
2019 - 2022
Ananta Balaji, Clayton Kimber, David Li, Shengzhi Wu, Ruofei Du, and David Kim
Modeling and Improving Text Stability in Live Captions
2022 - 2023
Xingyu Bruce Liu, Jun Zhang, Leonardo Ferrer, Susan Xu, Vikas Bahirwani, Boris Smus, Alex Olwal, and Ruofei Du
Visual Captions: Augmenting Verbal Communication with On-the-fly Visuals
2022 - 2023
Xingyu Bruce Liu, Vladimir Kirilyuk, Xiuxiu Yuan, Peggy Chi, Xiang 'Anthony' Chen, Alex Olwal, and Ruofei Du
ThingShare: Ad-Hoc Digital Copies of Physical Objects for Sharing Things in Video Meetings
2022 - 2023
Erzhen Hu, Jens Emil Sloth Grønbæk, Wen Ying, Ruofei Du, and Seongkook Heo
ChatDirector: Enhancing Video Conferencing with Space-Aware Scene Rendering and Speech-Driven Layout Transition
2022 - 2023
Xun Qian, Feitong Tan, Yinda Zhang, Brian Moreno Collins, Alex Olwal, David Kim, Karthik Ramani, and Ruofei Du
UI Mobility Control in XR: Switching UI Positionings Between Static, Dynamic, and Self Entities
2022 - 2023
Siyou Pei, David Kim, Alex Olwal, Yang Zhang, and Ruofei Du
Human I/O: Towards a Unified Approach to Detecting Situational Impairments in Everyday Activities
2022 - 2023
Xingyu Bruce Liu, Jiahao Nick Li, David Kim, Xiang 'Anthony' Chen, and Ruofei Du
InstructPipe: Building Visual Programming Pipelines with Human Instructions
2022 - 2024
Zhongyi Zhou, Jing Jin, Vrushank Phadnis, Xiuxiu Yuan, Jun Jiang, Xun Qian, Jingtao Zhou, Yiyi Huang, Zheng Xu, Yinda Zhang, Kristen Wright, Jason Mayes, Mark Sherwood, Johnny Lee, Alex Olwal, David Kim, Ram Iyengar, Na Li, and Ruofei Du
Beyond the Phone: Exploring Context-Aware Interaction Between Mobile and Mixed Reality Devices
2022 - 2024
Fengyuan Zhu, Xun Qian, Daniel Kalmar, Mahdi Tayarani, Eric Gonzalez, Mar Gonzalez-Franco, David Kim, and Ruofei Du
Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agent
2024 - 2025
Geonsun Lee, Min Xia, Nels Numan, Xun Qian, David Li, Yanhe Chen, Achin Kulshrestha, Ishan Chatterjee, Yinda Zhang, Dinesh Manocha, David Kim, and Ruofei Du
DialogLab: Authoring, Simulating, and Testing Dynamic Group Conversations in Hybrid Human-AI Conversations
2024 - 2025
Erzhen Hu, Yanhe Chen, Mingyi Li, Vrushank Phadnis, Pingmei Xu, Xun Qian, Alex Olwal, David Kim, Seongkook Heo, and Ruofei Du
Fun Projects
AtmoSPHERE: Representing Space and Movement Using Sand Traces in an Interactive Zen Garden
2013 - 2014
Ruofei Du, Kent Wills, Max Potasznik, and Jon E. Froehlich
DuEngine: An Efficient Interactive C++ Renderer
2017 - Present
Ruofei Du
DuCrawler: Mining Datasets From Google and Bing
2018 - Present
Ruofei Du
C-Flow: Visualizing Foot Traffic and Profit Data to Make Informative Decisions
2013 - 2013
Tiffany Chao, Ruofei Du, Jonathan Gluck, Hitesh Maidasani, Kent Wills, and Ben Shneiderman
JanCorpus: A Search Engine for Corpus with Millions of Janpanese Essays
2011 - 2012
Ruofei Du, Enpeng Yao, Bo Chen, Dongqing Zhang, and Jianhua Zhang
57aMP: Create Physical Stamps with the Thermal Power of Touch
2013 - 2013
Tiffany Chao, Ruofei Du, and Jon E. Froehlich
57fire: Creating Live Fire Illusion in Holographic Display with Heat
2013 - 2013
Ruofei Du, Tiffany Chao, and Jon E. Froehlich
Cubot: An In-hand or Wearable Input Device Attached Onto Everyday Object
2013 - 2013
Ruofei Du, Fan Du, and Jon E. Froehlich
3DEye: Eye-controlled 3D Wander System in 2010
2010 - 2011
Ruofei Du, Chengnan An, Zi Wang, Chenchen Shen, and Zhanxun Dong
Enchanting Ema: A Magical Interpretation of Shinto-Japanese Beliefs
2014 - 2014
Kent Wills, Ruofei Du, and Jon E. Froehlich