Rapsai: Rapid Application Prototyping System for AI

The project Rapsai aims to make the prototyping of machine learning (ML) based multimedia applications more efficient and accessible. In recent years, there has been a proliferation of multimedia applications that leverage machine learning (ML) for interactive experiences. Prototyping ML-based applications is, however, still challenging, given complex workflows that are not ideal for design and experimentation. To better understand these challenges, we conducted a formative study with seven ML practitioners to gather insights about common ML evaluation workflows. \n\nThe study helped us derive six design goals, which informed Rapsai, a visual programming platform for rapid and iterative development of end-to-end ML-based multimedia applications. Rapsai features a node-graph editor to facilitate interactive characterization and visualization of ML model performance. Rapsai streamlines end-to-end prototyping with interactive data augmentation and model comparison capabilities in its no-coding environment. Our evaluation of Rapsai in four real-world case studies (N=15) suggests that practitioners can accelerate their workflow, make more informed decisions, analyze strengths and weaknesses, and holistically evaluate model behavior with real-world input.

Publications

teaser image of Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications Through Visual Programming

Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications Through Visual Programming

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI), 2023.
Keywords: visual programming, node-graph editor, deep neural networks, data augmentation, deep learning, model comparison, visual analytics




teaser image of Experiencing Rapid Prototyping of Machine Learning Based Multimedia Applications in Rapsai

Experiencing Rapid Prototyping of Machine Learning Based Multimedia Applications in Rapsai

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA), 2023.
Keywords: visual programming, node-graph editor, deep neural networks, data augmentation, deep learning, model comparison, visual analytics


Videos

Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications Through Visual Programming


Talks

Interactive Perception & Graphics for a Universally Accessible Metaverse Teaser Image.

Interactive Perception & Graphics for a Universally Accessible Metaverse

Ruofei Du

Invited Talk at UCLA by Prof. Yang Zhang , Remote Talk.


Interactive Graphics for a Universally Accessible Metaverse Teaser Image.

Interactive Graphics for a Universally Accessible Metaverse

Ruofei Du

Invited Talk at ECL Seminar Series by Dr. Alaeddin Nassani , Remote Talk.


Cited By

Stay In Touch