Creative Technology
VJ & Audio-Visual
Software Engineer
Collaborated with
musician / artists -
DJ Rebolledo, ASUKA, YY, HACOIRI, Kylie
Manami Sakamoto, Demsky
Performed At
Public Visuals, LVMH Tokyo Summit
WOMB, HVEN, Mitsuki, Circus
Creative Technology
VJ & Audio-Visual
Software Engineer
Collaborated with
musician / artists -
DJ Rebolledo, ASUKA, YY, HACOIRI, Kylie
Manami Sakamoto, Demsky
Performed At
Public Visuals, LVMH Tokyo Summit
WOMB, HVEN, Mitsuki, Circus
September 2024
Unreal Engine 5
Blender
Open AI API, speechgen.io API
Exhibited at:
- GenAI Summit San Francisco 2024 Art Section, CA, US
- NYU Tisch International Arts Incubator Exhibition, La MaMa Experimental Theatre, NY, US
-
NYU ITP/IMA Spring Show 2024, NYU Tisch School of the Arts, NY, US
Press
Press Kit 2024 Version
About Rising River
Rising River is an AI-driven self-analysis VR game inspired by Jungian shadow theory and the VIA 24 Character Strengths. In this interactive experience, players use their voice to restore a symbolic river, engaging in both fixed and generative self-reflective questions. The journey climaxes with the appearance of a giant fish, representing the player’s shadow self.My role: AI-integration, UE5 development
Team Credits
Creative Director: Lois He
AI Integration: Vio Zhu
Visual Design: John Luo
Tech Art: John Luo, Lois He
Technical Consultant: Xuelong Mu
VR Development: Lois He, Ellen Chen
UI Design: John Luo, Lillian Sun
Sound & Music Design: Kaiyang Zhao
Psychology-Guided Design: Andrew Trousdale
Special Thanks: Dan Tomasulo, Victor Morales, Jonathan Turner, Tianqi Yang, Todd Bryant, Chenyi Lee
Inspiration and Concept
The inspiration comes from my childhood fantasies—a mysterious space that interacts with me, fulfilling the need to temporarily leave reality and enter a private, safe place. Even as adults, this need persists. People watch movies, play games, or go to clubs to briefly escape life. But these activities often take people to irrelevant places. What if there was a space where you can leave temporarily but still reflect on your life in some way?
Narrative Healing
Psychological research shows that storytelling helps people find meaning in their life experiences, construct identity, and cope with trauma. Due to my background in filmmaking, I realized that narrative itself is a way to organize memories and psychological structures. This research validates the significance of generative narrative healing experience.
-
Establish real-time communication with ChatGPT through the OpenAI API to customize questions for players with different personalities.
- Convert user input into GPT system settings and conversation prompts.
- Convert questions and dialogs generated by GPT into real-time audio within the game.
- Develop the game's storyline and user interaction logic.
Technical Difficulties
- Latency improvement from both game logic perspective and technical perspective.
- Performence optiization in VR mode.