Myself and I in the Stream of Virtual Subconsciousness

• Venue1: The Great Hall, YUZ Museum, Shanghai, China | November 29, 2020
• Venue2: The 2nd China Contemporary Dance Biennale | September 6 - 12, 2021
• Collaborators: Jess Chiayi Seetoo (Shanghai Theatre Academy) and Jung Min Moon (Chugye University for the Arts, Seoul)
• Performance Duration: Approximately 30 minutes.

Project Description

In the unfolded space of the subconscious, one encounters... self, and another self, and another... Is it dreams within a dream, memory, past lives, or the cyborgian butterfly mind?

Combining new media, motion capture, dance, and various settings of human-machine “improvisatory” frameworks, the performance explores the limits and fusions of sensorial and creative intelligence between human and technology. The performance draws inspiration from Roy Ascott's theorization of variable reality and Zhuangzi’s “Dream of the Butterfly.” In a large gallery space, bodies, virtual images, and technology come together to examine multi-dimensional realities and the perpetual fascination with the self.

The project was funded and exhibited as part of the Hyundai Art+Tech public program, organized in partnership with Hyundai Motor Company to explore the integration of art and technology.

The live media performance developed in November 2020 was re-developed and showcased as a video form for the 2nd China Contemporary Dance Biennale.

Contributions

• Conceptual Development: Led the conceptual development of the project, integrating diverse media contents, choreographies, and recorded sound.
• Motion Tracking System: Developed a real-time motion tracking and networking system visualizing dancers' movements on stage using three Kinects and a network system.
• "Virtual Memory": Created a 3D space for storing body shapes of performers and audiences using BodyPix, Firebase, and three.js shaders.
• Synchronization: Designed and controlled the synchronization of media content using Firebase Realtime Database.
• 3D Visualization: Simulated and controlled media equipment installations and live performance operations with 3D sketches.
• Sound Design: Provided frameworks for composing sounds and collaborated with the project’s sound designer.
• Installation: Supervised the installation of media equipment and managed the operation of the performance.

Space/Stage Plan

The performance took place in the 1st-floor gallery space of YUZ Museum, which was treated as a chamber of the subconscious mind. The performance included:

• 5 projectors and motion tracking systems for immersive experiences.
• A single darkroom for sound immersion.
• Audience participation as they followed the performer through space.

Figures and Visualizations

Space/Equipment Simulation in 3D

Finalized Installation Plan

Real-time visualization of abstract body figures

The point cloud data of the performer was visualized with 0.5 million particles in real-time.

Visualization of "Virtual Memory"

Performer memories were abstractly illustrated in 3D space with 0.8 million particles.

Real-time Visualization of "Memory in Different Times"

Performer appearances processed in real-time via BodyPix.

Visualization of "Speculation"

Integrated lights and mirrors enhanced choreography in a dark room.