ADITI GUPTA

2024 — website

Dance For Me is a web project that uses Google’s Pose Landmark Detection deep learning framework to pinpoint the coordinates of one’s body Dance For Me is a web project that uses the motion of a user’s body to create a sound experience in which a “song reacts to you as you react to it.” After enabling the webcam and uploading a .wav file within the site, coordinate data from MediaPipe’s Pose Landmark Detection model is used to create sound alterations of the song as the user moves in front of the camera. Sound alterations include pitch shift, a phaser, feedback delay, distortion, and playback rate.

By creating a codependent relationship between song playback and user movement, the piece introduces a sense of restricted motion, transforming the body into an instrument that must move deliberately. The initial concept emerged from an exploration of how webcam technology could serve as an instrumental interface for sound. After researching various webcam-compatible frameworks and libraries, the decision was made to use the entire body, rather than just isolated parts, to emphasize a “puppeteering” effect, where the user is compelled to keep their arms raised and body in motion to maintain the integrity of the sound. It becomes a subtle tug-of-war for control: the song forces you to keep dancing, while your movement, in turn, keeps the song alive.

Measurements such as lateral sway and arm height are derived from the pose detection model and mapped to specific sound alterations using Tone.js. These audio effects are chained together to create a layered soundscape, where variations in the user’s movement data dynamically influence the intensity of each effect. The result is a sound filter that transforms the uploaded track into a distinct and unfamiliar variation of the original tune.

The frontend of the site is built entirely using vanilla HTML, CSS, and JavaScript, styled in a way that evokes the aesthetics of early 2000s web design. Tacky font, pop-up menus, and the overall appearance of the stick figure are choices meant to capture the playful, unpolished spirit of the handmade web era—highlighting a time when websites were more experimental, unserious, and free from corporate polish. Read more about it here.

This work was created under the guidance of Sam Tarakajian at Cycling ’74.

Using: MediaPipe, Tone.js, Python, JavaScript

visit site

DANCE 4 ME

[2024 — web]

‘Dance4Me' (Dance For Me) is a web project that uses Google’s Pose Landmark Detection deep learning framework to pinpoint the coordinates of one’s body to create a sound experience in which a “song reacts to you as you react to it.” By enabling the webcam and uploading a .wav file within the site, the coordinate data is used to create sound alterations that will interfere with the playback of the song as the user moves in front of the camera. Some examples of sound alteration include pitch shift, a phaser, feedback delay, distortion, and playback rate.