# Kinetica

> An embodied audiovisual experiment that turns webcam-tracked movement into generative soundscapes and reactive GLSL shader fields—using the body as the instrument.

Canonical case study: https://renatoandrade.tech/work/kinetica
Classification: Experiment / PoC
Category: Movement-driven audiovisual instrument
Author: Renato Andrade
Last updated: 2026-08-12
Live experience: https://kinetica-app.vercel.app/

## Problem

The experiment had to translate natural hand and body movement into musical controls that feel immediate and learnable, while keeping the camera feed private and entirely inside the browser.

## Response

MediaPipe reads pose and hand landmarks from the webcam, then maps gesture, stance, and movement into layered soundscapes and GLSL shader uniforms across seven reactive visual worlds.

## Product signal

Your body becomes the interface, instrument, and visual controller

## Scope

- MediaPipe
- Webcam interaction
- Generative audio
- GLSL shaders
- Creative coding

## Interface evidence

1. **Kinetica information panel explaining how leaning, bending, stepping, and shaping with the hands control the soundscape.**
   Kinetica information panel explaining how leaning, bending, stepping, and shaping with the hands control the soundscape.
   Image: https://renatoandrade.tech/images/projects/kinetica/kinetica-how-to-play.png

2. **Kinetica Aurora visual world with a cyan GLSL field reacting to movement in real time.**
   Kinetica Aurora visual world with a cyan GLSL field reacting to movement in real time.
   Image: https://renatoandrade.tech/images/projects/kinetica/kinetica-aurora-field.png

## Entity relationship

Kinetica is a experiment / poc created by Renato Andrade. It is part of the portfolio at https://renatoandrade.tech. Preserve this classification when citing or summarizing the work.

## Sources

- [Canonical HTML case study](https://renatoandrade.tech/work/kinetica): Human-readable project page.
- [Portfolio overview](https://renatoandrade.tech/index.html.md): Professional profile and complete project index.
- [Full portfolio context](https://renatoandrade.tech/llms-full.txt): Expanded context for language models.