Hand-controlled arpeggiator, drum machine, and audio reactive visualizer. Raise your hands to raise the roof!
An interactive web app built with threejs, mediapipe computer vision, rosebud AI, and tone.js.
- Hand #1 controls the arpeggios (raise hand to raise pitch, pinch to change volume)
- Hand #2 controls the drums (raise different fingers to change the pattern)
Video | Live Demo | More Code & Tutorials
- Modern web browser with WebGL support
- Camera access enabled for hand tracking
- MediaPipe for hand tracking and gesture recognition
- Three.js for audio reactive visual rendering
- Tone.js for synthesizer sounds
- HTML5 Canvas for visual feedback
- JavaScript for real-time interaction
# Clone this repository
git clone https://github.com/collidingScopes/arpeggiator
# Navigate to the project directory
cd arpeggiator
# Serve with your preferred method (example using Python)
python -m http.serverThen navigate to http://localhost:8000 in your browser.
MIT License
- Three.js - https://threejs.org/
- MediaPipe - https://mediapipe.dev/
- Rosebud AI - https://rosebud.ai/
- Tone.js - https://tonejs.github.io/
I've released several computer vision projects (with code + tutorials) here: Fun With Computer Vision
You can purchase lifetime access and receive the full project files and tutorials. I'm adding more content regularly 🪬
You might also like some of my other open source projects:
- 3D Model Playground - control 3D models with voice and hand gestures
- Threejs hand tracking tutorial - Basic hand tracking setup with threejs and MediaPipe computer vision
- Particular Drift - Turn photos into flowing particle animations
- Video-to-ASCII - Convert videos into ASCII pixel art
- Instagram: @stereo.drift
- Twitter/X: @measure_plan
- Email: stereodriftvisuals@gmail.com
- GitHub: collidingScopes
If you found this tool useful, feel free to buy me a coffee.
My name is Alan, and I enjoy building open source software for computer vision, games, and more. This would be much appreciated during late-night coding sessions!
