Maestro: Conduct Music with Your Hands!

Andrea De Carlo, Gabriele Tangerini, Lucrezia Di Bari, Pietro Cau
University of Trento
Multisensory Interaction Systems Course Project
Demo Video Project Report

Maestro is an experimental project exploring a futuristic way to make music — no piano, no guitar... just your hands, a smart glove 🧤, a baton 🪄, and a bit of magic ✨

Abstract

Maestro is an experimental project exploring a futuristic way to make music through gestural control and haptic feedback. Inspired by orchestra conductors, Maestro lets users control musical loops through gestures and receive haptic feedback — gentle vibrations that make the experience more tactile and intuitive. The system is designed for anyone, even if they've never played an instrument before. All you need is rhythm and curiosity!

The system consists of a glove with flex sensors, an IMU, and a vibration motor, paired with a baton containing an accelerometer for beat tracking. The goal is to investigate whether adding haptics improves the playability and overall feel of this kind of "air instrument" for music creation.

🛠️ System Components

The system is made up of:

  • A glove with flex sensors, an IMU, and a vibration motor
  • A baton with an accelerometer for beat tracking
  • A Teensy microcontroller to gather and transmit data
  • A Processing sketch to display the visual interface
  • A Pure Data patch for triggering and controlling audio clips

💡 How It Works

When you wear the glove and pick up the baton:

  • Your left hand (the glove) selects instruments and adjusts their intensity
  • Your right hand (the baton) keeps the beat — just like a real conductor!
  • Feedback vibrations give you a tactile "yes!" when you select something or hit the beat

A projected interface helps guide your movements, with four musical tracks (like drums 🥁, bass 🎸, synth 🎹, etc.) and three levels of complexity.

🔬 Experimental Results

We ran tests with 18 participants — musicians and total beginners alike. They played with and without haptic feedback, and we compared:

  • How well they kept tempo 🎵
  • How easily they navigated the interface 🧭
  • How much fun they had 😄

Results: The glove was a hit with haptics — people found it easier to use and more satisfying. The baton still has room to grow (timing control was tricky), but overall, the feedback made the experience feel more immersive.

📈 What We Learned

  • ✅ Haptics can really help in "empty-hand" musical interfaces
  • 🎯 The glove's usability got a big boost with vibrations
  • ⚠️ The baton needs improvements in timing detection and user control
  • 🧠 Gesture recognition can be refined with better flexibility and feedback

BibTeX

@misc{decarlo2023maestro,
  title={Maestro: Conduct Music with Your Hands! Testing Haptics for Primary Feedback in a Music-Making Multisensory Interactive System},
  author={De Carlo, Andrea and Tangerini, Gabriele and Di Bari, Lucrezia and Cau, Pietro},
  year={2023},
  institution={University of Trento},
  type={Course Project},
  note={Multisensory Interaction Systems}
}