| Capstone Project: Remote Image Manipulation | |
| Client: Bob Pritchard and Keith Hamel, UBC Laptop Orchestra | |
| Student Team: Adam Berg, David Hu, Mike Jensen, Justin Siu | |
| Professor: Sid Fels |
Have you ever gone to an electronic music performance to watch a musician’s transfixed, blue-lit face, hunched over a laptop, one or two knob twiddles the only signs of life? Electronic music can be fresh and vivid, but the physical performance is often trapped within the genre, pointedly incorporeal.
The UBC Laptop Orchestra is a collection of artists working to expand the expressive possibilities of an elecroaucoustic performer and weave these possibilities more seamlessly with traditional instruments and performance forms. Music students in the orchestra are themselves very computer savvy but they have teamed up with Electrical and Computer Engineering students to benefit from the ECE students’ expertise in wireless applications, firmware, and advanced coding. A number of capstone teams have worked with the orchestra to provide performers access to the sensors in a smart phone so they can use them to track their gestures and control elements of their artworks. Using gestures as triggers for sonic or visual events brings the possibilities of the performer’s body to elecroaucoustic music.
One of the capstone teams working with the orchestra this year is developing an app to remotely control images. RIME (Remote Image Manipulation Environment) is a iOS smartphone application for controlling images in media performances. Performers can use the touch screen and on-phone sensors to control an image display and to control visual media applications. RIME has access to many of the built in sensors of the smart phone. This includes the accelerometer, gyroscope, and magnetic field sensors for detecting the movement and orientation of the phone. The app can also use lesser known sensors such as the humidity, ambient temperature, pressure, and proximity sensors. In performances the composers/performers use the gesture tracking data generated by the smartphones to simultaneously control audio and images. For Dr. Pritchard, “the results are a more sophisticated and engaging multimedia performance works.”
RIME is a continuation of a 2014 capstone project with the orchestra. The 2014 capstone team (Simon Hecker, Graeme Kirkpatrick, Pasan Weerasinghe, Jason Wong) developed sensorUDP, an app that allows performers to layer up to eight programmable sounds and modify them by moving their phone. SensorUDP is available in the Android app store and compatible with iPhones.
Find out more:
- The New Pulse Of Digital Music, Basil Waugh, UBC News
- Accelerating Rapid Damage Assessment
- Repairing Canada’s Aging Bridges
- Getting Power to the Internet of Things
- Adding a Sixth Sense: Air Quality
- Better Diagnostic Tools for Restless Legs Syndrome
- Finding New Ways to Visualize the Body: 3D Surgical Marking
- Become a Capstone Client
image: Diana Brownie performing with the RIME in collaboration with Greg Burgess (Composer), Tian Ip (Composer), Isha (Engineer ViPER) and Mike Jensen (Engineer RIME)