Audiences enjoying live music today have become used to spectacular audio-visual experiences with big production gear for visuals, that is demanding for both artists and venues to work with, due to the size, the time used and costs incurred. Additionally, the possibility for working with this gear is limited to larger venues and bigger productions, but a trend is seen where concerts are arranged outside of traditional venues, in spaces which are adapted for live music for a brief amount of time, such as living room concerts and pop up concerts.
This project explores the possibility of creating portable visuals that artists can use regardless of the venue they will play in, which are customisable by the artist, to provide a novel visual experience for audiences.
The possibility of creating a smart phone app for use in a Head Mounted Display (HMD) during concerts is investigated by interviewing artists and analyzing answers from an audience questionnaire. The app proposed uses Augmented Reality to generate the visuals, and computer vision to analyse the visual scene.
Visuals for artist Maya Westman’s song Rudaali Rising were created as part of this project and three prototypes were developed in Unity, which each utilised computer vision in different ways. Possible visual effects were tested, such as image processing, particle systems, and post processing image effects. A think aloud test was conducted by mimicking a concert environment and using the Samsung Gear VR HMD with a Samsung S7 smart phone. The test revealed that users prefer effects which happen in 360-degrees all around them and set a mood, and that image processing effects which change the overall look of the visual scene are very interesting, but not for longer durations of time, since too much visual information from the concert is lost.
The prototype utilizing Vuforia for image recognition was used in a final test. The test was conducted in three different locations with Maya Westman singing live, and a focus group was carried out that revealed many design guidelines for how portable mobile augmented reality (MAR) visuals should be created.
The project concludes that MAR visuals are indeed able to provide a novel visual experience for audiences, and that further investigation of many design directions, as well as solving some issues is still needed. A discussion was offered in the project report on possibilities for improvements and research areas for designers interested in pursuing this endeavour. Contact me for more information.
The prototype based on Vuforia can be seen in concert in the second video (test by Louise Lessel at Dj Flugvel og Geimship concert at Plugout Festival 2017, no copyright infringement intended). The video shows 3 cases: Visuals in 360′ degrees, Visuals coming from the artist’s instrument, and Visuals streaming from the stage position in the concert space through the audience.
Original project in collaboration with Luda Diepgrond.
With artist Maya Westman.