For three months leading up to summer, I will be working with the University of Salford on an aspect of the EU funded FascinatE Project. FascinatE is a €9.5m EU funded project involving a group of partners from across Europe including the BBC, Technicolor, TNO and ARRI amongst others. FascinatE stands for: Format-Agnostic SCript-based INterAcTive Experience and is looking at broadcasting live events to give the viewer a more interactive experience no matter what device they are using the view the broadcast.
The FascinatE project will develop a system to allow end-users to interactively view and navigate around an ultra-high resolution video panorama showing a live event, with the accompanying audio automatically changing to match the selected view. The output will be adapted to their particular kind of device, covering anything from a mobile handset to an immersive panoramic display. At the production side, this requires the development of new audio and video capture systems, and scripting systems to control the shot framing options presented to the viewer. Intelligent networks with processing components will be needed to repurpose the content to suit different device types and framing selections, and user terminals supporting innovative interaction methods will be needed to allow viewers to control and display the content.
Joining Ben Shirley and Rob Oldfield (the UoS team responsible for a major proportion of the project’s audio component), I will be working on further development of an audio object extraction system. A large focus of spatial audio research at the moment is object based audio, (similar to that currently used in video games) and how the technique could be applied to linear media or more specifically, television broadcast. Extracting audio objects in a live context is a tricky proposition. In a small set, actors could be close mic’ed and local GPS could be used to provide positional data, however, in a live sports set-up such as football, another approach must be taken. Instead, pertinent on pitch audio events such as ball kicks or whistle blows must be triangulated using the current large aperture microphone array infrastructure (12 pitch side microphones).
Part of what I will be developing is a way to reject audio events detected outwith the parameters of the pitch, primarily crowd noise. The detection algorithm that seeks for ball kicks can currently throw out false detections due to drumming in the crowd and similarly, crowd whistling can be mistook by the application to be referee whistle blows. If the application were able to determine which were on pitch audio events and render the audio output accordingly, these false detections could be avoided.
The final demo of FascinatE will be hosted at MediaCityUK on the 30th of May, where all of the partners will descend upon Manchester to bring together all of the constituent parts of the projects and create one cohesive whole, a glimpse at the future of interactive broadcast.