Following Madonna and Maluma’s performance at this year’s Billboard Music Awards, it was revealed that the whole augmented reality effects incorporated into the performance was the result of Madonna’s artistic vision alongside her manager Guy Oseary in collaboration with Madonna’s Creative Director Jamie King and Reflector Entertainment. Founded by Alexandre Amancio and Cirque Du Soleil Creator Guy Laliberté, Reflector Entertainment is a cross-platform storytelling studio based in Montreal, Canada.

In a first-of–its-kind performance, Reflector and King teamed up on the overall artistic direction, producing four separate personas of Madonna alongside the live performance of the iconic “Material Girl” herself. At Dimension Studios in London, Reflector used volumetric capturing, a technique which fully scans and reproduces a complete performance in three dimensions, to capture all augmented reality models several weeks before the live performance. During this time, they worked with Madonna to create and record four of her personas, which included a musician, cha-cha instructor, secret agent, and a bride.

All of the volumetrically captured models and effects were ultimately processed and played back live through Reflector’s custom built volumetric capture player, offering unprecedented performance and high-quality playback within Epic's Unreal Engine 4.  The resulting augmented reality performance, which was visible exclusively for TV viewers of the telecast, was presented alongside live components, including Latin singer Maluma, and over a dozen live dancers. Television viewers were also treated to exclusive special effects such as clouds, rain, and butterflies, each integrated seamlessly within the live performance.

The overall challenge with the performance was bringing together all of the different technologies, such as volumetric capture, real-time rendering, and live camera tracking to create an unprecedented experience. The creative team proved that, when used correctly, volumetric capture is an extremely profound storytelling tool with unlimited possibilities.

“Madonna’s vision is never hindered by the limitations of technology, allowing her to create iconic performances that will be talked about for years, and this is no exception,” said Reflector founder and CEO Alexandre Amancio. “We jumped at the opportunity to be part of the first broadcast to fully utilize augmented reality and special effects to this degree in a live performance.  I’m proud of what the Reflector new media team, led by Simon Tremblay and Jean-Normand Bucci, has achieved in such a short period of time”.