The Challenge
Pixotope partnered with production company That’s It Media to scope out possibilities for the first season of an experimental virtual character talent competition "2060". It will be a 10-part series aired on China's Jiangsu Satellite TV, the third-largest Chinese network.
That’s It Media and 2060’s producers wanted to exploit virtual production technology to create something never before seen by the target audience of 18 to 35-year-olds. They wanted to whet the appetite of Chinese audiences for the concept of futuristic virtual talent competitions.
The Background
The 2060 series sees 30 teams design, model, and animate 3D performers who are puppeted using motion capture. They compete live on a green screen studio stage. Live studio judges, together with a blended live and virtual audience, then watch the virtual stars perform and can interact with them on stage via two screens.
The action transitions seamlessly between live augmented reality and five landmark virtual scenes, known collectively as Star Ring City. To maximize the visual experience for viewers, the complexity and ambition of episodes increased throughout the series. Stunning visual effects were realized by the technical team as the world and environments evolved around the show’s characters.
The Pixotope Solution
Pixotope wrote a tailor-made high-performance virtual crowd generation system for real-time use to blend the virtual and live audience The system gave program makers a customizable framework that pulled in randomized audience characters and ensured optimum flexibility and performance of the virtual audience. It was also lightweight in terms of rendering, allowing many thousands of characters to run simultaneously in real-time.
Real-time Live Pickup Workflow
Collaborating closely with That’s It Media, Pixotope designed a bespoke virtual production workflow in advance of the start of filming. Pixotope then worked extensively with the production team on location in Shanghai and Nanjing, sharing knowledge and experience, while helping to refine novel workflows.
Pixotope designed and implemented a live pick-up workflow for the team, recording the tracking data from the 10 studio cameras and building a custom timecode-based tracking lock for the production, which enabled a second rendering workflow. This allowed a seamless transition from live recording into a fully operational post workflow so that if one of the motion capture routines went awry or any other content needed to be adapted, it could be recreated and rerun as live.
Integrated into this adaptable workflow were keying, automatic tracking labeling, live tracking match, and advanced compositing using the additional tools in the Pixotope compositing toolkit, all easily streamlined and accessible to aid the team of vendors and facilitate production.
Visually, 2060 is stunning. Considering the show’s focus on detailed animated characters, the production required the idols to be rendered at a quality not usually achieved in real time. In order to enhance the rendering process, Pixotope integrated deep learning tech from NVIDIA, enabling the animated stars of the show to be shown in cinematic quality to audiences in the studio and at home.
Success
Working closely with Pixotope as their technology partner and fully exploiting its feature-packed real-time engine, That’s It Media and 2060 demonstrated the potential for cinematic quality virtual talent shows on television. After the first episode aired, the show's V-life characters immediately captured the attention of audiences, sparking trends and discussion on Chinese social media. A second season is planned for 2022.
Leveraging the power of Unreal Engine for a brand new kind of variety performance, Pixotope provided an easy interface and bespoke modules for teams new to virtual production to leverage the power of Unreal Engine, whilst providing training and support to those teams throughout.
“We wanted a platform that was stable and, at the same time, allowed us to make last-minute changes in real-time. Pixotope allowed us to do all that. In addition, the team from Pixotope was extremely supportive and we had a really good experience.”
Gong Yizhao, CEO, That’s It Media
“Working in close collaboration with Space Warp Engine studio, we were able to develop and deliver fundamental workflow changes in Pixotope to help them realize their visions. Throughout the process, and as the requirements changed, we were able to advise and respond. The final result is ground-breaking and visually stunning. The creativity and resourcefulness of Space Warp Engine studio are exceptional and we are proud to be associated with this production.”
Andrew Tan, VP of Sales APAC for Pixotope
Discover our Live Event Solutions
Pixotope supports AR solutions at live events for ambitious productions.
Read more about how Pixotope can provide live event solutions.