Hamburg thriller provides pioneering services for virtual production

The newcomer production of the twoHamburg Media School-Directors Pascal Schröder and Daniel Schua (DoP) relied on a pioneering technology that was first used on a large scale in the Star Wars series "The Mandalorian". Instead of building expensive sets for short scenes and shooting in front of a green screen, the Hollywood production's VFX team set up a huge LED wall in the studio and used a photorealistic representation of the background from the Unreal game engine. The camera movements from the real studio were transferred to the LED wall via a tracking system that generates a parallax shift synchronised with the camera. In this way, elaborate outdoor sets can be realised in controllable studio conditions. The production of the "Star Wars" series served as a model for the thriller "The Social Experiment". The new technology allowed the team to film on the moon, in the desert or in an icy landscape in just a few days.

Directors Pascal Schröder and Raffaela Kraus wrote the script specifically for the use of virtual technology. "The Social Experiment" tells the story of five friends who embark on a supposed adventure in an escape room for a competition. But the so-called "game" actually turns out to be a behavioural experiment that soon gets out of hand.

Founded in 2020 by Andreas Schlieter, Kai Steinmetz and Pascal Schröder, the Hamburg-based production companyGipfelstürmer film production has as a partnerCinegate Hamburg and the event service provider PRG Germany, who set up the 90 square metre LED wall in Studio 4 on the Cinegate site. The virtual studio was created by the "Holobay Kollektiv", also based in the Hanseatic city, which used the Unreal game engine to create the virtual worlds.

For Director of Photography Daniel Schua and Directors Pascal Schröder, this technology is a great opportunity to do pioneering work for the German film landscape. And Hamburg is once again at the forefront of a development that has gained momentum over the past 12 months, not least due to the coronavirus pandemic in Germany. In a digital showcase of the films planned for this autumn at Filmfest HamburgExplorer KonferenzFilmförderung Hamburg Schleswig-Holstein, the Producers' Association and the Filmfest Hamburg, the team from "The Social Experiment" presented the innovative workflow to the industry.

Virtual Production opens up a functioning interface between games technology and film production for the first time, shifting a considerable amount of VFX work from post production to pre-production. Around a third of the scenes for the thriller could thus be realised in the studio with virtual backgrounds. "We had around six days for the studio plus two more days for preparation," reported Executive Producer Andreas Schlieter. Directors and DoP Daniel Schua met with founder Dan Weigl's Holobay Artists before shooting began in early January to explore the new possibilities, but also the limits. It is important to consider the angle and distance at which the camera and actors are positioned in front of the virtual LED wall in the studio set to prevent moiré effects from revealing the pixel structure of the screen. The basic parameters were worked out in a short test shoot in December to avoid any nasty surprises during filming. It was clear that the actors had to maintain a minimum distance of 1.50 metres from the LED wall.

One particular focus is on making the transition between the digital LED wall and the analogue studio set in the foreground invisible. For the scenes on the surface of the moon, DoP Daniel Schua focussed heavily on the lighting design in order to merge the two visual worlds invisibly. What appears to be a theatre-like impression in the studio setting must merge into a homogeneous image space in the camera. The tracking software, which tracks the camera in the studio in real time and adjusts the parallax shift for the backgrounds, provides a strong realism effect. "When the camera moves around the protagonists in the studio, you can see how the background shifts. This creates a very real spatial impression that draws the viewer into the scene," explains Daniel Schua. This effect is further emphasised by the use of a remote-controlled telescopic crane, which allows the camera to move freely around the studio backdrop. The Technocrane also helps to avoid footprints on the studio floor, which is decorated with artificial snow or desert sand, which the camera operators would leave behind when trudging through the sets during filming - tracks that would have required some retouching in post.

The virtual image worlds can either be created photorealistically from scratch using the Unreal game engine - it would even be possible to scan and edit photographed landscapes - or they can be acquired from existing game engine libraries and customised to suit your own requirements. A complete creation of the virtual landscapes would have been too time-consuming. Directors Pascal Schröder and his team therefore selected existing templates and had them modelled according to their own wishes. "It's like a location tour, where you choose the worlds in Unreal and have them modelled according to your own ideas and conditions. We had control over the position of the sun, the colour temperatures or whether it was hazy, for example," explains Pascal Schröder. Colour differences between the sand scattered in the studio setting and the virtually generated sand could also be balanced out in colour matching. A setting with longer dialogue scenes at dusk with sunset can be easily implemented in the virtual space.

If the use of the LED worlds and the studio setting are precisely prepared, many images can be finalised practically in the camera, eliminating the need for time-consuming corrections in post production. However, this technology also has its limits. This begins with the fact that the real playing area of the actors in the Cinegate Studio had to remain limited to 9 square metres. "It was primarily about how our story could be told using virtual sets and we wanted to test the limits for our production," explains Executive Producer Andreas Schlieter. For smaller dialogue scenes that can be staged in a small area, virtual production offers new possibilities and can even offer advantages. This is because there are no travel costs, there is perfect lighting control in the studio environment and the work involved in post is minimised. However, elaborate action sequences with driving shots, tracking shots and ultratotals, in which many protagonists and extras move around in the frame, are more likely to be shot "on location".