Virtual production has begun to cement its place in film making. This real-time technology strives to enhance the previsualization process, especially in the field of complex visual effects (VFX). More recently, its use in the field has also increased during production. This technology has come a long way. Where previously virtual production was used to create more complex VFX scenes, it is now used as the core of the entire production, paving the way for a more immersive approach to filmmaking.
With virtual production, both filmmakers and creators can create a digital environment in the preview phase before going into the physical scene. Then, using VR (virtual reality), filmmakers can access these environments on set. They just need to put on their helmets and be immersed in the world they’ve created.
This is a change for a film that values VFX, as it allows the director to see the action in the rest of the film’s background. Instead of relying on guesswork, they were able to edit and re-evaluate on the day of shooting, allowing them to create the best visuals for their films.
king of the jungle
Jon Favreau is one of the few directors to experiment with virtual production during filming. After exploring its possibilities in MPC’s The Jungle Book, Favreau took virtual production a step further to create a live-action remake of The Lion King. With the exception of the only real shot at the opening of the film, the film will be fully CGI to recreate a beloved classic. The realistic look of the film, combined with the way the camera moves, makes it look like a nature documentary – the camera takes the viewer on an animal-like journey.
To achieve this, the team at the Moving Picture Company (MPC) created a 3D virtual environment that encompasses the entire Savannah landscape and mirrors classic scenes from the original film. From elephant cemeteries to ancient trees in Rafic, Favreau and his team can don VR headsets and immerse themselves in virtual spaces, finding the best places and ways to create visuals for the film.
Outside of their virtual world, crews work in technical blank spaces on LED virtual set walls. There is a grid in space with a series of virtual reality sensors that feed back information to the computer, such as the position of a virtual camera in space. The film crew still mostly uses traditional production methods and equipment such as hand-held cameras, pulleys and rails. What they saw, however, was a simulation of what the final shot would look like; they could see how the camera would interact with the virtual environment and make changes on the spot.
This real-time scene visualization allows filmmakers to focus on the story rather than trying to create a shot without knowing what the finished product will be. Photographers can create and light their scenes and see them change in real time, just like in traditional production production. This did create some convenience while shooting, as crews were able to make adjustments or remove elements they didn’t want on set, but this method didn’t always yield the highest quality results.
After production is complete, a large post-production team and VFX artist are still required to produce the visuals to the quality typically seen on a movie theater screen. While Wall of the Future still requires a lot of post-production work, one of the great advantages of virtual production is that it opens up more creative collaborations earlier in the production process. Traditionally, while most directors work closely with VFX supervisors on set, there are often barriers between production and post-production teams. By leveraging virtual production, the two can work better together in the production process, blurring the lines of traditional film making we’re used to.
This can be seen in a relatively new phenomenon emerging in live production: the use of LED screens. The concept stems from the use of green screens, a staple of the film industry. It shows actors standing in front of a giant green backdrop interacting with imaginary landscapes and creatures. Otherwise, the director would have to wait until post-production to see all the elements of the film, which can all happen in real-time on LED screens as the realistic 3D landscape can be projected behind the actors.
While the technique has been used on a smaller scale before, it has recently come to fruition with The Mandalorian, where it was used to create many key scenes. To that end, ILM created an LED virtual set wall called Stagecraft, which consists of LED screens that run from wall to ceiling. These dynamic screens showcase unique views of the Star Wars universe, rendered with camera positioning data.
This means that as the camera moves, the 3D scene will change to reflect the movement. This gives the image depth and eliminates the possibility of the background looking flat when shot. However, the LED screen still poses a slight restriction on how the camera can be moved. There is a limit to how far the camera can be from the screen, as this can cause a moiré pattern or streak, which is often seen when shooting digital screens.
Despite this little problem, immersive LED screens have brought many new possibilities to the industry. One of them is reducing the need for live shoots. Not only does this save money, but it means that crews no longer have to wait for the perfect light or time to shoot – the immersive LED screen can show a specific time when needed. Beyond that, the immersive LED screens showcase the perfect example of the collaborative possibilities that virtual production and its inherent real-time capabilities unlock in VFX. For this technology to succeed, the artist must create the environment for the immersive LED screen ahead of time.
Once projected on the screen, the scenes can be manipulated as the director wants, which means the need for artists and animators to be on set, and if anything goes awry, cause all the different production areas to come together during production and play important role. REAL TIME AND BEYOND Look back and use virtual production on set to put the tools of storytelling back into the hands of filmmakers. Instead of working individually for the same goal, pre- and post-production teams can collaborate. Instead of reviewing footage later, the director can see what’s being created while shooting and make real-time decisions about the action and how things are being shot.
Artists envisioned by the executive director have an earlier understanding of these creative decisions, creating the perfect conditions for unfettered creative collaboration between the production team and other teams. Over the past 100 years, we’ve seen the human focus of film shift from pre-production — in the form of set construction, woodworking, and other technical trades — to post-production. The adoption of virtual production technology could see this reverse again.
Needless to say, the success of virtual production technologies in titles like The Lion King and The Mandalorian, and the speed at which they continue to evolve, signal that the industry—whether it’s directors, artists, or our viewing habits—has A very exciting time has come.
