As the Hollywood actors’ strike continues to persist with no end in sight, a recent technological breakthrough has addressed one of the actors’ major concerns: the ability to capture and store their performances and mannerisms as a 3D model. This new development involves the use of 3D scanning technology to record human bodies in motion, allowing studios to reuse these assets indefinitely.
In the past, 3D scanning technology in Hollywood involved complex setups with multiple cameras arranged 360-degrees around the actor’s body or the use of markers and tight-fitted bodysuits. Even with recent advancements using AI, multiple cameras were still required. However, a new method called Gaussian splatting has emerged as a game-changer. This technique, originally used to capture static 3D imagery from a single 2D camera, has now been modified to capture dynamic motion in 3D, including human body motions. This groundbreaking approach is known as “4D Gaussian splatting,” with time being the fourth dimension, allowing for changes in the captured image over time.
The Evolution of Gaussian Splatting
Gaussian splatting was initially introduced in 2001 by researchers at MIT, ETH Zurich, and Mitsubishi for scanning objects with lasers. It uses collections of particles to represent a 3D scene, each with its own attributes. Over the years, Gaussian splatting has advanced to the point where it can be rendered on modern web browsers and even on smartphones using a collection of 2D images.
However, extending Gaussian splatting to dynamic scenes has proven to be a complex challenge. According to a recent research paper published by Huawei and the Huazhong University of Science and Technology in China, the key difficulty lies in modeling intricate point motions from limited input. When multiple Gaussian splatters are joined together across different timestamps to create a moving image, points can deform inaccurately from image to image, leading to distorted representations of shapes and volumes.
Despite this challenge, the researchers were able to overcome it by utilizing a single set of canonical 3D Gaussians and employing predictive analytics to determine their movements from one timestamp to the next. This approach resulted in impressive 3D rotational effects, allowing viewers to explore the captured scenes from various angles.
Potential Applications in the Entertainment Industry
Although the current capabilities of 4D Gaussian splatting are limited to short 3D motion sequences, the technology shows great promise for the entertainment industry. Studios can now capture and reuse specific motions performed by actors, potentially reducing production costs and improving efficiency. Video game designers and XR/VR designers can also benefit from this technique, enhancing the immersive experience for users.
While the initial results are impressive, the research is ongoing, and the team aims to further improve rendering quality for complex real scenes in future developments. As technology continues to advance, it is likely that both the quality and quantity of captured motion will increase over time, making this method an invaluable asset for the industry.
