New research by Brown University computer scientists may be a key step in bringing volumetric video—video that can be viewed from virtually any perspective in a 3D scene—to computers and smart televisions.
The research introduces a new way of processing video called PackUV, which improves the capture of 3D action and makes the final product readily streamable, storable and compatible with the video codecs that currently power most video on the internet.
"With volumetric video, you can basically explore a scene from any vantage point you want," said Aashish Rai, a computer science graduate student at Brown who led the work. "It captures three dimensions of space, plus time, making it a 4D video. With our work, we basically convert this entire 4D scene into a normal video that you can stream over the internet and share with friends."
Rai, who works in the Interactive 3D Vision and Learning Lab at Brown led by Assistant Professor Srinath Sridhar, will present the work in June at the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
Volumetric video is an emerging way of both capturing and viewing video. Actions are recorded using multiple synchronized cameras that encircle a scene. Computer algorithms then rebuild that physical space in 3D, making a video reproduction that can be viewed from any perspective within the space. Directors could use it to show a scene from a perspective where a camera would have been impossible to place. With an added user interface, people can navigate their way through a scene—watching, for example, sporting events from on the field or a concert from the stage.
