How to shoot the best video for 3D reconstruction

How to Shoot the Best Video for 3D Reconstruction

The video quality used for Gaussian Training plays a decisive role in the final 3D reconstruction effect. So, how do you shoot 🎬 the best video for Gaussian Training? Here is a fundamental principle ⬇️:

The essence of Gaussian Training is to reconstruct the images into a cloud of points and then iteratively optimize 🔁 each angle's image effect to make the Gaussian function closely match the image's impact from every angle. Therefore, capturing 🎬 the object from as many different angles (top, bottom, left, right) as possible is crucial for achieving the best 3D reconstruction effect.

While it is possible to perform 3D Reconstruction from a video (or a set of images) captured from a single angle, the resulting 3D effect will only be noticeable when viewed from the same angle as the video display, as shown in the project below⬇️.

Does Higher Video Resolution Improve Reconstruction Quality?

There is no significant difference in reconstruction quality for videos with resolutions higher than 2K. Therefore, a 2K resolution is recommended for your videos.

Does a Higher Frame Rate Improve Reconstruction Quality?

The frame rate of the video does not affect the final training outcome. The DiverseShot client extracts frames at a fixed rate.

Some Limitations of Gaussian Files

• Objects with weak textures, hard surfaces, transparent glass, or high metallicity typically do not yield good results through Gaussian Training.
• The lighting during shooting cannot be separated from the inherent color, so shooting in a relatively uniform lighting environment is necessary.
• Gaussian Training can achieve basic lighting effects but does not support PBR (Physically Based Rendering) channels such as normals, metallicity, highlights, or roughness.
• Gaussian files cannot produce occlusion or shadows.