DiverseShot Tutorial

DiverseShot is a local AI graphics client that allows you to easily create photorealistic 3D scenes and objects from videos and photos in minutes. DiverseShot uses a technique called "3D Gaussian Splatting." The 3D files produced by DiverseShot differ from traditional 3D file formats, as they do not include concepts like mesh or textures. Instead, they support display expressions suitable for opening and sharing in a browser.

What Can DiverseShot Be Used For?

As a non-graphics researcher, What can I do with DiverseShot?

• As a game designer or CG enthusiast: Import the ply files produced by DiverseShot into game engines like Unity or Unreal.
• As an e-commerce professional: Create 3D visuals of your products to embed into e-commerce websites, leading to higher conversion rates.
• As a real estate agent: Capture 3D visuals of properties, allowing clients to experience the space immersively via mobile devices.
• As an everyday person: Preserve your precious memories by converting valuable items into 3D files.

DiverseShot's Gaussian Training Workflow

Capture the Object to be Reconstructed in 3D:

To perform 3D reconstruction with DiverseShot, you need to capture a video of the Object from multiple angles (or take several photos from different angles). The recording device can be a phone or a camera. The more diverse the angles captured in the video, the better the training results.

Attention⚠️: There should be no dynamic objects in the video to be trained, otherwise it will affect the reconstruction effect

Create a Gaussian Training Project:

After successfully downloading and installing the client, the initial run will auto-download some libraries necessary for machine learning.

After download libraries,the user interface appears below ⬇️. Users need to create a project when using DiverseShot. If using the Windows client, and if the user does not customize the project's save location, project files will be saved by default in the "projects" folder📁 within the installation directory.

Import Video or Frame Sequence into the Project:

Drag the video (or images) into the project to start splat training. By default, the output location of the training results is the same as the current folder 📁 where the video is located, but users can customize the output location.

Training Iteration:

After starting splat training, the client will extract images from the video and reconstruct the camera 📷 trajectory. Upon completing ✅ the camera trajectory reconstruction, machine learning will generate the splat file. An "Entity" will be created in the hierarchy on the right. Selecting this entity will display its parameters and iteration count. The client remains in training mode until the training limit is reached. At the top🔝 of the interface, there is a pause ⏸️ button to pause the training.

Modifying the Splat During Training:

During the filming 🎬 process, it is inevitable that background elements outside the main Object will be captured.

These background elements will be included in the machine learning process and reconstructed, using up training resources and time. After pausing the training, you can modify the splat file📃 in two ways:

• Add a GaussianCrop Component: After selecting an entity, click "Add component/GaussianCrop" to create a 3D crop volume. The client will display the intersection of the crop volume and the splat file, allowing for quick trimming of large amounts of unnecessary splat files.

• Select and Delete Unwanted Splats via the SplatEdit Panel: Use the SplatEdit panel on the left to select unwanted splat files through the "box/sphere/brush..." components and delete them.

Exporting 3D Files:

There are two ways to export the 3D files📃 obtained after training:

• After selecting an entity, choose "save to file" in the inspector panel on the right and select the desired format.

• Locate the corresponding ply file in the folder address specified as the "output location" during the initial training setup.

How to Use the Exported PLY Files

We recommend using Spline as your final display platform. Spline is a free 3D design software with real-time collaboration to create web interactive experiences in the browser. It allowed user import .ply files and perform secondary editing such as 'Apply Cropping' / 'Make animation'.

Apart from Spline, if you have some programming skills, you can use the open-source WebGL library PlayCanvas to deploy a webpage for sharing .ply files. If you are working on a project using game engines like Unity or Unreal Engine (UE), you can also import .ply files into your project using the following plugins: 3D Gaussians Plugin for ue / UnityGaussianSplatting

Video Tutorial