DiverseShot Tutorial

DiverseShot is a local AI graphics client that allows you to easily create photorealistic 3D scenes and objects from videos and photos in minutes. DiverseShot uses a technique called "3D Gaussian Splatting." The 3D files produced by DiverseShot differ from traditional 3D file formats, as they do not include concepts like mesh or textures. Instead, they support display expressions suitable for opening and sharing in a browser.

What is DiverseShot used for?

As a non-graphics researcher, what can I do with DiverseShot?

• As a game designer or CG enthusiast: Import the ply files produced by DiverseShot into game engines like Unity or Unreal.
• As an e-commerce professional: Create 3D visuals of your products to embed into e-commerce websites, leading to higher conversion rates.
• As a real estate agent: Capture 3D visuals of properties, allowing clients to experience the space immersively via mobile devices.
• As an everyday person: Preserve your precious memories by converting valuable items into 3D files.

Download & Install

First, users visit the DiverseShot official website and register an account before downloading the DiverseShot client.The recommended operating system for running the client is Windows 10 or later, and the minimum required graphics card is the Nvidia GeForce RTX 1060. We recommend the Nvidia GeForce RTX 3060 with 12GB of memory or a more powerful model for optimal performance.

After successfully downloading and installing the client, the initial run will auto-download necessary PyTorch libraries for machine learning.If you are a user from mainland China, please ensure that you have VPN access enabled throughout the installation process. Without it, you will not be able to download the PyTorch libraries successfully.

DiverseShot's Gaussian Training Workflow

Video Tutorial

Create a Project

After download libraries,the user interface appears below ⬇️.

Users must create a project when using DiverseShot. If the project’s save location is not customized, the project files will be saved by default in the ‘projects’ folder 📁 within the installation directory.

Start Training

Drag a video or a folder containing image sequences (supporting JPEG or PNG formats) into the view window to start splat training.

Start training with video

Start training with images

If the background of the PNG image you imported is transparent, please check the ‘MUSK’ option in the interface before starting the training.

Pause the training

After starting training, the client will extract images from the video and reconstruct the camera 📷 trajectory. Upon completing ✅ the camera trajectory reconstruction, machine learning will generate the splat file. An "Entity" will be created in the hierarchy on the right.

Selecting this entity will display its parameters and Max-iteration. The client remains in training mode until the training limit is reached.

At the top🔝 of the interface, there is a pause ⏸️ button to pause the training.

Modifying the Splat

After attaining the MaxIteraion, you can modify the splat file📃 .Click "Edit" to activate the sidebar editing menu.

Add a GaussianCrop Component: After selecting an entity, click "Add component/GaussianCrop" to create a 3D crop volume. The client will display the intersection of the crop volume and the splat file, allowing for quick trimming of large amounts of unnecessary splat files.

Select and Delete Unwanted Splats via the SplatEdit Panel: Use the SplatEdit panel on the left to select unwanted splat files through the "box/sphere/brush..." components and delete them.

Export

How to Use the Exported PLY Files

We recommend using Spline as your final display platform. Spline is a free 3D design software with real-time collaboration to create web interactive experiences in the browser. It allowed user import .ply files and perform secondary editing such as 'Apply Cropping' / 'Make animation'.

Apart from Spline, if you have some programming skills, you can use the open-source WebGL library PlayCanvas to deploy a webpage for sharing .ply files. If you are working on a project using game engines like Unity or Unreal Engine (UE), you can also import .ply files into your project using the following plugins: 3D Gaussians Plugin for ue / UnityGaussianSplatting