SparseConv3D

[Yzichen]

GaussianFormer/model/encoder/gaussian_encoder/spconv3d_module.py

Lines 53 to 67 in 3bf28d5

	indices = anchor_xyz - anchor_xyz.min(0, keepdim=True)[0]
	indices = indices / self.grid_size[None, :] # bg, 3
	indices = indices.to(torch.int32)
	batched_indices = torch.cat([
	torch.arange(bs, device=indices.device, dtype=torch.int32).reshape(
	bs, 1, 1).expand(-1, g, -1).flatten(0, 1),
	indices], dim=-1)
	spatial_shape = indices.max(0)[0]
	input = spconv.SparseConvTensor(
	instance_feature.flatten(0, 1), # bg, c
	indices=batched_indices, # bg, 4
	spatial_shape=spatial_shape,
	batch_size=bs)

Although interaction can be efficient using sparse 3D convolution, there may be multiple 3D GSs belonging to the same grid, and after convolution, these 3D GSs will share an equal feature. Is this a limitation?

[huang-yh]

Indeed, this is a limitation. We chose sparse 3D convolution due to its efficiency and 3D inductive bias.