Question on nan

[xyupeng]

Hi there,
This is a great work and many thanks for releasing the code. When I look into the code, I find some places where nan is handled, like

detr3d/projects/mmdet3d_plugin/models/utils/detr3d_transformer.py

Lines 366 to 367 in 34a4767

	output = torch.nan_to_num(output)
	mask = torch.nan_to_num(mask)

and

detr3d/projects/mmdet3d_plugin/models/dense_heads/detr3d_head.py

Lines 336 to 337 in 34a4767

	loss_cls = torch.nan_to_num(loss_cls)
	loss_bbox = torch.nan_to_num(loss_bbox)

I wonder in which cases nan would occur? It might help me better understand the model. Any advice would be very much appreciated!