Configuration issues #6
Description
Hello X-Drive team,
First of all, thank you for sharing the code for this amazing work.
I wanted to run the code and test various scenarios, but I encountered some issues during the configuration.
Could you provide more information about these problems?
1. diffusers==0.17.1
You implemented a custom diffusers==0.17.1 in the third_party folder, following MagicDrive.
However, RangeLDM uses diffusers==0.26.2, which leads to some crashes.
For example, in third_party/RangeLDM/ldm/utils.py#L134, there is the following code:
if type(target_attr) == diffusers.models.lora.LoRACompatibleConv or type(target_attr) == torch.nn.Conv2d:
diffusers 0.17.1 doesn't have lora.py in the models folder.
I think this check is not needed because it would always be false, but I resolved it by injecting diffusers==0.26.2's models/lora.py. Is this expected?
2. use_gsa
When I run the training code, the following error occurs:
xdrive/runner/pc_ldm_box_RangeLDM_runner.py", line 102, in _init_trainable_models
unet_config['use_gsa'] = cfg.model.unet.use_gsa
omegaconf.errors.ConfigAttributeError: Key 'use_gsa' is not in struct
The configuration file SDv2.1pc_RangeLDM_box.yaml does not have use_gsa attribute in the unet section. I assume this should be set to false, but I want to confirm you whether this is correct.
3. pkl file
pkl files that you uploaded in Google Drive does not match with your training codes.
For example,
- In the annotation, "description" is missing
- "camera_intrinsic" was named with "cam_intrinsic"
However, pkl files generated from the given preprocess code works fine.
4. requirements
There are some libraries whose version needs to be specified.
- pillow < 10.0.0 (ANTIALIAS was deprecated)