"Failed to match keys" when using trained LoRA model

[magpie514]

I have been doing experiments with DAdaptation. After tinkering a bit I found a very good setup and obtained a really good output model out of it.
The model seems to work fine when used, both with the standard webui LoRA implementation and your own extension.

But when loaded, with both implementations, it'll give a strange warning:

LoRA model megamedi_last(b2d96e8fff26) loaded: _IncompatibleKeys(missing_keys=[], unexpected_keys=['lora_unet_input_blocks_1_0_emb_layers_1.alpha', 'lora_unet_input_blocks_1_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_1_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_2_0_emb_layers_1.alpha', 'lora_unet_input_blocks_2_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_2_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_4_0_skip_connection.alpha', 'lora_unet_input_blocks_4_0_skip_connection.lora_down.weight', 'lora_unet_input_blocks_4_0_skip_connection.lora_up.weight', 'lora_unet_input_blocks_4_0_emb_layers_1.alpha', 'lora_unet_input_blocks_4_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_4_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_5_0_emb_layers_1.alpha', 'lora_unet_input_blocks_5_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_5_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_7_0_skip_connection.alpha', 'lora_unet_input_blocks_7_0_skip_connection.lora_down.weight', 'lora_unet_input_blocks_7_0_skip_connection.lora_up.weight', 'lora_unet_input_blocks_7_0_emb_layers_1.alpha', 'lora_unet_input_blocks_7_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_7_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_8_0_emb_layers_1.alpha', 'lora_unet_input_blocks_8_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_8_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_10_0_emb_layers_1.alpha', 'lora_unet_input_blocks_10_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_10_0_emb_layers_1.lora_up.weight', 'lora_unet_input_blocks_11_0_emb_layers_1.alpha', 'lora_unet_input_blocks_11_0_emb_layers_1.lora_down.weight', 'lora_unet_input_blocks_11_0_emb_layers_1.lora_up.weight', 'lora_unet_middle_block_0_emb_layers_1.alpha', 'lora_unet_middle_block_0_emb_layers_1.lora_down.weight', 'lora_unet_middle_block_0_emb_layers_1.lora_up.weight', 'lora_unet_middle_block_2_emb_layers_1.alpha', 'lora_unet_middle_block_2_emb_layers_1.lora_down.weight', 'lora_unet_middle_block_2_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_0_0_skip_connection.alpha', 'lora_unet_output_blocks_0_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_0_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_0_0_emb_layers_1.alpha', 'lora_unet_output_blocks_0_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_0_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_1_0_skip_connection.alpha', 'lora_unet_output_blocks_1_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_1_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_1_0_emb_layers_1.alpha', 'lora_unet_output_blocks_1_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_1_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_2_0_skip_connection.alpha', 'lora_unet_output_blocks_2_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_2_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_2_0_emb_layers_1.alpha', 'lora_unet_output_blocks_2_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_2_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_3_0_skip_connection.alpha', 'lora_unet_output_blocks_3_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_3_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_3_0_emb_layers_1.alpha', 'lora_unet_output_blocks_3_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_3_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_4_0_skip_connection.alpha', 'lora_unet_output_blocks_4_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_4_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_4_0_emb_layers_1.alpha', 'lora_unet_output_blocks_4_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_4_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_5_0_skip_connection.alpha', 'lora_unet_output_blocks_5_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_5_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_5_0_emb_layers_1.alpha', 'lora_unet_output_blocks_5_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_5_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_6_0_skip_connection.alpha', 'lora_unet_output_blocks_6_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_6_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_6_0_emb_layers_1.alpha', 'lora_unet_output_blocks_6_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_6_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_7_0_skip_connection.alpha', 'lora_unet_output_blocks_7_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_7_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_7_0_emb_layers_1.alpha', 'lora_unet_output_blocks_7_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_7_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_8_0_skip_connection.alpha', 'lora_unet_output_blocks_8_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_8_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_8_0_emb_layers_1.alpha', 'lora_unet_output_blocks_8_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_8_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_9_0_skip_connection.alpha', 'lora_unet_output_blocks_9_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_9_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_9_0_emb_layers_1.alpha', 'lora_unet_output_blocks_9_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_9_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_10_0_skip_connection.alpha', 'lora_unet_output_blocks_10_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_10_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_10_0_emb_layers_1.alpha', 'lora_unet_output_blocks_10_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_10_0_emb_layers_1.lora_up.weight', 'lora_unet_output_blocks_11_0_skip_connection.alpha', 'lora_unet_output_blocks_11_0_skip_connection.lora_down.weight', 'lora_unet_output_blocks_11_0_skip_connection.lora_up.weight', 'lora_unet_output_blocks_11_0_emb_layers_1.alpha', 'lora_unet_output_blocks_11_0_emb_layers_1.lora_down.weight', 'lora_unet_output_blocks_11_0_emb_layers_1.lora_up.weight'])
setting (or sd model) changed. new networks created.

The webui implementation shows a similar warning as well.
As demonstrated by the image, the training worked, and very well, so this is just noise about extra blocks?
Should I be concerned about it? Did something fail in the training? I see no visible warnings or errors in the training console.

I'm using your scripts at commit dd05d99 and the config of this training is as follows:

{
ss_batch_size_per_device: "1",
ss_bucket_info: "{"buckets": {"0": {"resolution": [320, 768], "count": 6}, "1": {"resolution": [448, 704], "count": 26}, "2": {"resolution": [512, 640], "count": 28}, "3": {"resolution": [576, 576], "count": 22}, "4": {"resolution": [704, 448], "count": 2}}, "mean_img_ar_error": 0.04893075256764087}",
ss_bucket_no_upscale: "False",
ss_cache_latents: "True",
ss_caption_dropout_every_n_epochs: "0",
ss_caption_dropout_rate: "0.0",
ss_caption_tag_dropout_rate: "0.0",
ss_clip_skip: "2",
ss_color_aug: "False",
ss_enable_bucket: "True",
ss_epoch: "20",
ss_face_crop_aug_range: "None",
ss_flip_aug: "True",
ss_full_fp16: "False",
ss_gradient_accumulation_steps: "1",
ss_gradient_checkpointing: "False",
ss_keep_tokens: "1",
ss_learning_rate: "1.0",
ss_lowram: "False",
ss_lr_scheduler: "constant",
ss_lr_warmup_steps: "5",
ss_max_bucket_reso: "768",
ss_max_grad_norm: "1.0",
ss_max_token_length: "225",
ss_max_train_steps: "1680",
ss_min_bucket_reso: "320",
ss_mixed_precision: "fp16",
ss_network_alpha: "64.0",
ss_network_dim: "128",
ss_network_module: "networks.lora",
ss_new_sd_model_hash: "89d59c3dde4c56c6d5c41da34cc55ce479d93b4007046980934b14db71bdb2a8",
ss_noise_offset: "0.06",
ss_num_batches_per_epoch: "84",
ss_num_epochs: "20",
ss_num_reg_images: "0",
ss_num_train_images: "84",
ss_optimizer: "dadaptation.dadapt_adam.DAdaptAdam(decouple=True,weight_decay=0.01)",
ss_output_name: "megamedi_last",
ss_prior_loss_weight: "1.0",
ss_random_crop: "False",
ss_reg_dataset_dirs: "{}",
ss_resolution: "(576, 576)",
ss_sd_model_hash: "925997e9",
ss_sd_model_name: "animefinal-full-pruned.ckpt",
ss_sd_scripts_commit_hash: "dd05d99efd2f83efd4a0d26430dccdcac4f29480",
ss_seed: "420",
ss_session_id: "31633648",
ss_shuffle_caption: "True",
ss_text_encoder_lr: "0.5",
ss_total_batch_size: "1",
ss_training_comment: "LORA:megamedi",
ss_training_finished_at: "1678416179.0352871",
ss_training_started_at: "1678412720.0291808",
ss_unet_lr: "1.0",
ss_v2: "False"
}

[kohya-ss]

Thank you for reporting!

The training will be OK. This is an issue in Additional Networks extension. Some LoRA modules for Conv2d-3x3 are not enabled, and the weights for these modules are not loaded as the warning says. I think the Web UI's built-in LoRA also doesn't support LoRA modules for Conv2d-3x3, therefore it will bring almost same result. I will fix this after work.

[magpie514]

Thank you!

[kohya-ss]

I think this is fixed by kohya-ss/sd-webui-additional-networks#128

If you have any problem, please re-open the issue.

[magpie514]

The updated extension is fine with the older model. Then I updated the scripts and trained again, and now sd-webui doesn't output warnings anymore, so seems fully fixed! Thank you.