Requirements and install ?

[aschelch]

Hi, your works looks interesting, gg ;)

Is it possible to add requirements.txt file and installation information ?

Via trial and error installing it, I have an error with lmd dependency

Traceback (most recent call last):
  File "C:\Users\XX\Documents\Development\MagicFusion.github.io\scripts\stable_txt2img.py", line 19, in <module>
    from ldm.util import instantiate_from_config
  File "C:\Users\XX\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\ldm.py", line 20
    print self.face_rec_model_path
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

thx in adance

[aschelch]

Btw, how to try the recontextualization part ?

[MagicFusion]

Btw, how to try the recontextualization part ?

For Application 2, i.e., recontextualization, you need to fine-tune the stable diffusion model for a specific object
like Dreambooth (https://github.com/XavierXiao/Dreambooth-Stable-Diffusion) does. To generate the specific object with
the placeholder [ ] (e.g. sks), both model0 and model1 are specified as the fine-tuned model. Then, the fusion
result can be abtained by giving the prompt a photo of [] <class> and a photo of [] <class> <environment>.

[MagicFusion]

Hi, your works looks interesting, gg ;)

Is it possible to add requirements.txt file and installation information ?

Via trial and error installing it, I have an error with lmd dependency

Traceback (most recent call last):
  File "C:\Users\XX\Documents\Development\MagicFusion.github.io\scripts\stable_txt2img.py", line 19, in <module>
    from ldm.util import instantiate_from_config
  File "C:\Users\XX\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\ldm.py", line 20
    print self.face_rec_model_path
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

thx in adance

set-up the ldm enviroment following the instruction from textual inversion repo, or the original Stable Diffusion repo.

[csyxwei]

Btw, how to try the recontextualization part ?

For Application 2, i.e., recontextualization, you need to fine-tune the stable diffusion model for a specific object like Dreambooth (https://github.com/XavierXiao/Dreambooth-Stable-Diffusion) does. To generate the specific object with the placeholder [ ] (e.g. sks), both model0 and model1 are specified as the fine-tuned model. Then, the fusion result can be abtained by giving the prompt a photo of [] <class> and a photo of [] <class> <environment>.

Hi, could you explain why both model0 and model1 are specified as the fine-tuned model? According to the paper, model0 should be a general model, and model1 should be dreambooth model.

[MagicFusion]

Btw, how to try the recontextualization part ?

For Application 2, i.e., recontextualization, you need to fine-tune the stable diffusion model for a specific object like Dreambooth (https://github.com/XavierXiao/Dreambooth-Stable-Diffusion) does. To generate the specific object with the placeholder [ ] (e.g. sks), both model0 and model1 are specified as the fine-tuned model. Then, the fusion result can be abtained by giving the prompt a photo of [] <class> and a photo of [] <class> <environment>.

Hi, could you explain why both model0 and model1 are specified as the fine-tuned model? According to the paper, model0 should be a general model, and model1 should be dreambooth model.

Both the general model and the fine-tuned model can generate images of a photo of <class>, however, only the fine-tuned model understands the meaning of [ ]. Hence, for generating a photo of [ ] <class> <environment>, only a fine-tuned model can be used as model1, while both the general model and fine-tuned model are applicable for generating a photo of <class> (i.e., model0). Setting model0 and model1 as fine-tuned models, and providing prompts of a photo of [ ] <class> and a photo of [ ] <class> <environment> respectively, mainly aims to demonstrate the enhancement effect of Magicfusion on Dreambooth.

Indeed, Magicfusion not only integrates the strengths of different models to generate images that meet the requirements, but it can also merge by specifying two prompts for the same model, in order to achieve generation effects that are unattainable with complex prompts.

[csyxwei]

Thanks for your prompt reply!

Another question, how to reproduce the recontextualization results in the paper. Are the results shown in the Fig. 1(b) obtain by model0 = general model, model1 = dreambooth model ? or model0 = model1 = dreambooth model?

[MagicFusion]

Thanks for your prompt reply!

Another question, how to reproduce the recontextualization results in the paper. Are the results shown in the Fig. 1(b) obtain by model0 = general model, model1 = dreambooth model ? or model0 = model1 = dreambooth model?

To begin with, it is essential to obtain a fine-tuned model for a specific image based on the instructions provided by Dreambooth. Next, in Magicfusion, specify model0 and model1 as the general model and the fine-tuned Dreambooth model, respectively, using "a photo of <class> <environment>" and "a photo of [] <class>" as prompts. It is worth noting that fine-tuning for the specific image is a crucial step, as inadequate training or overfitting can result in poor Magicfusion performance. To ensure fairness, it is best to conduct a comparative observation with Dreambooth as outlined above. In Fig. 1(b), model0 is the general model, and model1 is the fine-tuned Dreambooth model.

[csyxwei]

Get it. Thanks very much!