Skip to content

Latest commit

 

History

History
79 lines (67 loc) · 4.49 KB

README.md

File metadata and controls

79 lines (67 loc) · 4.49 KB
Raw

HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending (ICCV2023)

This repository hosts the official PyTorch implementation of the paper: "HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending".

HairCLIPv2 supports hairstyle and color editing individually or jointly with unprecedented user interaction mode support, including text, mask, sketch, reference image, etc.

Tianyi Wei1, Dongdong Chen2, Wenbo Zhou1, Jing Liao3, Weiming Zhang1, Gang Hua4, Nenghai Yu1
1University of Science and Technology of China, 2Microsoft Cloud AI, 3City University of Hong Kong, 4Xi'an Jiaotong University

News

2023.12.08: The initial version of the code is released.
2023.10.12: HairCLIPv2 is accepted by ICCV2023! 🎉

Getting Started

Prerequisites

$ pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
$ pip install ftfy regex tqdm matplotlib jupyter ipykernel opencv-python scikit-image kornia==0.6.7 face-alignment==1.3.5 dlib==19.22.1
$ pip install git+https://github.com/openai/CLIP.git

Pretrained Models

Download and put all the downloaded pretrained weights into the pretrained_models directory.

Path Description
FFHQ StyleGAN StyleGAN model pretrained on FFHQ with 1024x1024 output resolution.
Face Parse Model Pretrained face parse model taken from Barbershop.
Face Landmark Model Used to align unprocessed images.
Bald Proxy Bald proxy weights from HairMapper.
Sketch Proxy Sketch proxy weights trained on hair-sketch dataset using E2style.

Input Instructions

All inputs are in the test_images folder and all images should be made sure to be aligned to 1024x1024 according to FFHQ standards.

Path Description
unaligned_img Natural images that should be aligned using python scripts/align_face.py Results are stored in aligned_img.
aligned_img Depending on the usage, they should be moved to src_img or ref_img.
src_img Aligned image to be edited.
src_F The latent code of src_img being inverted is not required to be provided by the user.
ref_img Aligned hairstyle reference image, hair color reference image.
ref_latent The latent code of ref_img being inverted is not required to be provided by the user.

Enjoy

Open hairclip_v2_demo.ipynb to unlock your versatile hairstyles!

If you run it on a remote server, you need to run jupyter notebook --port=20367 --allow-root --ip=0.0.0.0 first. then use e.g. VS Code to select that Jupyter Server as your kernel.

To Do

  • Develop an interactive interface

Acknowledgements

This code is based on Barbershop.

Citation

If you find our work useful for your research, please consider citing the following papers :)

@article{wei2023hairclipv2,
  title={HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending},
  author={Wei, Tianyi and Chen, Dongdong and Zhou, Wenbo and Liao, Jing and Zhang, Weiming and Hua, Gang and Yu, Nenghai},
  journal={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2023}
}

@article{wei2022hairclip,
  title={Hairclip: Design your hair by text and reference image},
  author={Wei, Tianyi and Chen, Dongdong and Zhou, Wenbo and Liao, Jing and Tan, Zhentao and Yuan, Lu and Zhang, Weiming and Yu, Nenghai},
  journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}