Awesome Image Composition

We co-founded a startup company miguo.ai, dedicated to accelerating the production of comics and animations using AIGC technology. If you are seeking internship or full-time positions, please feel free to send your resume to hr@miguocomics.com.

---

A curated list of resources including papers, datasets, and relevant links pertaining to image composition (object insertion). The goal of image composition is inserting one foreground into a background image to get a realistic composite image, by addressing the inconsistencies (appearance, geometry, and semantic inconsistency) between foreground and background. Generally speaking, image composition could be used to combine the visual elements from different images.

Contributing

Contributions are welcome. If you wish to contribute, feel free to send a pull request. If you have suggestions for new sections to be included, please raise an issue and discuss before sending a pull request.

Table of Contents

• Online Demo
• Survey
• Toolbox
• Papers
• Datasets
• Evaluation

Online Demo

Try this online demo for image composition and have fun!

Survey

• Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, Liqing Zhang: "Making Images Real Again: A Comprehensive Survey on Deep Image Composition." arXiv preprint arXiv:2106.14490 (2021). [arXiv] [slides]

Toolbox

We integrate 10+ image composition related functions into libcom (the library of image composition), including image blending, standard/painterly image harmonization, shadow generation, object placement, generative composition, quality evaluation, etc. The ultimate goal of this library is solving all the problems related to image composition with simple import libcom.

Papers

1. Image Blending

  Awesome-Image-Blending

2. Image Harmonization

  Awesome-Image-Harmonization

3. Object Shadow Generation

  Awesome-Object-Shadow-Generation

4. Object Reflection Generation

• Daniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, Yedid Hoshen: "ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion." arXiv preprint arXiv:2403.18818 (2024) [arXiv]
• Shengjie Ma, Qian Shen, Qiming Hou, Zhong Ren, Kun Zhou: "Neural Compositing for Real-time Augmented Reality Rendering in Low-frequency Lighting Environments." Science China Information Sciences (2021) [pdf]

5. Object Placement

  Awesome-Object-Placement

6. Perspective Transformation

• Junhong Gou, Bo Zhang, Li Niu, Jianfu Zhang, Jianlou Si, Chen Qian, Liqing Zhang: "Virtual Accessory Try-On via Keypoint Hallucination." arXiv preprint arXiv:2310.17131 (2023) [arXiv]
• Bo Zhang, Yue Liu, Kaixin Lu, Li Niu, Liqing Zhang: "Spatial Transformation for Image Composition via Correspondence Learning." arXiv preprint arXiv:2207.02398 (2022) [arXiv]
• Fangneng Zhan, Hongyuan Zhu, Shijian Lu: "Spatial Fusion GAN for Image Synthesis." CVPR (2019) [pdf]
• Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, Simon Lucey: "ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing." CVPR (2018) [pdf] [code]

7. Occlusion

• Jonghyun Lee, Hansam Cho, Youngjoon Yoo, Seoung Bum Kim, Yonghyun Jeong: "Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image Synthesis." arXiv preprint arXiv:2401.09048 (2024) [pdf]
• Zan Li, Wencheng Wang, Fei Hou: "Image Composition with Depth Registration." IJCAI (2023) [paper]
• Fangneng Zhan, Jiaxing Huang, Shijian Lu: "Hierarchy Composition GAN for High-fidelity Image Synthesis." Transactions on cybernetics (2021) [arXiv]
• Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, Trevor Darrell: "Compositional GAN: Learning Image-Conditional Binary Composition." IJCV (2020) [arXiv] [code]

8. Resolution/Sharpness/Noise Discrepancy

• Jizhizi Li, Jing Zhang, Stephen J.Maybank, Dacheng Tao: "Bridging Composite and Real: Towards End-to-End Deep Image Matting." IJCV (2021) [pdf] [code]

9. Foreground Object Search

  Awesome-Foreground-Object-Search

10. Generative Image Composition

  Awesome-Generative-Image-Composition

Datasets

• Datasets for image harmonization [link]
• Datasets for object shadow generation [link]
• Datasets for object placement [link]
• Datasets for foreground object search [link]
• Datasets for perspective transformation [link]
• Datasets for generative image composition [link]

Evaluation

• Composite-Image-Evaluation