A curated list of resources including papers, datasets, and relevant links pertaining to object shadow generation. Shadow generation aims to generate plausible shadow for the inserted foreground object in a composite image. For more complete resources on general image composition, please refer to Awesome-Image-Composition.
Contributions are welcome. If you wish to contribute, feel free to send a pull request. If you have suggestions for new sections to be included, please raise an issue and discuss before sending a pull request.
A brief review on object shadow generation is included in the following survey on image composition:
Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, Liqing Zhang: "Making Images Real Again: A Comprehensive Survey on Deep Image Composition." arXiv preprint arXiv:2106.14490 (2021). [arXiv] [slides]
Try this online demo for shadow generation and have fun!
- Daniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, Yedid Hoshen: "ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion." arXiv preprint arXiv:2403.18818 (2024) [arXiv]
- Qingyang Liu, Junqi You, Jianting Wang, Xinhao Tao, Bo Zhang, Li Niu: "Shadow Generation for Composite Image Using Diffusion Model." CVPR (2024) [arXiv] [dataset&code]
- Xinhao Tao, Junyan Cao, Yan Hong, Li Niu: "Shadow Generation with Decomposed Mask Prediction and Attentive Shadow Filling." AAAI (2024) [arXiv] [dataset]
- Lucas Valença, Jinsong Zhang, Michaël Gharbi, Yannick Hold-Geoffroy, Jean-François Lalonde: "Shadow Harmonization for Realistic Compositing." SIGGRAPH Asia (2023) [paper]
- Quanling Meng, Shengping Zhang, Zonglin Li, Chenyang Wang, Weigang Zhang, Qingming Huang: "Automatic Shadow Generation via Exposure Fusion." T-MM (2023) [paper]
- Yichen Sheng, Jianming Zhang, Julien Philip, Yannick Hold-Geoffroy, Xin Sun, He Zhang, Lu Ling, Bedrich Benes: "PixHt-Lab: Pixel Height Based Light Effect Generation for Image Compositing." CVPR (2023) [paper] [code]
- Tianyanshi Liu, Yuhang Li, Youdong Ding: "Shadow Generation for Composite Image with Multi-level Feature Fusion." EITCE (2022) [pdf]
- Yan Hong, Li Niu, Jianfu Zhang: "Shadow Generation for Composite Image in Real-world Scenes." AAAI (2022) [arXiv] [dataset&code]
- Yichen Sheng, Yifan Liu, Jianming Zhang, Wei Yin, Oztireli Cengiz, He Zhang, Lin Zhe, Shechtman Eli, Bedrich Benes: "Controllable Shadow Generation Using Pixel Height Maps." ECCV (2022) [arXiv] [Project]
- Yichen Sheng, Jianming Zhang, Bedrich Benes: "SSN: Soft shadow network for image compositing." CVPR (2021) oral [pdf] [code] [Project]
- Daquan Liu, Chengjiang Long, Hongpan Zhang, Hanning Yu, Xinzhi Dong, Chunxia Xiao: "ARshadowGAN: Shadow generative adversarial network for augmented reality in single light scenes." CVPR (2020) [pdf] [code]
- Shuyang Zhang, Runze Liang, Miao Wang: "ShadowGAN: Shadow synthesis for virtual objects with conditional adversarial networks." Computational Visual Media (2019) [pdf]
- Fangneng Zhan, Shijian Lu, Changgong Zhang, Feiying Ma, Xuansong Xie: "Adversarial Image Composition with Auxiliary Illumination." ACCV (2020) [pdf]
-
Bin Liao, Yao Zhu, Chao Liang, Fei Luo, Chunxia Xiao: "Illumination animating and editing in a single picture using scene structure estimation." Computers & Graphics (2019) [pdf]
-
Bin Liu, Kun Xu, Ralph R. Martin: "Static scene illumination estimation from videos with applications." Journal of Computer Science and Technology (2017) [pdf]
-
Kevin Karsch, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, Hailin Jin, Rafael Fonte, Michael Sittig, David Forsyth: "Automatic scene inference for 3d object compositing." ACM Transactions on Graphics (2014) [arXiv]
-
Kevin Karsch, Varsha Hedau, David Forsyth, Derek Hoiem: "Rendering synthetic objects into legacy photographs." ACM Transactions on Graphics (2011) [arXiv]
- Shadow-AR: It contains 3,000 quintuples, Each quintuple consists of 5 images 640×480 resolution: a synthetic image without the virtual object shadow and its corresponding image containing the virtual object shadow, a mask of the virtual object, a labeled real-world shadow matting and its corresponding labeled occluder. [pdf] [link]
- DESOBA: It contains 840 training images with totally 2,999 object-shadow pairs and 160 test images with totally 624 object-shadow pairs. [pdf] [link]
- DESOBAv2: It is a real-world shadow generation dataset constructed using object-shadow detection and inpainting models. It has 21, 575 images with 28, 573 valid object-shadow pairs. [pdf] [link]
- RdSOBA: It is a large-scale Rendered Shadow Generation dataset with 30 3D scenes, 788 3D foreground objects, and 280,000 object-shadow pairs. [pdf] [link]
