Conditional generation with images

[kallivad]

Thank you for very interesting and outstanding work in generation field. I am interested in conditional generation and as far as I see you can generate objects from ABO using only labels. But if I want generate 3D assets from one image containing projection of domain object how it needs to condition DiT for effective generation?

[QhelDIV]

It is indeed a direction to explore. I believe the simplest way is to add the condition through adding cross attention to DiT.

[kallivad]

I completely agree with you here, but then the question of efficiency arises. What kind of encoder should I use to take features from the image condition, do I need to concatenate with a temporary condition, as you do with a class label in your work? How to prepare features that most effectively convey the geometry of the input image? I understand that these are research questions, but your generator could change the world for systems where you need to generate optimal and editable grids.