docs: ControlNet idea

docs/gsoc/2025/ideas.rst

@@ -87,6 +87,71 @@ Synfig is quite a huge application. Most likely this app would start with only v
 **Expected size of project:** 350 hours
 
 
+
+Synfig + controlnet-openpose integration (350 hrs)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**Description:**
+Project Idea:
+Integration of a skeletal animation system into Synfig using ControlNet for generating images based on descriptions.
+
+Problem:
+Modern neural networks are capable of generating high-quality images, but achieving precise alignment with the author's vision requires additional control. ControlNet allows for specifying desired character poses, but the process of manually creating such poses can be labor-intensive, especially for animators who work with animation and require precision in details (e.g., foot placement, hand positions, or body rotation angles).
+
+Solution:
+1. **Adding a New Skeletal Layer to Synfig:**
+   - Synfig will introduce a special layer containing a standard character skeleton. This skeleton will be integrated with ControlNet, allowing users to define character poses in an intuitive interface.
+   - The layer will support character descriptions (e.g., "a mighty hero in adamantine armor"), which will be used for image generation.
+
+2. **Image Generation via ControlNet:**
+   - When the "Generate" button is clicked, Synfig will send skeleton data and text descriptions to the selected neural network (e.g., Stable Diffusion with ControlNet).
+   - The neural network will generate an image, which will be automatically imported back into Synfig as a new layer (image?).
+
+3. **Additional Features:**
+   - To simplify the creation of animation sketches, a neural network that generates only character silhouettes can be integrated. This will allow for quick creation of basic poses for animation without the need for detailed rendering.
+   - The ability to adjust the level of detail in the generated image (from silhouette to full render).
+
+Advantages:
+- Simplifying the animation process by automating character pose generation.
+- Reducing the time spent on planning and rendering complex poses.
+- An intuitive interface for animators, allowing them to focus on the creative aspects of their work.
+- The ability to use the system for both full-fledged image creation and quick sketches.
+
+Use Case:
+An animator creates a character skeleton in Synfig, provides a description ("a knight in armor standing in a combat stance"), and selects a pose. After clicking the "Generate" button, the neural network creates an image of the knight in the specified pose, which is automatically added to the project. This allows the animator to focus on animation without spending time rendering each detail.
+
+Future Development:
+- The ability to import poses from other 3D animation software.
+- Integration with other neural networks for generating backgrounds, textures, or additional elements.
+- Adding support for custom skeletons and anatomical templates.
+
+**Requirements:** Video adapter from NVidia/AMD/Intel with at least 8 GB of video memory, for running a local neural network. (need to check this)
+
+**Where to begin:**
+To begin with, it's necessary to see how ControlNet works and manually generate an image with the desired pose.
+You can start with this website: https://huggingface.co/lllyasviel/sd-controlnet-openpose
+
+
+**Expected outcome:**
+
+- Added a new layer "OpenPose" with an additional property "Description", where the user can specify a description for the neural network.
+- The "OpenPose" layer should contain a standard humanoid skeleton, with the ability to set the desired pose.
+- Right-clicking on the "OpenPose" layer opens a menu with an additional "Generate" command.
+- The "Generate" command sends the necessary data to the neural network server (this can be implemented by calling a local Python script that will handle the required task).
+- After receiving a response from the script, the generated image is loaded into Synfig.
+- A new tab should be added to the Synfig settings, containing settings for the neural network server.
+
+
+**Difficulty:** Medium
+
+**Skills required/preferred:** Python/C++
+
+**Possible mentor(s):** `Mohamed Adham <https://github.com/mohamedAdhamc>`_ , `Rodolfo Ribeiro Gomes <https://github.com/rodolforg>`_
+
+**Expected size of project:** 350 hours
+
+
+
 Brush tool (175hrs)
 ~~~~~~~~~~~~~~~~~~~
 
@@ -115,6 +180,8 @@ A working tool that allows users to freely hand-draw their artwork, which can th
 
 **Expected size of project:** 175 hours
 
+
+
 Propose a Project
 ------------------
 If you have a project idea, edit the "Project Ideas" section below by filling the required details and sending a pull request (this page is editable at  https://github.com/synfig/synfig-docs-dev/blob/master/docs/gsoc/2025/ideas.rst), even if you could not mentor (we will find a mentor).