diff --git a/docs/index.rst b/docs/index.rst index 0763bd1..e5ab295 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -17,6 +17,7 @@ Welcome to Synfig developers documentation! community/contribution guidelines ide/Setting up your preferred IDE common/structure + renderer building/Building Synfig packaging/packaging tutorials diff --git a/docs/renderer.rst b/docs/renderer.rst new file mode 100644 index 0000000..5013079 --- /dev/null +++ b/docs/renderer.rst @@ -0,0 +1,16 @@ +.. _renderer: + +Renderer Docs +===================== + +This section explains the different parts of Cobra Engine and the algorithm which uses them to render images. + +.. toctree:: + :maxdepth: 1 + :glob: + + renderer_docs/introduction + renderer_docs/target_surface + renderer_docs/tasks + renderer_docs/render_queue + renderer_docs/optimizers diff --git a/docs/renderer_docs/introduction.rst b/docs/renderer_docs/introduction.rst new file mode 100644 index 0000000..dcd100c --- /dev/null +++ b/docs/renderer_docs/introduction.rst @@ -0,0 +1,49 @@ +.. _renderer_intro: + +Introduction +============ + +There are two ways to render a Synfig document, by using the **synfig** CLI tool or via **Synfig Studio**. The rendering code is the same, but some concepts would be easier explained using the CLI tool. A basic command to render a Synfig document using the CLI looks like this + +.. code-block:: bash + + synfig $FILE -o out.png --time=0 --width=1920 --height=1080 + +This will render only the first frame(``--time=0``) of ``$FILE`` with dimensions 1920x1080 to target *out.png*. Synfig supports rendering to multiple file formats. These file formats are represented as ``Target`` in Synfig's code base. The CLI performs the following tasks to render an image: + +* Boot Synfig by using ``synfig::Main``, which initializes different modules and systems. +* Read the document file and create the internal representation of the document. +* Extract and execute a ``Job``. + +The first two steps are not part of the Renderer. Therefore this section only covers the last step. + +Job +~~~ + +This class is present in CLI only, and it is simple. This class stores information used by other functions to start rendering the file. The most important fields it stores are ``synfig::RendDesc desc`` and a handle to the ``synfig::Canvas``. After all the initialization and reading of the document is done, the first step taken by the CLI is to create and fill a Job. + +Usually, only one Job is created, but two jobs are created if the user wants to extract Alpha to another file. These jobs are first set up and run. The setup step attempts to find the ``Target`` specified by the user or by the file extension, Render Engine to use, permissions, etc. + +To start rendering the file, ``job.target->render(..)`` is called. This function actually starts the rendering process. + +Target +~~~~~~ + +``Target`` represents the output (file or memory) and handles the frame-by-frame rendering process. The base class ``Target`` has a few virtual methods overridden by derived classes like ``Target_Scanline``. Targets for output files are modules that derive mostly from ``Target_Scanline``. Details about how the correct ``Target`` class is acquired and the actual working of ``Target::render()`` can be found in :ref:`renderer_target_surface`. + +The Cobra engine is multithreaded, executing independent Tasks on different threads. The function ``Canvas::build_rendering_task`` creates a Task for rendering a frame. This function is called by ``Target::render()``. + +Tasks +~~~~~ + +Tasks are the main objects of the Cobra engine, which writes and transforms pixels. There are tasks for blending, rendering shapes, transformation, etc. Tasks can have dependencies stored in a ``sub_tasks`` list inside the class ``Task``. ``Canvas::build_rendering_task`` builds this graph of Tasks, which is then sent to the Render Engine for execution. Details on how this Task list is build can be found in :ref:`renderer_tasks`. + +Renderer +~~~~~~~~ + +A Renderer in Synfig receives a Task list, processes it, and runs it. Synfig has multiple renderers, like Draft SW, Preview SW, etc. Currently, there are only Software Renderers in Synfig. ``Tasks`` are specialized based on the chosen renderer. This is because Software Tasks can not be directly run using GPU. The base class ``Renderer`` does most of the work. It is responsible for Optimizing the Task List, constructing the Tasks, and then sending them to the Render Queue. More details in :ref:`renderer_queue`. + +Render Queue +~~~~~~~~~~~~ + +There is a singleton Render Queue always waiting for new Tasks to run. It creates threads that are always waiting for new Tasks. More details in :ref:`renderer_queue`. diff --git a/docs/renderer_docs/optimizers.rst b/docs/renderer_docs/optimizers.rst new file mode 100644 index 0000000..7d9af21 --- /dev/null +++ b/docs/renderer_docs/optimizers.rst @@ -0,0 +1,93 @@ +.. _renderer_optimizers: + +Optimizers +========== + +Optimizers can be found in ``synfig-core/src/synfig/rendering/common/optimizer``. They derive from the base class ``Optimizer``. + +Renderer::optimize +~~~~~~~~~~~~~~~~~~ + +This function is responsible for running all the optimizations. It needs to take care of many things like handling changes to the ``Task::List`` by optimizers. The overall functions pseudocode looks like this: + +.. code-block:: cpp + + + Task::List list; // task list to optimize + + while(categories_to_process & ALL_CATEGORIES) // each category has an ID, which is used to create its bitmask. 1 << CATEGORY_ID and all ones is ALL_CATEGORIES + { + // this doesnt have to be while loop, since prepared_category >= current_category - 1 always + // but it is while in code so I kept it like this + while(prepared_category_id < current_category_id) + { + switch(++prepared_category_id) { + // if theres some step required before running the categories optimizers do it here + // example: + case SPECIALIZED: + specialize(list); break; // specialize tasks before running optimizers which work on specialized tasks + } + } + + // check if we need to process this category, if not then skip + if(!((1 << current_category_id) & categories_to_process)) + { + // reset indexes + optimizer_index = 0; + current_category_id++; + } + + Optimizer::List optimizers; // list of optimizers to run, depending on whether this category allows simultaneous run or not, if is a list of multiple optimizers or just one + + // run all for_list optimizers + for(auto opt in optimizers) + { + if(opt->for_list) + { + opt->run(params); + } + } + + // for_task are recursive unless specefied by the task after running once + bool nonrecursive = false; + + // run all for_task/for_root_task optimizers + for(auto task in list) + { + Optimizer::RunParams params; // create params from list + Renderer::optimize_recursive(optimizers, params, for_root_task ? 0 : nonrecursive ? 1 : INT_MAX); // only let it run recursively if optimizer wants + nonrecursive = false; + + task = params.ref_task; + if((task.ref_mode & Optimizer::MODE_REPEAT_LAST) == Optimizer::MODE_REPEAT_LAST) + { + // dont go next + if(!(task.ref_mode & Optimizer::MODE_RECURSIVE)) nonrecursive = true; + } + + if(!params.ref_task) remove_from_list(task); // and dont go next + + categories_to_process |= params.ref_affects_to; // optimizer can ask to re run a category, it does that by setting ref_affects_to + // only go next if the optimizer does not want to repeat optimization + } + + optimizer_index += optimizers.size(); + } + + remove_dummy(list); + +Renderer::optimize_recursive +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This function is responsible for running the optimizers on the task and its subtasks. It executes 4 main steps, + +* Call non-deep-first optimizers +* Create a ``jump`` array, where each index stores index to next non-null sub task +* While there is a sub task to optimize + * for each sub task in ``jump`` + * Call ``optimize_recursive`` on each subtask in ``jump`` + * Merge the result to ``params``, like ``ref_affects_to`` + * Remove sub task from ``jump``, unless optimizer tells to repeat +* Call deep-first optimizers + +It uses a ``ThreadPool::Group`` to run ``optimize_recursive`` on subtasks in parallel. diff --git a/docs/renderer_docs/render_queue.rst b/docs/renderer_docs/render_queue.rst new file mode 100644 index 0000000..999677e --- /dev/null +++ b/docs/renderer_docs/render_queue.rst @@ -0,0 +1,69 @@ +.. _renderer_queue: + +Renderer and Render Queue +========================= + +A renderer in Synfig is responsible to take a ``Task::List``, optimize it, specialize the tasks and then run them. Renderers apply different optimizations and settings on the tasks. For example, the LowRes SW renderer changes various settings like resolution, blur type, etc. to make the render faster. The Safe SW Renderer does no optimizations, so its slower than other renderers. + +The renderer then sends the optimized and specialized task list to the Render Queue. + +Renderer +~~~~~~~~ + +The renderer is selected by the user from Synfig Studio UI or by passing a CLI argument. + +Each renderer has some modes registers, these modes are used for specializing tasks. For example, a software renderer will register the ``TaskSW::mode_token`` like so, + +.. code-block:: cpp + + register_mode(TaskSW::mode_token.handle()); // function in Renderer class + +Renderers derive from the ``Renderer`` class, which has most of the functionality already built in. So, creating a new renderer is as simple as, + +* derive from ``Renderer`` class, +* override ``get_name()``, +* register optimizers and mode in the constructor, +* register renderer in ``Renderer::initialize_renderers``. + +Renderer::enqueue +----------------- + +This function takes a ``Task::List list`` and a ``TaskEvent finish_event``. It then makes ``finish_event`` dependent on every task in the ``list``, so once all the tasks in the ``list`` are finished executing, the ``finish_event`` task is ran and it signals the rendering as finished. + +Before inserting the ``finish_event`` into the list, this function calls two very important functions ``optimize`` and ``find_deps``. + +Renderer::Optimize +------------------ + +This function changes the ``list`` by adding/removing/modifying tasks to improve the render times. It also calls ``linearize`` and ``specialize``. More details in :ref:`render_optimizers`. + +Renderer::find_deps +------------------- + +This function fills ``deps`` and ``back_deps`` members of ``Task::RenderData`` for each task in the passed linearized task list. It first finds dependencies based on same target surface. Tasks are also depended on other tasks if their sub tasks share the target with the other task. Dependency direction is based on position in the linear list. Tasks that come later are dependend on the tasks that come earlier if they share taret. + +It also removes dependencies between tasks if their target rect is non-overlapping. + +Now sub tasks don't have the same target surface as their parent task. So by the logic above the parent task is not depended on the sub task, but in reality it should be. This is handled by ``Renderer::linearize``. + +Renderer::linearize +------------------- + +This function turns the tree of tasks into a linear list. Sub tasks are inserted before the parent task in the list, and are converted into ``TaskSurface`` in the parents ``sub_task`` list. Since sub tasks come before parent tasks in the list and the ``TaskSurface`` has the same target surface as the sub task, ``find_deps`` is able to find the dependency. + +Render Queue +~~~~~~~~~~~~ + +This class is responsible for running tasks that support multithreading and those that don't. A separate thread is for running just the tasks which don't support multithreading. ``Renderer`` initializes a static ``RenderQueue``. It creates a set number of threads and calls ``process(thread_index)`` on each thread. + +This class uses ``std::condition_variable`` for notifying other threads when new tasks are available. And it uses ``std::mutex`` when any changes are being made to the queue. + +process +------- + +This function picks up any task in the ``ready_tasks`` or ``single_ready_tasks`` (for tasks that don't allow multithreading). It does so by calling ``get()`` which waits on ``cond``. It then runs the task and calls ``done()`` after completion. + +done +---- + +This function goes through all the tasks that depend on the completed task and then removes the completed task from their dependency. If no more dependencies exist, it inserts the task to ``ready_tasks`` or ``single_ready_tasks``. After that, it calls ``cond.notify_one()`` some times, which depends on the number of tasks added to ``ready_tasks``. A similar thing is done for ``single_cond``. diff --git a/docs/renderer_docs/target_surface.rst b/docs/renderer_docs/target_surface.rst new file mode 100644 index 0000000..e3f2a0f --- /dev/null +++ b/docs/renderer_docs/target_surface.rst @@ -0,0 +1,83 @@ +.. _renderer_target_surface: + +Target and Surface +================== + +Synfig supports rendering to different file formats and uses different modules for writing to those file formats. These modules are called Targets. They inherit from the ``Target`` class and can be selected by the user or automatically (depending on the file extension). + +Selecting Target +~~~~~~~~~~~~~~~~ + +Synfig stores a dictionary(key-value pair) of all available targets with their name as the key and a factory function as value(``book``). It keeps another dictionary where the file extension to which this target can write is used as the key and the target's name as value(``ext_book``). + +Macros are used to fill these dictionaries; check ``synfig-core/src/modules/mod_imagemagick/main.cpp`` and ``synfig-core/src/synfig/module.h``. + +Rendering to a Target +~~~~~~~~~~~~~~~~~~~~~ + +Synfig calls the ``Target::render(...)`` function to start the rendering process. The function is responsible for rendering each frame and then writing the output files. Progress is reported using ``ProgressCallback`` passed as the function parameter. + +Target_Scanline +--------------- + +``Target_Scanline`` is the base class for most targets. It writes the output of Tasks to files line by line. The frame-by-frame render loop looks like this: + +.. code-block:: cpp + + do + { + frame = next_frame(t); // increase frame and time, returns 0 when no frames left + + start_frame(); + + // create a new surface + surface = new SurfaceResource(); + + // build and execute tasks + call_renderer(); + + for(int y = 0; y < height; y++) + { + Color* colorData = start_scanline(y); + // fill values from surface to colorData + end_scanline(); // finally write scanline(colorData) to file + } + + end_frame(); + } while(frame); + +The functions ``start_scanline`` and ``end_scaline`` are overridden by modules. The actual data is written to file in these functions only. + +Surface +~~~~~~~ + +See file ``synfig-core/src/synfig/surface.cpp``. + +Tasks exchange pixels using Surfaces. Tasks do not write to Targets directly. They write on Surfaces given to them by the Targets. Surfaces store actual pixel data. For OpenGL, a surface is like a Framebuffer. + +The ``Surface`` base class only declares essential virtual functions like ``create_vfunc`` for creating a new Surface of this type, ``assign_vfunc`` for assigning data from another surface to this surface, etc. + +Since the Cobra engine is multi-threaded and supports different render engines(ex. software and hardware), there are a few requirements that Surfaces must meet: + +* Reading and writing from multiple threads with proper locking mechanisms must be possible. +* There should be an easy way to convert Surfaces from one type to another. + +Thread-Safety +------------- + +Synfig ensures thread-safety of Surfaces using ``std::mutex`` and ``Glib::Threads::RWLock``. (We use ``Glib::Threads::RWLock`` because we still support C++11 and unfortunately it doesn't have the same primitive). To keep locking Surfaces simple, these are not used directly but by ``SurfaceResource::LockBase``. To safely read from a Surface, all you need to do is: + +.. code-block:: cpp + + SurfaceResource::LockRead lock(resource); // read locks the surface, unlocks on going out of scope(desctructor called) + + const Surface surface = lock->get_surface(); // calls get_surface() of SurfaceSW + +Conversion +---------- + +``SurfaceResource`` can store more than one surface. But only one of each type, i.e., when ``SurfaceResource::get_surface`` and ``SurfaceResource(surface)`` is called, it stores the surface in a map where ``surface->token`` is the key. ``surface->token`` is like a string used to distinguish/name surfaces of different types. Token is static for each surface. + +Conversion is mainly done by ``SurfaceResource::get_surface``. It takes multiple arguments, but its main job is to attempt to convert any available surfaces from the map into the requested surface type. It stores the conversion in the same map. When a lock is created, it converts the passed resource to the type argument and stores it. + +This pattern of using tokens to distinguish between types and convert from one to another can be seen multiple times in Synfig. See :ref:`renderer_tasks`; tasks use a similar pattern. diff --git a/docs/renderer_docs/tasks.rst b/docs/renderer_docs/tasks.rst new file mode 100644 index 0000000..b3533c1 --- /dev/null +++ b/docs/renderer_docs/tasks.rst @@ -0,0 +1,177 @@ +.. _renderer_tasks: + +Layers and Tasks +================ + +Synfig documents are made up of layers. Synfig supports many different types of layers. Layers also have properties like opacity and z-depth. These are important when rendering. The layers are first sorted based on their depth and then rendered. + +All the visual information under a layer is called its ``Context``. + +Context +~~~~~~~ + +See file ``synfig-core/src/synfig/context.cpp``. + +In code, ``Context`` is a const iterator over a list of layers. So it supports operators like ``++`` and ``--``. When ``Canvas::build_rendering_task`` is called, it first creates a sorted list of layers: the Context. Then it calls ``Context::build_rendering_task`` on the ``Context``. ``Context::build_rendering_task`` finds the first active layer, and calls ``Layer::build_rendering_task``, and sends ``context.get_next()`` as the Context for the layer. + +``Layer::build_rendering_task`` calls ``Layer::build_rendering_task_vfunc``, this function is overridden by layers. It returns an **abstract** task for rendering this layer and everything below it(Context). + +Tasks +~~~~~ + +See file ``synfig-core/src/synfig/rendering/task.cpp``. + +Synfig has multiple engines, and not all tasks are compatible with every engine. For example, a blur task would differ for OpenGL and software rendering. Therefore, tasks need to be specialized based on the engine used. + +Abstract tasks are converted to Real tasks before running. Abstract tasks are used just for defining the overall task list. In contrast, Real tasks are the actual implementation of Abstract tasks. For example, the clamp task is implemented using an Abstract task ``TaskClamp`` and a Real task ``TaskClampSW``. + +.. code-block:: cpp + + rendering::Task::Token TaskClamp::token( + DescAbstract("Clamp") ); + rendering::Task::Token TaskClampSW::token( + DescReal("ClampSW") ); + + +``DescAbstract`` tells the token that it is a token for an Abstract task. Whereas ``DescReal`` tells it that it is a token for a Real task ``TaskClampSW``, which implements the Abstract task ``TaskClamp``. + +Token +~~~~~ + +Before we understand the conversion, we need to understand Tokens in Synfig. Class ``Token`` in Synfig is used mainly for identification. It is used to create sort of an internal language for storing the different types used in Synfig. + +The ``Token`` class is a doubly linked list. It has ``static`` members for the first and last token. The linked list operations are handled in the Constructor and Destructor. It also stores information like parents and all parents. Tokens also have a state ``prepared``, valid only after ``prepare_vfunc`` is called. ``prepare_vfunc`` is used for extra initialization done by any derived class. +For example, ``Task::Token`` has a map called ``alternatives_``. This map is filled when ``Task::Token.prepare_vfunc`` is called. + +This initialization is started by ``Token::rebuild``, called by ``Synfig::Main``. + +In most cases, classes have a synfig static member variable called ``token``, which is redeclared by every derived class. You will also see that these classes have a virtual function ``get_token``, which is implemented by derived classes and returns the redeclared token variable. + +The base class ``Token`` doesn't store any information. The derived classes usually store more information; for example, they can store information for converting between types. + +Task::Token +----------- + +``Task::Token`` derieves from both ``Synfig::Token`` and ``Task::DescBase``. ``Task::DescBase`` is a class that stores some function pointers and other information useful during specialization. There are function pointers for creating, cloning, and converting. ``DescAbstract``, ``DescReal``, etc., are derived classes from ``DescBase``, and they assign the function pointers using the type arguments. + +Looking at the ``TaskClamp`` example again, the code above initializes ``TaskClamp::token`` using a ``DescAbstract``. An abstract can be created and cloned but not converted from another task. So, the ``convert`` function pointer is null, whereas the ``create`` function pointer is set using ``DescBase::create_func`` and the ``clone`` function pointer is set using ``DescBase::convert_func`` which copies in a case like this. ``mode`` is assigned an empty handle for abstract tasks. + +Then it initializes ``TaskClampSW::token`` using a ``DescReal``, which sets the convert function pointer using ``DescBase::convert_func``. It stores ``TaskClamp::token`` in ``abstract_task`` member variable. ``mode`` is assigned value of ``TaskClampSW::mode_token.handle()``. Then the most important step is done, in ``prepare_vfunc`` of ``Task::token`` if the task is a Real task, then its abstract task's ``alternative_`` map is filled, i.e., ``abstract_task.alternatives()[mode] = _Handle(*this)``. The abstract task is ``TaskClamp`` in this case. ``mode`` is explained in the next section. + +Then an abstract task can be easily converted to a Real task given a mode by using ``alternatives_[mode]->convert(*this)``. ``alternatives_[mode]`` is storing the token of ``TaskClampSW`` in this example. + +Specialization +~~~~~~~~~~~~~~ + +Tasks in Synfig are specialized based on ``mode``. Currently, in Synfig, there is only one mode ``TaskSW`` because there is only a software renderer. Each rendered has some modes associated with it. This gives users the ability to run only software or hardware tasks. It's not like hardware tasks cannot work with software tasks. They can work, which is possible due to automatic surface conversion. But it's faster if dependent tasks are of the same mode. + +So, how does a renderer know which task runs on which mode? Real tasks derive from the ``Mode`` class, which stores a static token for its type and some additional functions which help the renderer. Now instead of deriving directly from the ``Mode`` class, software tasks derive from the ``TaskSW`` class, which derives from the ``Mode`` class. This is so that the mode token is the same for every software task. + +Renderers register the mode they work on using the ``register_mode`` function. A software renderers calls ``register_mode(TaskSW::mode_token.handle())``. A renderer uses the registered modes to specialize tasks before sending them to the render queue. + +Now that we know, what is required to implement a task, lets learn how to do it. + +Implementation +~~~~~~~~~~~~~~ + +First, we need to create an Abstract task class. This will store all the properties necessary for executing the task. + +.. code-block:: cpp + + class MyTask : public Task + { + public: + typedef etl::handle Handle; + static Token token; + virtual Token::Handle get_token() const { return token.handle(); } + + // properties/settings + float mul; + + // virtual functions as required or redeclare as required + + MyTask() : mul(0) {} + } + +Then we need to create its software implementation. + +.. code-block:: cpp + + class MyTaskSW : public MyTask, public TaskSW + { + public: + typedef etl::handle Handle; + static Token token; + virtual Token::Handle get_token() const { return token.handle(); } + + virtual bool run(RunParams ¶ms) const; + } + +Then we need to initialize ``MyTask::token`` and ``MyTaskSW::token`` in a cpp file. + +.. code-block:: cpp + + rendering::Task::Token MyTask::token( + DescAbstract("MyTask") ); + rendering::Task::Token MyTaskSW::token( + DescReal("MyTaskSW") ); + +Implementation of ``run`` for ``ClampSW`` looks like, + +.. code-block:: cpp + + bool + TaskClampSW::run(RunParams&) const + { + RectInt r = target_rect; + if (r.valid()) + { + VectorInt offset = get_offset(); + RectInt ra = sub_task()->target_rect + r.get_min() + get_offset(); + if (ra.valid()) + { + rect_set_intersect(ra, ra, r); + if (ra.valid()) + { + LockWrite ldst(this); // lock target surface of this task + if (!ldst) return false; + LockRead lsrc(sub_task()); // lock target surface of sub_task, assumes only 1 sub task + if (!lsrc) return false; + + const synfig::Surface &a = lsrc->get_surface(); + synfig::Surface &c = ldst->get_surface(); + + for(int y = ra.miny; y < ra.maxy; ++y) + { + const Color *ca = &a[y - r.miny + offset[1]][ra.minx - r.minx + offset[0]]; + Color *cc = &c[y][ra.minx]; + for(int x = ra.minx; x < ra.maxx; ++x, ++ca, ++cc) + clamp_pixel(*cc, *ca); + } + } + } + } + + return true; + } + + +Special Tasks +~~~~~~~~~~~~~ + +There are some special tasks in Synfig, they do not do any processing but are used as utilities. Their tokens are created using ``DescSpecial``. + +TaskSurface +----------- + +This special task does nothing. It is used by the renderer for dependency. + +TaskList +-------- + +This task is used to denote a list of tasks that need to executed sequentially. + +TaskEvent +--------- + +This task is used for notifying when rendering has finished(using ``std::contidion_variable``). When ``TaskEvent::run`` is called, it signals that rendering has finished, atleast till the stage where this event was inserted.