software_architecture.md

Software Architecture

Now that we know more about the concepts in Pyblish, let's take a closer look at it fits together in technical terms.

Overview

From afar, Pyblish consists of three major components.

The core is designed and developed in isolation and runs without supporting libraries or dependencies. The remainder is optional but useful, such as a graphical user interface along with the glue involved in bridging a host, such as Autodesk Maya, to Pyblish.

The integration consists of a server and a series of plug-ins, both of which we will talk more about later.

Module structure

To the user, Pyblish is an executable they run to publish things. To you, there are many decoupled modules involved in making Pyblish work and in making it maintainable long term.

Each module is versioned using Semantic Versioning and developed independently. Each hosting source code, issue tracker and wiki.

Here are some examples of modules and packages.

	Module	Description
	pyblish-win	Officially supported modules and binaries for Windows (package)
	pyblish-linux	Officially supported modules and binaries for Linux (package)
	pyblish-osx	Officially supported modules and binaries for Linux (package)
	pyblish	Officially supported modules (package)
	pyblish-base	Core module, the heart of Pyblish.
	pyblish-qml	User interface module, the face of Pyblish.
	pyblish-tray	The Pyblish control panel
	pyblish-rpc	Communication bridge between the core and user interface of Pyblish.
	pyblish-integration	Supporting module for pyblish-rpc
	pyblish-maya	Integration module for Autodesk Maya
	pyblish-houdini	Integration module for SideFx Houdini
	pyblish-ci	Continuous integration server for Pyblish.
	pyblish-event	Reference implementation of a cloud-based event monitor.

Source structure

The following is the file structure of the core Pyblish module.

	Module	Description
	plugins	Default plug-ins.
	vendor	Third party dependencies (no external dependencies).
	__init__.py	Global, private variables.
	__main__.py	Making this package executable.
	api.py	The developer facing interface to Pyblish.
	cli.py	Command-line interface, written with the Click support library.
	compat.py	Backwards and forward-compatibility features.
	lib.py	Helper functions used across surrounding Python modules.
	util.py	Convenience module for publishing via scripting.
	version.py	The semantic version of this Pyblish module.

And finally, the most relevant files in terms of developing for Pyblish:

	Module	Description
	plugin.py	Plug-in definition, registration and discovery on disk.
	logic.py	The brains of processing.

Processing Pipeline

As soon as the user hits "publish", three things happen.

1. Create
2. Sort
3. Run

Or, more specifically.

1. A new Context is instantiated and plug-ins discovered.
2. logic.process determines which plug-in to run next.
3. plugin.process performs the actual processing.

The logic module is used primarily in the interfaces with which a user interacts, namely the command-line interface and graphical user interface. It only handles delegating of tasks to functions capable of producing results. In this case, a literal dictionary called results.

We'll see a bit later how this works with inter-process communication.

plugin.process then is the brains of the operation and is the function actually running your plug-in, passing along the currently active Instance. Its responsibility is to..

1. Intercept log messages
2. Handle exceptions
3. Format the resulting data
4. Build the results dictionary.

During processing, the context may be extended with Instance's. These may be added at any part of the pipeline, but only subsequent plug-ins will see them, which is why they are typically added as early as possible in what is commonly referred to as "Collection". By the end of processing, a results dictionary is created and stored within the Context.

The results dictionary is appended to a list within the Context - accessible as data["results"] - and captures information regarding the order in which plug-ins and instances are processed and all messages therein. It is primarily intended for graphical user interfaces to visualise the events that occur during processing, but is open to developers to produce visualisations of their own. For example you may be interested in storing the results in a log somewhere on the cloud for auditing purposes such that one may "go back in time" and inspect what actually went on back then.

{{ file.mtime }}