New standalone console tool for running processing algorithms (Request in QGIS)

[qgis-bot]

Request for documentation

From pull request qgis/QGIS#34617
Author: @nyalldawson
QGIS version: 3.14 (Feature)

New standalone console tool for running processing algorithms

PR Description:

UPDATE: the final tool is called qgis_process, all examples here have been updated to reflect the change

This new qgis_transform qgis_process tool allows users to run processing algorithms (both built-in, and those provided by plugins) directly from the console.

Running:

• qgis_process list will output a complete list of all available algorithms, grouped by provider.
• qgis_process plugins lists available and activated plugins which advertise the hasProcessingProvider metadata option (only these plugins are loaded by the tool)
• qgis_process help algid outputs the help and input descriptions for the specified algorithm, e.g. qgis_process help native:centroids

qgis_process run: runs an algorithm. Parameters are specified by a
--param=value syntax. E.g.

qgis_process run native:centroids --INPUT="my_shapefile.shp" --OUTPUT="centroids.kml"

or

qgis_process run native:buffer --INPUT=/home/me/my.shp --DISTANCE=20 --OUTPUT=/home/me/buffered.shp

While running an algorithm a text-based feedback bar is shown, and the operation can be cancelled via CTRL+C

Sponsored by the Swedish User Group

More detail is available in qgis/QGIS-Enhancement-Proposals#140

Still to come (follow up PRs):

• running a model directly from a model file
• running a script directly from an algorithm script file
• using the .env variable file approach when running under windows

Commits tagged with [need-docs] or [FEATURE]

"[FEATURE][processing] New standalone console tool for running processing algorithms\n\nThis new qgis_transform tool allows users to run processing algorithms\n(both built-in, and those provided by plugins) directly from the console.\n\nRunning:\n\n- "qgis_transform list" will output a complete list of all available\nalgorithms, grouped by provider.\n- "qgis_transform plugins" lists available and activated plugins which\nadvertise the hasProcessingProvider metadata option (only these plugins\nare loaded by the tool)\n- "qgis_transform help algid" outputs the help and input descriptions\nfor the specified algorithm, e.g. "qgis_transform help native:centroids"\n\n"qgis_transform run": runs an algorithm. Parameters are specified by a\n"--param=value" syntax. E.g.\n\n qgis_transform run native:centroids --INPUT="my_shapefile.shp" --OUTPUT="centroids.kml"\n\nor\n\n qgis_transform run native:buffer --INPUT=/home/me/my.shp --DISTANCE=20 --OUTPUT=/home/me/buffered.shp\n\nWhile running an algorithm a text-based feedback bar is shown, and the\noperation can be cancelled via CTRL+C\n\nSponsored by the Swedish User Group" "[FEATURE] Allow running model files direct from standalone qgis_process tool"

[havatv]

@nyalldawson, I tried to test this with master (compiled from source on Ubuntu 18.04) just now

QGIS version | 3.13.0-Master | QGIS code revision | da042bb61d

When trying in the Python console, i get:

>>> qgis_process
Traceback (most recent call last):
  File "/usr/lib/python3.6/code.py", line 91, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
NameError: name 'qgis_process' is not defined

Missing import?

[nyalldawson]

It's not a python tool. It's a standalone console tool, similar to "ogr2ogr" .

[havatv]

It's not a python tool. It's a standalone console tool, similar to "ogr2ogr" .

Yep, that worked

bin/qgis_process list | wc
   1065    3626   46358

[havatv]

So, now we have a console tool to document. Where to place the documentation? A new appendix to the user manual?
What do you say @ghtmtt, @DelazJ?

There is also the qgis_bench tool, that has been around for some time (pre 2013).

[nyalldawson]

There is also the qgis_bench tool, that has been around for some time (pre 2013).

I wouldn't bother with this one -- it's more of a developer focused testing tool vs something which is useful for end users

[havatv]

I wouldn't bother with this one -- it's more of a developer focused testing tool vs something which is useful for end users

Something for the developers guide?

[nyalldawson]

Not even that -- it's even "rawer" then that!

[SrNetoChan]

Let me tell you something @nyalldawson , I know a few user quite exited with this new tool. I think it's worth putting the instructions in the users manual.

Since It's focused on using processing algorithms, I believe that's the place to add information about it.

[nyalldawson]

@SrNetoChan I'm referring to the "qgis_bench" tool above!

[SrNetoChan]

AH :-| That makes more sense.

[havatv]

Since It's focused on using processing algorithms, I believe that's the place to add information about it.

Seems sensible, but should we also have a section for QGIS command-line utilities/tools? Are there more of them coming up?

[havatv]

Not even that -- it's even "rawer" then that!

qgis_bench does not seem to be included in the "standard distributions", so I guess that means that we do not need to document it.

[ghtmtt]

So, now we have a console tool to document. Where to place the documentation? A new appendix to the user manual?
What do you say @ghtmtt, @DelazJ?

I think the user manual is the best place. We can have a dedicated chapter in Processing with the basic commands and the basic syntax.