Complete input argument type parsing

[paleolimbot]

Basically, need to figure out which R objects can be interpreted as QGIS input types. I've done a bit of this (e.g., for sf, raster, numeric, character, and logical). It's always possible to pass a length-one character to represent these in a pinch, but it would be nice to enable more R-like objects where possible (e.g., sf::bbox, raster::extent, sf::st_crs()). A calculated table of argument types and the first 20 algorithms for each:

qgis_type	algorithms
aggregates	native:aggregate
band	native:equaltofrequency, native:fillnodata, native:fuzzifyrastergaussianmembership, native:fuzzifyrasterlargemembership, native:fuzzifyrasterlinearmembership, native:fuzzifyrasternearmembership, native:fuzzifyrasterpowermembership, native:fuzzifyrastersmallmembership, native:greaterthanfrequency, native:lessthanfrequency, native:pixelstopoints, native:pixelstopolygons, native:rasterlayerstatistics, native:rasterlayeruniquevaluesreport, native:rasterlayerzonalstats, native:rasterlayerzonalstats, native:rastersurfacevolume, native:reclassifybylayer, native:reclassifybytable, native:rescaleraster
boolean	native:addautoincrementalfield, native:addautoincrementalfield, native:angletonearest, native:atlaslayouttoimage, native:atlaslayouttoimage, native:atlaslayouttopdf, native:atlaslayouttopdf, native:atlaslayouttopdf, native:atlaslayouttopdf, native:buffer, native:cellstatistics, native:centroids, native:dropmzvalues, native:dropmzvalues, native:equaltofrequency, native:extractbyextent, native:geometrybyexpression, native:geometrybyexpression, native:greaterthanfrequency, native:highestpositioninrasterstack
color	qgis:tilesxyzdirectory, qgis:tilesxyzmbtiles
coordinateoperation	native:reprojectlayer
crs	native:addxyfields, native:assignprojection, native:bookmarkstolayer, native:createconstantrasterlayer, native:creategrid, native:createpointslayerfromtable, native:createrandombinomialrasterlayer, native:createrandomexponentialrasterlayer, native:createrandomgammarasterlayer, native:createrandomgeometricrasterlayer, native:createrandomnegativebinomialrasterlayer, native:createrandomnormalrasterlayer, native:createrandompoissonrasterlayer, native:createrandomuniformrasterlayer, native:mergevectorlayers, native:printlayoutmapextenttolayer, native:randompointsinextent, native:reprojectlayer, native:tinmeshcreation, gdal:assignprojection
distance	native:angletonearest, native:arrayoffsetlines, native:buffer, native:creategrid, native:creategrid, native:creategrid, native:creategrid, native:dbscanclustering, native:densifygeometriesgivenaninterval, native:extendlines, native:extendlines, native:interpolatepoint, native:joinbynearest, native:linedensity, native:linedensity, native:linesubstring, native:linesubstring, native:multiringconstantbuffer, native:offsetline, native:pointsalonglines
enum	native:addfieldtoattributestable, native:arrayoffsetlines, native:atlaslayouttoimage, native:atlaslayouttopdf, native:bookmarkstolayer, native:buffer, native:buffer, native:cellstatistics, native:combinestyles, native:createconstantrasterlayer, native:creategrid, native:createrandombinomialrasterlayer, native:createrandomexponentialrasterlayer, native:createrandomgammarasterlayer, native:createrandomgeometricrasterlayer, native:createrandomnegativebinomialrasterlayer, native:createrandomnormalrasterlayer, native:createrandompoissonrasterlayer, native:createrandomuniformrasterlayer, native:detectvectorchanges
execute_sql	qgis:executesql
expression	native:addautoincrementalfield, native:atlaslayouttoimage, native:atlaslayouttoimage, native:atlaslayouttoimage, native:atlaslayouttopdf, native:atlaslayouttopdf, native:extractbinary, native:extractbyexpression, native:fieldcalculator, native:geometrybyexpression, native:layertobookmarks, native:layertobookmarks, native:orderbyexpression
extent	native:createconstantrasterlayer, native:creategrid, native:createrandombinomialrasterlayer, native:createrandomexponentialrasterlayer, native:createrandomgammarasterlayer, native:createrandomgeometricrasterlayer, native:createrandomnegativebinomialrasterlayer, native:createrandomnormalrasterlayer, native:createrandompoissonrasterlayer, native:createrandomuniformrasterlayer, native:extenttolayer, native:extractbyextent, native:randompointsinextent, native:rasterize, native:writevectortiles_mbtiles, native:writevectortiles_xyz, gdal:cliprasterbyextent, gdal:clipvectorbyextent, gdal:rasterize, gdal:warpreproject
field	native:addautoincrementalfield, native:adduniquevalueindexfield, native:collect, native:countpointsinpolygon, native:countpointsinpolygon, native:createattributeindex, native:createpointslayerfromtable, native:createpointslayerfromtable, native:createpointslayerfromtable, native:createpointslayerfromtable, native:explodehstorefield, native:extractbinary, native:extractbyattribute, native:fieldcalculator, native:hublines, native:hublines, native:joinattributestable, native:joinattributestable, native:linedensity, native:meancoordinates
fields_mapping	native:refactorfields
file	native:atlaslayouttoimage, native:importphotos, native:repairshapefile, native:setlayerstyle, native:shpencodinginfo, gdal:colorrelief, grass7:i.atcorr, grass7:i.cca, grass7:i.cluster, grass7:i.in.spotvgt, grass7:i.maxlik, grass7:i.smap, grass7:m.cogo, grass7:r.category, grass7:r.colors, grass7:r.in.lidar, grass7:r.in.lidar.info, grass7:r.li.cwed, grass7:r.li.cwed, grass7:r.li.cwed.ascii
fileDestination	native:atlaslayouttopdf, native:combinestyles, native:filedownloader, native:nearestneighbouranalysis, native:package, native:printlayouttoimage, native:printlayouttopdf, native:rasterlayerstatistics, native:rasterlayeruniquevaluesreport, native:rastersurfacevolume, native:savefeatures, native:stylefromproject, native:tinmeshcreation, native:writevectortiles_mbtiles, gdal:gdal2xyz, gdal:gdalinfo, gdal:ogrinfo, grass7:i.cluster, grass7:i.cluster, grass7:i.gensig
folderDestination	native:extractbinary, native:splitvectorlayer, native:writevectortiles_xyz, gdal:gdal2tiles, gdal:retile, grass7:i.aster.toar, grass7:i.cca, grass7:i.landsat.toar, grass7:i.pca, grass7:i.tasscap, grass7:i.topo.corr, grass7:r.colors, grass7:r.horizon, grass7:r.series.interp, grass7:r.stats.quantile.rast, grass7:r.texture, grass7:r.tile, qgis:hypsometriccurves, qgis:pointstopath, qgis:tilesxyzdirectory
layer	native:polygonfromlayerextent, native:setlayerstyle
layout	native:atlaslayouttoimage, native:atlaslayouttopdf, native:printlayoutmapextenttolayer, native:printlayouttoimage, native:printlayouttopdf
layoutitem	native:printlayoutmapextenttolayer
maptheme	native:rasterize
matrix	native:reclassifybytable, saga:orderedweightedaveraging, saga:reclassifyvalues, saga:reclassifyvaluessimple, saga:reclassifyvaluestable
multilayer	native:calculatevectoroverlaps, native:cellstatistics, native:combinestyles, native:equaltofrequency, native:greaterthanfrequency, native:highestpositioninrasterstack, native:lessthanfrequency, native:lowestpositioninrasterstack, native:mergevectorlayers, native:package, native:rasterbooleanand, native:rasterize, native:rasterlogicalor, gdal:buildvirtualraster, gdal:buildvirtualvector, gdal:merge, gdal:retile, gdal:tileindex, grass7:i.albedo, grass7:i.cluster
number	native:addautoincrementalfield, native:addfieldtoattributestable, native:addfieldtoattributestable, native:arrayoffsetlines, native:arrayoffsetlines, native:arrayoffsetlines, native:arraytranslatedfeatures, native:aspect, native:buffer, native:bufferbym, native:cellstatistics, native:converttocurves, native:converttocurves, native:createconstantrasterlayer, native:createconstantrasterlayer, native:createrandombinomialrasterlayer, native:createrandombinomialrasterlayer, native:createrandombinomialrasterlayer, native:createrandomexponentialrasterlayer, native:createrandomexponentialrasterlayer
point	native:pointtolayer, native:rotatefeatures, native:serviceareafrompoint, native:shortestpathlayertopoint, native:shortestpathpointtolayer, native:shortestpathpointtopoint, native:shortestpathpointtopoint, gdal:viewshed, grass7:m.cogo, grass7:r.horizon.height, grass7:r.lake, grass7:r.spreadpath, grass7:r.viewshed, grass7:r.water.outlet, grass7:v.mkgrid, grass7:v.net.visibility
range	grass7:r.rescale, grass7:r.rescale, grass7:r.rescale.eq, grass7:r.rescale.eq, grass7:v.in.lidar
raster	native:aspect, native:cellstatistics, native:equaltofrequency, native:fillnodata, native:fuzzifyrastergaussianmembership, native:fuzzifyrasterlargemembership, native:fuzzifyrasterlinearmembership, native:fuzzifyrasternearmembership, native:fuzzifyrasterpowermembership, native:fuzzifyrastersmallmembership, native:generatepointspixelcentroidsinsidepolygons, native:greaterthanfrequency, native:highestpositioninrasterstack, native:hillshade, native:lessthanfrequency, native:lowestpositioninrasterstack, native:pixelstopoints, native:pixelstopolygons, native:rasterbooleanand, native:rasterlayerstatistics
raster_calc_expression	qgis:rastercalculator
rasterDestination	native:aspect, native:cellstatistics, native:createconstantrasterlayer, native:createrandombinomialrasterlayer, native:createrandomexponentialrasterlayer, native:createrandomgammarasterlayer, native:createrandomgeometricrasterlayer, native:createrandomnegativebinomialrasterlayer, native:createrandomnormalrasterlayer, native:createrandompoissonrasterlayer, native:createrandomuniformrasterlayer, native:equaltofrequency, native:fillnodata, native:fuzzifyrastergaussianmembership, native:fuzzifyrasterlargemembership, native:fuzzifyrasterlinearmembership, native:fuzzifyrasternearmembership, native:fuzzifyrasterpowermembership, native:fuzzifyrastersmallmembership, native:greaterthanfrequency
relief_colors	qgis:relief
sink	native:addautoincrementalfield, native:addfieldtoattributestable, native:adduniquevalueindexfield, native:adduniquevalueindexfield, native:addxyfields, native:affinetransform, native:aggregate, native:angletonearest, native:antimeridiansplit, native:arrayoffsetlines, native:arraytranslatedfeatures, native:assignprojection, native:bookmarkstolayer, native:boundary, native:boundingboxes, native:buffer, native:bufferbym, native:calculatevectoroverlaps, native:centroids, native:clip
source	native:addautoincrementalfield, native:addfieldtoattributestable, native:adduniquevalueindexfield, native:addxyfields, native:affinetransform, native:aggregate, native:angletonearest, native:angletonearest, native:antimeridiansplit, native:arrayoffsetlines, native:arraytranslatedfeatures, native:assignprojection, native:boundary, native:boundingboxes, native:buffer, native:bufferbym, native:calculatevectoroverlaps, native:centroids, native:clip, native:clip
string	native:addautoincrementalfield, native:addfieldtoattributestable, native:adduniquevalueindexfield, native:addxyfields, native:angletonearest, native:countpointsinpolygon, native:dbscanclustering, native:dbscanclustering, native:explodehstorefield, native:extractbyattribute, native:extractmvalues, native:extractspecificvertices, native:extractzvalues, native:fieldcalculator, native:filedownloader, native:intersection, native:joinattributesbylocation, native:joinattributestable, native:joinbynearest, native:kmeansclustering
tininputlayers	native:tinmeshcreation
vector	native:atlaslayouttoimage, native:atlaslayouttopdf, native:createattributeindex, native:createspatialindex, native:flattenrelationships, native:saveselectedfeatures, native:setlayerencoding, native:truncatetable, gdal:ogrinfo, qgis:definecurrentprojection, qgis:importintospatialite
vectorDestination	gdal:buffervectors, gdal:buildvirtualvector, gdal:contour, gdal:contour_polygon, gdal:convertformat, gdal:dissolve, gdal:executesql, gdal:offsetcurve, gdal:onesidebuffer, gdal:pointsalonglines, gdal:polygonize, gdal:tileindex, grass7:r.carve, grass7:r.contour, grass7:r.drain, grass7:r.flow, grass7:r.random, grass7:r.resamp.bspline, grass7:r.sim.sediment, grass7:r.sim.water
vectortilewriterlayers	native:writevectortiles_mbtiles, native:writevectortiles_xyz

[paleolimbot]

(edits by @florisvdh: references added to #13 (comment))

• source (character, sf work)
• sink (defaults to qgis_tmp_vector()
• raster (character, raster work)
• band (1-based integer works)
• boolean (logical works)
• string (character works)
• rasterDestination (defaults to qgis_tmp_raster())
• crs need to interpret st_crs() and raster::crs()/raster::projection()
• distance (Complete input argument type parsing #13 (comment))
• field (character works, multiple values separated by ';')
• vectorDestination (defaults to qgis_tmp_vector()
• multilayer (character probably works, scalar sf/raster works) (Complete input argument type parsing #13 (comment))
• enum (character and integer work)
• extent need to to interpret st_bbox() and raster::extent()
• number (numeric/integer work)
• file (character works_
• folderDestination (defaults to qgis_tmp_folder()
• fileDestination (defaults to qgis_tmp_file("")
• vector (sf works)
• point need to interpret sf::st_point() or length-2 numeric
• range (Complete input argument type parsing #13 (comment))
• expression need to convert rlang::quo() (Complete input argument type parsing #13 (comment))
• aggregates need to convert rlang::quo() (Complete input argument type parsing #13 (comment))
• layout (character probably works)
• layer (sf/raster work)
• layoutitem (character probably works)
• maptheme (character probably works)
• matrix need to interpret matrix() (Complete input argument type parsing #13 (comment))
• fields_mapping need to interpret named character (Complete input argument type parsing #13 (comment))
• coordinateoperation need to interpret st_crs()
• tininputlayers (Complete input argument type parsing #13 (comment))
• vectortilewriterlayers (Complete input argument type parsing #13 (comment))
• execute_sql (character probably works, probably overkill to translate SQL from quo)
• raster_calc_expression need to convert rlang::quo() (Complete input argument type parsing #13 (comment))
• relief_colors (Complete input argument type parsing #13 (comment))
• color (Complete input argument type parsing #13 (comment))

[nyalldawson]

Some tips:
-crs: I'd suggest converting these to a wkt2 representation of the crs and passing that to qgis_process. It's the most straightforward way to get a lossless representation of a crs.

• co-ordinate operation: this is always a proj string representing a co-ordinate operation, not a crs itself. Unless I'm missing something st_crs doesn't apply here.
• color: easiest way would be to pass a hex encoded color value

[paleolimbot]

Thanks!

[paleolimbot]

We're having trouble specifying multilayer inputs (#25)...comma-separated filenames and multiple --INPUT doesn't seem to work. Are we missing something obvious?

[JanCaha]

crs, extent and point should be done and can be checked out.

distance should quite likely be interpreted as a number. In QGIS GUI it is linked to specific layer and recalculates the distance based on layer CRS and units sets for this parameter. I don't think this is achievable from command line.

[florisvdh]

Some thoughts while preparing to tackle potential remaining opportunities here.

as_qgis_argument() deals with several aspects:

1. always: handle R object passed as input argument in a way that depends on the R object's class, resulting in a specifically formatted output that will be accepted by qgis_process (through qgis_serialize_arguments()), distinguishing between JSON-input and no-JSON-input methods.
2. sometimes: also take into account the qgis_type property of the argument, either to handle the R object differently (only seen for character class currently), or to restrict acceptance of this R object class to specific values of qgis_type.

For now, I intend to focus on aspect 1, especially to look after interesting R object classes which we still don't handle or handle correctly. Handling aspect 2 more completely, especially for the acceptance part, would potentially require lots of 'class - qgis_type' combinations to handle in R, with future maintenance burden, while qgis_process itself raises an error when an input is not accepted (e.g. when passing "abc" to the distance argument in native:buffer, while "10" will be OK). Dispatching such checks to qgis_process can be considered sufficient for now.

[florisvdh]

Handled the following in branch args:

• since QGIS 3.30.0, qgis_types distance, number, boolean, string and color can also take a field or QGIS expression string (aka a data-defined override). Because in R this will be represented as a (specifically formatted) string, those cases are handled well already. Cf. the vignette on using QGIS expressions.
• distance usually expects numeric: current handling by as_qgis_argument() of numeric input is sufficient.
• multilayer is sufficiently implemented for the vector case since multiple vector layers in R (even if belonging to same dsn) are always represented as separate R objects, which is also required for the multilayer input. Passing multiple layers is done either by repetition of the same multilayer argument (providing one layer each time) or by wrapping the layers in qgis_list_input().
  • BTW it doesn't work well if passing single multi-layer files (as filepath), e.g. a vector GeoPackage or a multi-band raster; in both cases only the first layer will be used by qgis_process. This understanding of 'multilayer' appears to match more the QGIS meaning of a 'single layer', i.e. an entry in the 'Layers' panel (such a layer can still be a multiband raster).
  • While filepath arguments are the user's responsibility, multiband raster objects could be taken care of automatically by splitting them before feeding to the multilayer argument, but that would essentially not match the QGIS GUI behaviour. Such cases will raise a warning (08abc26, 118c09e, 3ebb604) and solution is left to the user, in order not to override QGIS behaviour.
• range: QGIS expects "min,max" format. This can be provided directly as a string in R. The no-JSON-input approach in the numeric method of as_qgis_argument() will also convert a c(<min>, <max>) vector to the expected string format already. While with the JSON input approach the vector is turned into a JSON array, the latter appears to also be correctly processed by qgis_process! So vectors are perfect.
• expression is always a string and this is supported well (see the vignette on using QGIS expressions)
• matrix: comparable to range; must be a single string of comma-separated values, and this is already achieved for a vector. However the matrix argument type in the QGIS GUI has some table structure, e.g. in native:reclassifybytable this has three columns (Minimum, Maximum, Value). The comma-separated string (vector) is derived by concatenating all rows and dropping column names. Clearly it's a bonus to provide similar handling in R of a matrix or dataframe object. Provided by e702c46 and 545f69c respectively.
• aggregates is only supported with JSON input, where it is represented as a nested object. Nested list objects prepared with the qgis_list_input() and qgis_dict_input() helpers can handle this well.
• fields_mapping: same case as aggregates (only supported with JSON input, where it is represented as a nested object).
• tininputlayers: same case as aggregates (only supported with JSON input, where it is represented as a nested object).
• vectortilewriterlayers: same case as aggregates (only supported with JSON input, where it is represented as a nested object).
• raster_calc_expression: same as expression (string)
• relief_colors: the expected format is a string of numeric values with commas and semi-colons, composed as "min1, max1, r1, g1, b1;min2, max2, r2, g2, b2;<etc>". Each set of values delimited by semi-colons represents an interval (min & max) with a color value as RGB. So we could allow various types of R objects here. Unless the string is directly provided, it has been chosen to only accept a dataframe or matrix with three columns, the third column being an R color string that can be accepted by col2rgb(alpha = FALSE).
• color: always provided as a colour string to QGIS (hexadecimal representation starting with "#" or a function string "rgba(redval, greenval, blueval, alpha)"). This has been exploited to accept R colors in any string representation that col2rgb(alpha = TRUE) supports, always converting to the "rgba(...)" function string format.