Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC 96 implementation: Deferred C++ plugin loading #8695

Merged
merged 90 commits into from
Nov 21, 2023

Conversation

rouault
Copy link
Member

@rouault rouault commented Nov 12, 2023

Implements #8648

  • All drivers that can be built as plugins and depends on external libraries have been converted to use the deferred plugin loading capability. Plus a few other drivers that only depend on core
  • The PNG, JPEG, GIF, MRF, NITF and planetary (PDS, PDS4, ISIS2, ISIS3, VICAR) drivers can also now be built as plugins (when not using the in-tree vendored libraries such as libpng, libjpeg, etc.), and use deferred plugin loading
  • The "raw" drivers can also now be built as plugins, but not as deferred plugin loading (doable, but significant effort required)

On my full configuration with all drivers that can be built as plugins effectively built as plugins (that is those that only depend on libgdal and those that depend on third-party libraries, with -DGDAL_ENABLE_PLUGINS=ON and -DGDAL_ENABLE_PLUGINS_NO_DEPS=ON), the initialization time has dropped from ~300 ms to 70 ms. If building only as plugins drivers that depend on third-party libraries with -DGDAL_ENABLE_PLUGINS=ON (which is the config which makes most sense), it drops to 26 ms, with all 55 plugins in deferred loading mode. One of the biggest culprit identified during the conversion process was the DGNv8 / DWG driver with the gazillions shared libraries of the Teigha SDK.

@coveralls
Copy link
Collaborator

coveralls commented Nov 14, 2023

Coverage Status

coverage: 67.862% (-0.007%) from 67.869%
when pulling 9d201ef on rouault:deferred_plugin
into 7d5725c on OSGeo:master.

gcore/gdaldriver.cpp Outdated Show resolved Hide resolved
gcore/gdaldrivermanager.cpp Outdated Show resolved Hide resolved
…FROM to the list of items that must be set on the proxy
@rouault
Copy link
Member Author

rouault commented Nov 21, 2023

Following comment added in GDALOpenEx():

    // Lookup of matching driver for dataset can involve up to 2 passes:
    // - in the first pass, all drivers that are compabile of the request mode
    //   (raster/vector/etc.) are probed using their Identify() method if it
    //   exists. If the Identify() method returns FALSE, the driver is skipped.
    //   If the Identify() methods returns GDAL_IDENTIFY_UNKNOWN and that the
    //   driver is a deferred-loading plugin, it is added to the
    //   apoSecondPassDrivers list for potential later probing, and execution
    //   continues to the next driver in the list.
    //   Otherwise if Identify() returns non-FALSE, the Open() method is used.
    //   If Open() returns a non-NULL dataset, the loop stops and it is
    //   returned. Otherwise looping over remaining drivers continues.
    // - the second pass is optional, only if at least one driver was added
    //   into apoSecondPassDrivers during the first pass. It is similar
    //   to the first pass except it runs only on apoSecondPassDrivers drivers.
    //   And the Open() method of such drivers is used, causing them to be
    //   loaded for real.

@rouault rouault merged commit 8821682 into OSGeo:master Nov 21, 2023
30 of 31 checks passed
jef-n added a commit to jef-n/OSGeo4W that referenced this pull request Dec 4, 2023
' update proj-data to 1.16
* update gdal to 3.8.1
* grass*: run clean
* fix failed build of gdal-dev (probably after OSGeo/gdal#8695)
* grass and qgis rebuilds also fix qgis/QGIS#55476
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants