Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel code refinement #13

Closed
matamadio opened this issue Oct 17, 2022 · 2 comments
Closed

Parallel code refinement #13

matamadio opened this issue Oct 17, 2022 · 2 comments
Assignees

Comments

@matamadio
Copy link
Collaborator

matamadio commented Oct 17, 2022

Parallelization WORKS on Linux and Win! Thanks @artessen and @ConnectedSystems for this magic!

Remains some issues to solve:

  • Does work for function, but not for classes
  • The code can be more efficient: we don't need EAI calculation as a raster. EAI calculation is done on the table output, after zonal aggregation, and presented as output chart. See Plot Exceedance Frequency Curve #2
@matamadio
Copy link
Collaborator Author

  • The zonal step seem to introduce an overestimation error ONLY for total Pop; when tested on DOM against original code and QGIS zonal, the error is between 1% and 50%. The raster creation is checked to be correct according to exported byproducts.
     with mp.Pool(cores) as p:
        # Get total exposure for each ADM region
        func = partial(zonal_stats_partial, raster=exp_ras, stats="sum")
        stats_parallel = p.map(func, np.array_split(adm_data.geometry, cores))

    exp_per_ADM = list(it.chain(*stats_parallel))

immagine

@matamadio
Copy link
Collaborator Author

Thanks Arthur for refining the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants