Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resolver parallelization #225

Merged
merged 6 commits into from
Feb 13, 2018
Merged

Resolver parallelization #225

merged 6 commits into from
Feb 13, 2018

Commits on Feb 13, 2018

  1. Add locks module for postgres advisory locking

    @msimacek
    msimacek committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    bff095b View commit details
    Browse the repository at this point in the history

  2. [backend] Keep connections across commits 

    SQLA returns connections to the connection pool after a commit/rollback.
That makes usage of session-bound locks tricky, as the following
statements may not get the same connection (and thus the same session in
postgres sense). This keeps a connection in KoscheiBackendSession across
commits.
    @msimacek
    msimacek committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    36886b5 View commit details
    Browse the repository at this point in the history

  3. [repo_resolver] Use advisory locks 

    Ensure that only one collection is being processed at a time.
    @msimacek
    msimacek committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    95d756f View commit details
    Browse the repository at this point in the history

  4. [test] Add simple test for locking

    @msimacek
    msimacek committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    a4fc659 View commit details
    Browse the repository at this point in the history

  5. [build_resolver] Allow parallel execution 

    One resolver process locks single repo ID. Different processes can
resolve different repo IDs. More granular parallelism would likely be
counterproductive as the repo setup is expensive.
    @msimacek
    msimacek committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    32d19c1 View commit details
    Browse the repository at this point in the history

  6. [resolver] Make dependency_cache.get_or_create_nevra concurrency-safe

    @msimacek
    msimacek committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    1ea3e5d View commit details
    Browse the repository at this point in the history