Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: add graph building workflow #1468

Merged
merged 6 commits into from
Jun 13, 2023
Merged

ci: add graph building workflow #1468

merged 6 commits into from
Jun 13, 2023

Conversation

TheGreatRefrigerator
Copy link
Contributor

@TheGreatRefrigerator TheGreatRefrigerator commented Jun 13, 2023 •
edited
Loading

Pull Request Checklist

  • 1. I have rebased the latest version of the master branch into my feature branch and all conflicts
    have been resolved.
  • 2. I have added information about the change/addition to functionality to the CHANGELOG.md file under the
    [Unreleased] heading.
  • 3. I have documented my code using JDocs tags.
  • 4. I have removed unnecessary commented out code, imports and System.out.println statements.
  • 5. I have written JUnit tests for any new methods/classes and ensured that they pass.
  • 6. I have created API tests for any new functionality exposed to the API.
  • 7. If changes/additions are made to the ors-config.json file, I have added these to the ors config documentation
    along with a short description of what it is for, and documented this in the Pull Request (below).
  • 8. I have built graphs with my code of the Heidelberg.osm.gz file and run the api-tests with all test passing
  • 9. I have referenced the Issue Number in the Pull Request (if the changes were from an issue).
  • 10. For new features or changes involving building of graphs, I have tested on a larger dataset
    (at least Germany), and the graphs build without problems (i.e. no out-of-memory errors).
  • 11. For new features or changes involving the graphbuilding process (i.e. changing encoders, updating the
    importer etc.), I have generated longer distance routes for the affected profiles with different options
    (avoid features, max weight etc.) and compared these with the routes of the same parameters and start/end
    points generated from the current live ORS.
    If there are differences then the reasoning for these MUST be documented in the pull request.
  • 12. I have written in the Pull Request information about the changes made including their intended usage
    and why the change was needed.
  • 13. For changes touching the API documentation, I have tested that the API playground renders correctly.

Information about the changes

  • Key functionality added: workflow for building graphs for GitHub environments
  • Reason for change: make building graphs easier to trigger for the whole team

@github-actions github-actions bot added ci 👷 and removed ci 👷 labels Jun 13, 2023
@TheGreatRefrigerator
Copy link
Contributor Author

CHANGELOG.md entries missing.
Add after release is merged.

@TheGreatRefrigerator TheGreatRefrigerator force-pushed the ci/automated-graph-building branch from 88a779a to 8e194b2 Compare June 13, 2023 13:02
@TheGreatRefrigerator TheGreatRefrigerator linked an issue Jun 13, 2023 that may be closed by this pull request
Trigger graph build from github actions #1469
Closed
@TheGreatRefrigerator TheGreatRefrigerator force-pushed the ci/automated-graph-building branch from d4ff573 to 5b5f0f1 Compare June 13, 2023 13:49
@TheGreatRefrigerator
Copy link
Contributor Author

CHANGELOG.md entries missing. Add after release is merged.

Done

@github-actions github-actions bot added ci 👷 and removed ci 👷 labels Jun 13, 2023
koebi and others added 6 commits June 13, 2023 16:13
@koebi @TheGreatRefrigerator
build: add build_graph.yml workflow
c9cdae1 
The workflow can only be run on workflow_dispatch from the GitHub UI
by ors team members.

Inputs are:
- `osm_file_url`: Download URL to the OSM pbf file (best use geofabrik)
- `environment`: GitHub environment to use
- `config_in_war`: whether the war file should include the config

Check how to create an environment on:
https://heigit.atlassian.net/wiki/spaces/openrouteservice/pages/129695876/Graph+Builder+Usage

Outputs graphs, logs, elevation_cache, PBF, war and config to the
`<gh_runner_ors_repo_dir>/envs/ENV_NAME` folder on the server the GitHub runner is on.

Detailed changes in #1468

Co-authored-by: Amandus Butzer <amandus.butzer@heigit.org>
@TheGreatRefrigerator @koebi
feat(ApiConfig): add environment variables for graph building
f2f0294 
the following env vars are now available for configuring paths:
- 'GRAPHS_FOLDER': overrides 'ors.services.routing.profiles.default_params.graphs_root_path' config value
- 'ELEVATION_CACHE_FOLDER': overrides 'ors.services.routing.profiles.default_params.elevation_cache_path' config value
- 'PBF_FILE_PATH': overrides the first array entry of 'ors.services.routing.sources' config value
- 'LOGS_FOLDER': overrides 'ors.logging.location' config value

Co-authored-by: Jakob Schnell <jakob.schnell@heigit.org>
@koebi @TheGreatRefrigerator
build: add report_every parameter & custom sleep to url_check
4654251 
report_every takes the number of execution iterations on which
it should output the log entry.

Co-authored-by: Amandus Butzer <amandus.butzer@heigit.org>
@TheGreatRefrigerator
chore: add envs and ors-config-repo to .gitignore
3f2e90c
@koebi @TheGreatRefrigerator
chore: add envs folder to .dockerignore
3d37fef
@TheGreatRefrigerator @koebi
docs: add changelog entries for graph builder changes
5f1f5a7 
Co-authored-by: Jakob Schnell <jakob.schnell@heigit.org>
@TheGreatRefrigerator TheGreatRefrigerator force-pushed the ci/automated-graph-building branch from 5b5f0f1 to 5f1f5a7 Compare June 13, 2023 14:16
@github-actions github-actions bot added ci 👷 and removed ci 👷 labels Jun 13, 2023
@TheGreatRefrigerator TheGreatRefrigerator merged commit 90939b6 into master Jun 13, 2023
@TheGreatRefrigerator TheGreatRefrigerator deleted the ci/automated-graph-building branch June 13, 2023 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@koebi koebi koebi approved these changes

Assignees
No one assigned
Labels
ci 👷
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Trigger graph build from github actions
2 participants
@TheGreatRefrigerator @koebi