Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Lua 5.4 in Docker image #6346

Merged
merged 2 commits into from
Aug 28, 2022
Merged

Use Lua 5.4 in Docker image #6346

merged 2 commits into from
Aug 28, 2022

Conversation

SiarheiFedartsou
Copy link
Member

Issue

Updates Lua in Dockerfile to the latest 5.4 version. Some benchmarks say it is up to 40% faster, but I got only improvement from ~8.1s to ~7.9s on my M1 Pro for time docker run -t -v "${PWD}:/data" osrm-test osrm-extract -p /opt/car.lua /data/berlin-latest.osm.pbf.

Tasklist

Requirements / Relations

Link any requirements here. Other pull requests this PR is based on?

@mjjbell
Copy link
Member

mjjbell commented Aug 28, 2022

Is that total running time for osrm-extract or just for the scripting stage?
I think it would be good to run it on a continental sized dataset to get a better gauge of performance.

sol2 is likely to have a big impact on the scripting performance in any case.

@SiarheiFedartsou
Copy link
Member Author

SiarheiFedartsou commented Aug 28, 2022 •
edited
Loading

Is that total running time for osrm-extract or just for the scripting stage? I think it would be good to run it on a continental sized dataset to get a better gauge of performance.

sol2 is likely to have a big impact on the scripting performance in any case.

It is total running time of osrm-extract. Tbh I had no goal to do it and have some significant performance improvement after update - the goal here is probably to just update dependency. I just noticed these promises and decided to check the difference just out of curiousity not spending more than 10 minutes on it. Do you think we should invest more time into it?
Btw as per sol2 docs they have really small overhead over C API https://sol2.readthedocs.io/en/latest/benchmarks.html

@SiarheiFedartsou SiarheiFedartsou merged commit 06719be into master Aug 28, 2022
datendelphin added a commit to fossgis-routing-server/osrm-backend that referenced this pull request Dec 18, 2022
@datendelphin
Merge tag 'v5.27.0'
c83c8c5 
v5.27.0

- Changes from 5.26.0
    - API:
      - ADDED: Add Flatbuffers support to NodeJS bindings. [Project-OSRM#6338](Project-OSRM#6338)
      - CHANGED: Add `data_version` field to responses of all services. [Project-OSRM#5387](Project-OSRM#5387)
      - FIXED: Use Boost.Beast to parse HTTP request. [Project-OSRM#6294](Project-OSRM#6294)
      - FIXED: Fix inefficient osrm-routed connection handling [Project-OSRM#6113](https://gihub.com/Project-OSRM/osrm-backend/pull/6113)
      - FIXED: Fix HTTP compression precedence [Project-OSRM#6113](Project-OSRM#6113)
    - NodeJS:
      - FIXED: Support `skip_waypoints` in Node bindings [Project-OSRM#6060](Project-OSRM#6060)
    - Misc:
      - ADDED: conanbuildinfo.json for easy reading of dependencies [Project-OSRM#6388](Project-OSRM#6388)
      - CHANGED: Improve performance of JSON rendering. Fix undefined behaviour in JSON numbers formatting. [Project-OSRM#6380](Project-OSRM#6380)
      - ADDED: Add timestamps for logs. [Project-OSRM#6375](Project-OSRM#6375)
      - CHANGED: Improve performance of map matching via getPathDistance optimization. [Project-OSRM#6378](Project-OSRM#6378)
      - CHANGED: Optimize RestrictionParser performance. [Project-OSRM#6344](Project-OSRM#6344)
      - ADDED: Support floats for speed value in traffic updates CSV. [Project-OSRM#6327](Project-OSRM#6327)
      - CHANGED: Use Lua 5.4 in Docker image. [Project-OSRM#6346](Project-OSRM#6346)
      - CHANGED: Remove redundant nullptr check. [Project-OSRM#6326](Project-OSRM#6326)
      - CHANGED: missing files list is included in exception message. [Project-OSRM#5360](Project-OSRM#5360)
      - CHANGED: Do not use deprecated Callback::Call overload in Node bindings. [Project-OSRM#6318](Project-OSRM#6318)
      - FIXED: Fix distance calculation consistency. [Project-OSRM#6315](Project-OSRM#6315)
      - FIXED: Fix performance issue after migration to sol2 3.3.0. [Project-OSRM#6304](Project-OSRM#6304)
      - CHANGED: Pass osm_node_ids by reference in osrm::updater::Updater class. [Project-OSRM#6298](Project-OSRM#6298)
      - FIXED: Fix bug with reading Set values from Lua scripts. [Project-OSRM#6285](Project-OSRM#6285)
      - FIXED: Bug in bicycle profile that caused exceptions if there is a highway=bicycle in the data. [Project-OSRM#6296](Project-OSRM#6296)
      - FIXED: Internal refactoring of identifier types used in data facade [Project-OSRM#6044](Project-OSRM#6044)
      - CHANGED: Update docs to reflect recent build and dependency changes [Project-OSRM#6383](Project-OSRM#6383)
    - Build:
      - REMOVED: Get rid of Mason. [Project-OSRM#6387](Project-OSRM#6387)
      - CHANGED: Use clang-format from CI base image. [Project-OSRM#6391](Project-OSRM#6391)
      - ADDED: Build Node bindings on Windows. [Project-OSRM#6334](Project-OSRM#6334)
      - ADDED: Configure cross-compilation for Apple Silicon. [Project-OSRM#6360](Project-OSRM#6360)
      - CHANGED: Use apt-get to install Clang on CI. [Project-OSRM#6345](Project-OSRM#6345)
      - CHANGED: Fix TBB in case of Conan + NodeJS build. [Project-OSRM#6333](Project-OSRM#6333)
      - CHANGED: Migrate to modern TBB version. [Project-OSRM#6300](Project-OSRM#6300)
      - CHANGED: Enable performance-move-const-arg clang-tidy check. [Project-OSRM#6319](Project-OSRM#6319)
      - CHANGED: Use the latest node on CI. [Project-OSRM#6317](Project-OSRM#6317)
      - CHANGED: Migrate Windows CI to GitHub Actions. [Project-OSRM#6312](Project-OSRM#6312)
      - ADDED: Add smoke test for Docker image. [Project-OSRM#6313](Project-OSRM#6313)
      - CHANGED: Update libosmium to version 2.18.0. [Project-OSRM#6303](Project-OSRM#6303)
      - CHANGED: Remove EXACT from find_package if using Conan. [Project-OSRM#6299](Project-OSRM#6299)
      - CHANGED: Configure Undefined Behaviour Sanitizer. [Project-OSRM#6290](Project-OSRM#6290)
      - CHANGED: Use Conan instead of Mason to install code dependencies. [Project-OSRM#6284](Project-OSRM#6284)
      - CHANGED: Migrate to C++17. Update sol2 to 3.3.0. [Project-OSRM#6279](Project-OSRM#6279)
      - CHANGED: Update macOS CI image to macos-11. [Project-OSRM#6286](Project-OSRM#6286)
      - CHANGED: Enable even more clang-tidy checks. [Project-OSRM#6273](Project-OSRM#6273)
      - CHANGED: Configure CMake to not build flatbuffers tests and samples. [Project-OSRM#6274](Project-OSRM#6274)
      - CHANGED: Enable more clang-tidy checks. [Project-OSRM#6270](Project-OSRM#6270)
      - CHANGED: Configure clang-tidy job on CI. [Project-OSRM#6261](Project-OSRM#6261)
      - CHANGED: Use Github Actions for building container images [Project-OSRM#6138](Project-OSRM#6138)
      - CHANGED: Upgrade Boost dependency to 1.70 [Project-OSRM#6113](Project-OSRM#6113)
      - CHANGED: Upgrade Ubuntu CI builds to 20.04  [Project-OSRM#6119](Project-OSRM#6119)
      - CHANGED: Make building osrm-routed optional [Project-OSRM#6144](Project-OSRM#6144)
      - FIXED: Run all unit tests in CI [Project-OSRM#5248](Project-OSRM#5248)
      - FIXED: Fix installation of Mason CMake and 32 bit CI build [Project-OSRM#6170](Project-OSRM#6170)
      - FIXED: Fixed Node docs generation check in CI. [Project-OSRM#6058](Project-OSRM#6058)
      - CHANGED: Docker build, enabled arm64 build layer [Project-OSRM#6172](Project-OSRM#6172)
      - CHANGED: Docker build, enabled apt-get update/install caching in separate layer for build phase [Project-OSRM#6175](Project-OSRM#6175)
      - FIXED: Bump CI complete meta job to ubuntu-20.04 [Project-OSRM#6323](Project-OSRM#6323)
      - CHANGED: Node packages are now scoped by @Project-OSRM [Project-OSRM#6386](Project-OSRM#6386)
    - Routing:
      - CHANGED: Lazily generate optional route path data [Project-OSRM#6045](Project-OSRM#6045)
      - FIXED: Completed support for no_entry and no_exit turn restrictions. [Project-OSRM#5988](Project-OSRM#5988)
      - ADDED: Add support for non-round-trips with a single fixed endpoint. [Project-OSRM#6050](Project-OSRM#6050)
      - FIXED: Improvements to maneuver override processing [Project-OSRM#6125](Project-OSRM#6125)
      - ADDED: Support snapping to multiple ways at an input location. [Project-OSRM#5953](Project-OSRM#5953)
      - FIXED: Fix snapping target locations to ways used in turn restrictions. [Project-OSRM#6339](Project-OSRM#6339)
      - ADDED: Support OSM traffic signal directions. [Project-OSRM#6153](Project-OSRM#6153)
      - FIXED: Ensure u-turn exists in intersection view. [Project-OSRM#6376](Project-OSRM#6376)
      - FIXED: Gracefully handle no-turn intersections in guidance processing. [Project-OSRM#6382](Project-OSRM#6382)
    - Profile:
      - CHANGED: Bicycle surface speeds [Project-OSRM#6212](Project-OSRM#6212)
    - Tools:
      - CHANGED: Do not generate intermediate .osrm file in osrm-extract. [Project-OSRM#6354](Project-OSRM#6354)
datendelphin added a commit to fossgis-routing-server/osrm-backend that referenced this pull request Dec 18, 2022
@datendelphin
Merge tag 'v5.27.0'
5552c2b 
v5.27.0

- Changes from 5.26.0
    - API:
      - ADDED: Add Flatbuffers support to NodeJS bindings. [Project-OSRM#6338](Project-OSRM#6338)
      - CHANGED: Add `data_version` field to responses of all services. [Project-OSRM#5387](Project-OSRM#5387)
      - FIXED: Use Boost.Beast to parse HTTP request. [Project-OSRM#6294](Project-OSRM#6294)
      - FIXED: Fix inefficient osrm-routed connection handling [Project-OSRM#6113](https://gihub.com/Project-OSRM/osrm-backend/pull/6113)
      - FIXED: Fix HTTP compression precedence [Project-OSRM#6113](Project-OSRM#6113)
    - NodeJS:
      - FIXED: Support `skip_waypoints` in Node bindings [Project-OSRM#6060](Project-OSRM#6060)
    - Misc:
      - ADDED: conanbuildinfo.json for easy reading of dependencies [Project-OSRM#6388](Project-OSRM#6388)
      - CHANGED: Improve performance of JSON rendering. Fix undefined behaviour in JSON numbers formatting. [Project-OSRM#6380](Project-OSRM#6380)
      - ADDED: Add timestamps for logs. [Project-OSRM#6375](Project-OSRM#6375)
      - CHANGED: Improve performance of map matching via getPathDistance optimization. [Project-OSRM#6378](Project-OSRM#6378)
      - CHANGED: Optimize RestrictionParser performance. [Project-OSRM#6344](Project-OSRM#6344)
      - ADDED: Support floats for speed value in traffic updates CSV. [Project-OSRM#6327](Project-OSRM#6327)
      - CHANGED: Use Lua 5.4 in Docker image. [Project-OSRM#6346](Project-OSRM#6346)
      - CHANGED: Remove redundant nullptr check. [Project-OSRM#6326](Project-OSRM#6326)
      - CHANGED: missing files list is included in exception message. [Project-OSRM#5360](Project-OSRM#5360)
      - CHANGED: Do not use deprecated Callback::Call overload in Node bindings. [Project-OSRM#6318](Project-OSRM#6318)
      - FIXED: Fix distance calculation consistency. [Project-OSRM#6315](Project-OSRM#6315)
      - FIXED: Fix performance issue after migration to sol2 3.3.0. [Project-OSRM#6304](Project-OSRM#6304)
      - CHANGED: Pass osm_node_ids by reference in osrm::updater::Updater class. [Project-OSRM#6298](Project-OSRM#6298)
      - FIXED: Fix bug with reading Set values from Lua scripts. [Project-OSRM#6285](Project-OSRM#6285)
      - FIXED: Bug in bicycle profile that caused exceptions if there is a highway=bicycle in the data. [Project-OSRM#6296](Project-OSRM#6296)
      - FIXED: Internal refactoring of identifier types used in data facade [Project-OSRM#6044](Project-OSRM#6044)
      - CHANGED: Update docs to reflect recent build and dependency changes [Project-OSRM#6383](Project-OSRM#6383)
    - Build:
      - REMOVED: Get rid of Mason. [Project-OSRM#6387](Project-OSRM#6387)
      - CHANGED: Use clang-format from CI base image. [Project-OSRM#6391](Project-OSRM#6391)
      - ADDED: Build Node bindings on Windows. [Project-OSRM#6334](Project-OSRM#6334)
      - ADDED: Configure cross-compilation for Apple Silicon. [Project-OSRM#6360](Project-OSRM#6360)
      - CHANGED: Use apt-get to install Clang on CI. [Project-OSRM#6345](Project-OSRM#6345)
      - CHANGED: Fix TBB in case of Conan + NodeJS build. [Project-OSRM#6333](Project-OSRM#6333)
      - CHANGED: Migrate to modern TBB version. [Project-OSRM#6300](Project-OSRM#6300)
      - CHANGED: Enable performance-move-const-arg clang-tidy check. [Project-OSRM#6319](Project-OSRM#6319)
      - CHANGED: Use the latest node on CI. [Project-OSRM#6317](Project-OSRM#6317)
      - CHANGED: Migrate Windows CI to GitHub Actions. [Project-OSRM#6312](Project-OSRM#6312)
      - ADDED: Add smoke test for Docker image. [Project-OSRM#6313](Project-OSRM#6313)
      - CHANGED: Update libosmium to version 2.18.0. [Project-OSRM#6303](Project-OSRM#6303)
      - CHANGED: Remove EXACT from find_package if using Conan. [Project-OSRM#6299](Project-OSRM#6299)
      - CHANGED: Configure Undefined Behaviour Sanitizer. [Project-OSRM#6290](Project-OSRM#6290)
      - CHANGED: Use Conan instead of Mason to install code dependencies. [Project-OSRM#6284](Project-OSRM#6284)
      - CHANGED: Migrate to C++17. Update sol2 to 3.3.0. [Project-OSRM#6279](Project-OSRM#6279)
      - CHANGED: Update macOS CI image to macos-11. [Project-OSRM#6286](Project-OSRM#6286)
      - CHANGED: Enable even more clang-tidy checks. [Project-OSRM#6273](Project-OSRM#6273)
      - CHANGED: Configure CMake to not build flatbuffers tests and samples. [Project-OSRM#6274](Project-OSRM#6274)
      - CHANGED: Enable more clang-tidy checks. [Project-OSRM#6270](Project-OSRM#6270)
      - CHANGED: Configure clang-tidy job on CI. [Project-OSRM#6261](Project-OSRM#6261)
      - CHANGED: Use Github Actions for building container images [Project-OSRM#6138](Project-OSRM#6138)
      - CHANGED: Upgrade Boost dependency to 1.70 [Project-OSRM#6113](Project-OSRM#6113)
      - CHANGED: Upgrade Ubuntu CI builds to 20.04  [Project-OSRM#6119](Project-OSRM#6119)
      - CHANGED: Make building osrm-routed optional [Project-OSRM#6144](Project-OSRM#6144)
      - FIXED: Run all unit tests in CI [Project-OSRM#5248](Project-OSRM#5248)
      - FIXED: Fix installation of Mason CMake and 32 bit CI build [Project-OSRM#6170](Project-OSRM#6170)
      - FIXED: Fixed Node docs generation check in CI. [Project-OSRM#6058](Project-OSRM#6058)
      - CHANGED: Docker build, enabled arm64 build layer [Project-OSRM#6172](Project-OSRM#6172)
      - CHANGED: Docker build, enabled apt-get update/install caching in separate layer for build phase [Project-OSRM#6175](Project-OSRM#6175)
      - FIXED: Bump CI complete meta job to ubuntu-20.04 [Project-OSRM#6323](Project-OSRM#6323)
      - CHANGED: Node packages are now scoped by @Project-OSRM [Project-OSRM#6386](Project-OSRM#6386)
    - Routing:
      - CHANGED: Lazily generate optional route path data [Project-OSRM#6045](Project-OSRM#6045)
      - FIXED: Completed support for no_entry and no_exit turn restrictions. [Project-OSRM#5988](Project-OSRM#5988)
      - ADDED: Add support for non-round-trips with a single fixed endpoint. [Project-OSRM#6050](Project-OSRM#6050)
      - FIXED: Improvements to maneuver override processing [Project-OSRM#6125](Project-OSRM#6125)
      - ADDED: Support snapping to multiple ways at an input location. [Project-OSRM#5953](Project-OSRM#5953)
      - FIXED: Fix snapping target locations to ways used in turn restrictions. [Project-OSRM#6339](Project-OSRM#6339)
      - ADDED: Support OSM traffic signal directions. [Project-OSRM#6153](Project-OSRM#6153)
      - FIXED: Ensure u-turn exists in intersection view. [Project-OSRM#6376](Project-OSRM#6376)
      - FIXED: Gracefully handle no-turn intersections in guidance processing. [Project-OSRM#6382](Project-OSRM#6382)
    - Profile:
      - CHANGED: Bicycle surface speeds [Project-OSRM#6212](Project-OSRM#6212)
    - Tools:
      - CHANGED: Do not generate intermediate .osrm file in osrm-extract. [Project-OSRM#6354](Project-OSRM#6354)
mattwigway pushed a commit to mattwigway/osrm-backend that referenced this pull request Jul 20, 2023
@SiarheiFedartsou @mattwigway
Use Lua 5.4 in Docker image (Project-OSRM#6346)
da352cc
@DennisOSRM DennisOSRM deleted the sf-lua-5.4 branch May 3, 2024 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mjjbell mjjbell mjjbell approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@SiarheiFedartsou @mjjbell