Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: lazily creating a tempdir in NativeLoader #1326

Merged
merged 1 commit into from
Jan 3, 2022
Merged

fix: lazily creating a tempdir in NativeLoader #1326

merged 1 commit into from
Jan 3, 2022

Conversation

ocworld
Copy link
Contributor

@ocworld ocworld commented Jan 3, 2022

This pr is recreated. origin pr is #945


When predicting with a lightgbm model, there are so many lightgbm's so files on runtime.
It made failure on allocating memory on tmpfs in my application.

So, I tried to use java.library.path that has already "so" files.
When trying this approach, I found that it is possible to create so many /tmp/mml-nativesXXXX directories. Though, copied "so" files were not used.

Therefore, this pr is created to create lazily tempdir in NativeLoader to using getOrCreateTempDir() instead of making it in class's initializer.

@ocworld ocworld requested a review from mhamilton723 as a code owner January 3, 2022 01:49
@ocworld
Copy link
Contributor Author

ocworld commented Jan 3, 2022

@mhamilton723 Semantic Pull Request is failed T.T How to fix it?

@mhamilton723
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@codecov-commenter
Copy link

codecov-commenter commented Jan 3, 2022

Codecov Report

Merging #1326 (639e419) into master (906b408) will decrease coverage by 3.50%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1326      +/-   ##
==========================================
- Coverage   84.69%   81.18%   -3.51%     
==========================================
  Files         284      284              
  Lines       13916    13916              
  Branches      673      673              
==========================================
- Hits        11786    11298     -488     
- Misses       2130     2618     +488     
Impacted Files Coverage Δ
...m/src/main/scala/com/microsoft/lightgbm/SWIG.scala 0.00% <0.00%> (-100.00%) ⬇️
...soft/azure/synapse/ml/cognitive/AudioStreams.scala 0.00% <0.00%> (-87.88%) ⬇️
...azure/synapse/ml/lightgbm/LightGBMClassifier.scala 11.11% <0.00%> (-80.00%) ⬇️
...t/azure/synapse/ml/cognitive/SpeechToTextSDK.scala 18.43% <0.00%> (-72.16%) ⬇️
...crosoft/azure/synapse/ml/cognitive/SpeechAPI.scala 0.00% <0.00%> (-70.00%) ⬇️
...t/azure/synapse/ml/lightgbm/LightGBMDelegate.scala 0.00% <0.00%> (-50.00%) ⬇️
.../synapse/ml/lightgbm/booster/LightGBMBooster.scala 54.41% <0.00%> (-35.30%) ⬇️
...crosoft/azure/synapse/ml/lightgbm/TrainUtils.scala 56.05% <0.00%> (-29.94%) ⬇️
.../synapse/ml/lightgbm/dataset/LightGBMDataset.scala 42.68% <0.00%> (-20.74%) ⬇️
...osoft/azure/synapse/ml/lightgbm/LightGBMBase.scala 79.77% <0.00%> (-15.17%) ⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b64094...639e419. Read the comment docs.

@mhamilton723 mhamilton723 merged commit fc76deb into microsoft:master Jan 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants