Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix stock performance issues #5143

Merged

Conversation

jniles
Copy link
Collaborator

@jniles jniles commented Nov 24, 2020

This PR addresses two performance issues:

  1. We cache the results of the stock out/expired stock components on the depot page to prevent multiple calls to that route from hitting the server simultaneously.
  2. We've removed calls to stock_consumption in the stock controller and merged the calls to getCMM into a single function. Then we optimized the function to only call getCMM() for unique depot/inventory pairs.

While doing so, I've refactored the CMM calculations in stock/core.js to make the more streamlined and rename generic sounding names to more meaningful names . There are two basic paths through core.js:

  1. getBulkInventoryCMM() -> computeInventoryIndicators()
  2. getBulkInventoryCMM() -> computeInventoryIndicators() -> computeLotIndicators()

The first path computes S_MAX, S_MIN, and other inventory indicators that relate to the change of stock out. This is good enough for the "Items in Stock" registry and other reports that need to see the data on an inventory level.

The second path does everything in the first, but also computes the S_RISK and S_RISK_QUANTITY and other lot indicators at are related to the risk of expiration of an individual lot.

Closes #4711
Partially addresses #5127
Partially addresses #5073

@jniles jniles force-pushed the fix-stock-performance-issues branch 5 times, most recently from bc839b2 to 3ce70f7 Compare November 25, 2020 11:17
@jniles jniles force-pushed the fix-stock-performance-issues branch 2 times, most recently from e315c58 to fbd5b08 Compare November 25, 2020 20:52
@jniles jniles requested a review from mbayopanda November 25, 2020 20:55
@jniles
Copy link
Collaborator Author

jniles commented Nov 25, 2020

@mbayopanda can I get a review?

I discovered that on master, the months of stock for the CMM was 5, according to the getCMM() function. I think this was a bug - it should have been 6, since that is what we put in the database. That is why I had to update a few tests.

@jniles jniles requested a review from jmcameron November 25, 2020 20:57
@jniles jniles force-pushed the fix-stock-performance-issues branch 2 times, most recently from f522586 to a7a1e35 Compare November 26, 2020 15:22
@jniles
Copy link
Collaborator Author

jniles commented Nov 30, 2020

@mbayopanda can this be merged?

@jmcameron
Copy link
Collaborator

I would like to review this PR, but I'm unclear how to evaluate its performance improvement.
What DB and Bhima page best shows the speed up?

@jniles
Copy link
Collaborator Author

jniles commented Nov 30, 2020

@jmcameron, stock exit is the best example of this. Here are the things to look for:

  1. When the page is refreshed (F5), there was two HTTP requests sent for each route. Now there are one.
  2. You'll observe a minor performance improvement in the queries, but this could just be noise as well.

@jniles
Copy link
Collaborator Author

jniles commented Nov 30, 2020

Another thing you can do, I suppose is to print the DB logs (DEBUG=db*) and see that we call getCMM() much less frequently.

@jniles jniles force-pushed the fix-stock-performance-issues branch from a7a1e35 to 48e6c25 Compare November 30, 2020 17:23
Caches the results for the stock components to ensure that we do not
overwhelm our servers with HTTP requests.
Removes extraneous code paths from the CMM calculation and combines the
duplicated code into a single function to calculate the CMM.  Also
remove the redundant calls to getCMM().
Removed unused indicators from the stock core reporting.
@jniles jniles force-pushed the fix-stock-performance-issues branch from 48e6c25 to abd0a3c Compare December 1, 2020 19:23
Copy link
Collaborator

@jmcameron jmcameron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. My review of the code was more like a skim of changes, but the changes seem okay. I tested the latest BHIMA and IMCK data with and without this PR. The number of calls to getCMM() is definitely reduced with this PR. But there are still possibly some duplicate calls. I noticed a definite speed up. With the IMCK data, the "loading data" part of the Stock exit page took 2-3 seconds with master and was less than a second with this PR. So this is a worthwhile refactor.

@@ -4,27 +4,27 @@

<table
class="table table-condensed table-bordered"
ng-show="$ctrl.soldOutInventories.length > 0">
ng-show="$ctrl.stockOutInventories.length > 0">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better variable name!

Fixes the fact that we have some months with 30 days and others with 31
days.
@jniles
Copy link
Collaborator Author

jniles commented Dec 2, 2020

@jmcameron thanks for your review! I'm going to include this in the quick fix release we do at IMCK today, since our production users will hammer it and let us know if there are any breaking issues. Much better to find that our while we can work with them here than have them call us from afar...

@jniles
Copy link
Collaborator Author

jniles commented Dec 2, 2020

bors r+

@bors
Copy link
Contributor

bors bot commented Dec 2, 2020

Build succeeded:

@bors bors bot merged commit d104d9e into Third-Culture-Software:master Dec 2, 2020
@jniles jniles deleted the fix-stock-performance-issues branch December 2, 2020 19:07
@jniles jniles mentioned this pull request Dec 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize stock/inventories/depots route
2 participants