Implement the batching design for position prepare messages

[PaulGregoryBaker]

Goal:

As a development team working on mojaloop transfer performance enhancement
I want to Implement the transfer batching design for the position handler
so that I get the performance benefit of batching

This story is to implement the first part of this which is the prepare handler. The implmenetation design is to run old and new handlers together and use configration to and seperate kafka topics to seperate the message flows.

Acceptance Criteria:

• Verify that the batch is designed for the position handler as defined in story Fix the transfer batching design for the position handler #3488
• Verify that the position handler characterisation is performed with the batch functionality implemented

Note
As this story will need to be resized/or checked for size after the design story is completed #3488

Complexity: <High|Medium|Low> > A short comment to remind the reason for the rating

Uncertainty: <High|Medium|Low> > A short comment to remind the reason for the rating

---

Tasks:

• Add skeleton for new handler, bin processor and prepare domain function [ @vijayg10 ]
• Implement new handler [@vijayg10 ]
  • Add unit tests [@vijayg10 ]
• Implement domain function: processPositionPrepareBin [@kleyow ]
  • Add unit tests [@kleyow ]
• Implement domain function: processBins [@vijayg10 ]
  • Add unit tests [@vijayg10 ]
• Add integration tests for happy path [@vijayg10 ] [@kleyow ]
• Add integration tests for NDC limit and payer liquidity [@kleyow ]
• Combine everything and execute functional tests [@vijayg10 ]
• Add integration tests / functional tests for negative scenarios for end to end functionality [@sri-miriyala ]
• Characterise the performance in isolation [@vijayg10 ]
• Document the observations and difference in performance compared to non batch mode [ @vijayg10 ]
• Sprint review - followup tasks
  • Try batchSize of 1 and document the observations
  • Check if the notifications are published before commit or after and align the design diagram according to the implementation and make sure that there will be no issue if a service dies at any point of execution - Checked the implementation, the order is like Kafka Commit->DB Commit->Send Notifications. Changed the diagrams accordingly.
  • Test a new scenarios for batching performance with a managed DB instead of local DB As we could push the CPU utilisation to 99% and got a throughput of 282 ops/sec, decided to not do this test for now
• Create followup stories

Done

• Acceptance Criteria pass
• Designs are up-to date
• Unit Tests pass
• Integration Tests pass
• Code Style & Coverage meets standards
• Changes made to config (default.json) are broadcast to team and follow-up tasks added to update helm charts and other deployment config.
• TBD

Pull Requests:

• feat(mojaloop/#3636): batching implementation for position prepare messages central-ledger#968
  • feat: add position prepare bin logic central-ledger#970
  • chore: add tests central-ledger#972
  • feat(mojaloop/#3489): add negative integration tests central-ledger#975
• feat(mojaloop/3489): characterization batch implementation of position prepare messages ml-perf-characterization#13

Follow-up:

• N/A

Dependencies:

• N/A

Accountability:

• Owner: TBC
• QA/Review: TBC