Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA: Mojaloop Helm v14.1.0 Release - Bulk Tests fail on first run #3027

Closed
mdebarros opened this issue Nov 25, 2022 · 4 comments
Closed

QA: Mojaloop Helm v14.1.0 Release - Bulk Tests fail on first run #3027

mdebarros opened this issue Nov 25, 2022 · 4 comments
Assignees
Labels
bug Something isn't working or it has wrong behavior on a Mojaloop Core service core-dev-squad oss-core This is an issue - story or epic related to a feature on a Mojaloop core service or related to it

Comments

@mdebarros
Copy link
Member

mdebarros commented Nov 25, 2022

Summary:

On a fresh installation of Mojaloop Helm v14.1.0 Release, the Bulk Tests may fail intermitantly on the first run due to timing/warm-up issues.

Work-around

Configure the TTK Tests timeout (refer to https://github.com/mojaloop/helm/blob/d6005c22910fc10a24e73f44a6aa7f3e5513902b/ml-testing-toolkit/chart-backend/values.yaml#L133)

ml-testing-toolkit:
  ml-testing-toolkit-backend:
    ....
    config:
      user_config.json:
          DEFAULT_REQUEST_TIMEOUT: 5000 <-- default value. This can be manually changed in the mojaloop/values.yaml to a higher number to cater for the initial slow system response, and will be specific to your deployment topology.
          SCRIPT_TIMEOUT: 5000 <-- default value. This can be manually changed in the mojaloop/values.yaml to a higher number to cater for the initial slow system response, and will be specific to your deployment topology.

Severity:

Low

Priority:

Low

Expected Behavior

On a fresh installation of Mojaloop Helm v14.1.0 Release, the Bulk Tests should pass constistently on first runs.

Steps to Reproduce

  1. Deploy Mojaloop Helm v14.1.0 release -> helm install moja --version 14.1.0
  2. Wait for all services to report healthy (approximately 6-8m)
  3. Run helm test moja

Specifications

  • Component (if known): Helm Tests
  • Version: v14.1.0
  • Platform: N/A
  • Subsystem: N/A
  • Type of testing: Manual
  • Bug found/raised by: @mdebarros

Notes:

  • Severity when opened: Low
  • Priority when opened: Low
@mdebarros mdebarros added bug Something isn't working or it has wrong behavior on a Mojaloop Core service oss-core This is an issue - story or epic related to a feature on a Mojaloop core service or related to it core-dev-squad labels Nov 25, 2022
@mdebarros mdebarros changed the title Mojaloop Helm v14.1.0 Release - Bulk Tests fail on first run QA: Mojaloop Helm v14.1.0 Release - Bulk Tests fail on first run Nov 25, 2022
mdebarros added a commit to mojaloop/helm that referenced this issue Nov 25, 2022
- added Known issue 10.4. [#3027 - QA: Mojaloop Helm v14.1.0 Release - Bulk Tests fail on first run](mojaloop/project#3027)
mdebarros added a commit to mojaloop/helm that referenced this issue Nov 25, 2022
docs: update release-v14.1.0 notes [skip ci]
- added Known issue 10.4. [#3027 - QA: Mojaloop Helm v14.1.0 Release - Bulk Tests fail on first run](mojaloop/project#3027)
@elnyry-sam-k
Copy link
Member

Hey team! Please add your planning poker estimate with Zenhub @mdebarros @vijayg10

@kleyow
Copy link

kleyow commented Dec 2, 2022

Warm up tests added in order to hopefully mitigate this issue mojaloop/testing-toolkit-test-cases#102

@kleyow
Copy link

kleyow commented Dec 2, 2022

Observation made that there are no bulk quotes funtional tests in the testing-toolkit-test-cases collection. Story added for that #3039.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working or it has wrong behavior on a Mojaloop Core service core-dev-squad oss-core This is an issue - story or epic related to a feature on a Mojaloop core service or related to it
Projects
None yet
Development

No branches or pull requests

3 participants