3.12 testplan

[ramezsaeed]

Approach

Test plan for v 3.12.x should cover:

• development project: https://github.com/orgs/threefoldtech/projects/203

• Testlodge Run:

  • QAnet:
    • iteration 1: https://app.testlodge.com/a/26076/projects/40893/runs/721107
    • iteration 2: https://app.testlodge.com/a/26076/projects/40893/runs/725034
  • Testnet:
    • iteration 1: https://app.testlodge.com/a/26076/projects/40893/runs/722799
    • iteration 2: https://app.testlodge.com/a/26076/projects/40893/runs/725177
  • Mainnet:
    • https://app.testlodge.com/a/26076/projects/40893/runs/727876

• Known Issues: 3.12.x Release Note draft #1471

Schedule

• 3.12.x expected to be on QAnet OCT 1st
• All tests and verification should be within one week maximum.

Scope

Main Features:

• [package]: create an iperf runtime package for zos zos#2034
• explorer nodes UI needs to be better and more informative #1457
• tools for TF Cloud Admin & Sign Up #1461
• we need a basic perf test in ZOS as primitive #1454
• Add actions to vms to add/remove gateway workload to/from a vm tfgrid-sdk-ts#1055

Quick verification:

• [package]: create an iperf runtime package for zos zos#2034
• install and support rfs v2 with backward compatibility support zos#2071
• need RMB to be redundant #1458
• [perf] research cpu performance tools zos#2036
• implement a network test against public nodes zos#2050
• Missing steps in docs of running the UI package tfgrid-sdk-ts#1127
• initial 3bot for threefoldcloud (spec phase) #1449
• explorer nodes UI needs to be better and more informative #1457
• Add Warning in Contract Lists tfgrid-sdk-ts#1041
• Should have same result when there's no dedicate node match the filter  tfgrid-sdk-ts#1034
• Playground Docs: Config Script tfgrid-sdk-ts#1066

Validation and Add new testcases to testlodge:

• explorer nodes UI needs to be better and more informative #1457
• Add actions to vms to add/remove gateway workload to/from a vm tfgrid-sdk-ts#1055
• productize webgw in relation to deployed VM #1452
• tools for TF Cloud Admin & Sign Up #1461
• perf tests framework inside noded zos#2037
• Add actions to vms to add/remove gateway workload to/from a vm tfgrid-sdk-ts#1055
• limit number of cpus reserved in VM to max number of CPUs on host zos#2056
• WordPress uploads folder owned by root tf-images#163
• 3script editor component tfgrid-sdk-ts#1057
• Create a workflow for building bridge image with releases tfchain#864

PerfTest Reference Docs:

• https://github.com/threefoldtech/zos/tree/main/docs/tasks

Testlodge coverage:

• https://app.testlodge.com/a/26076/projects/40893/suites/234919

verify basic perf test and node performance in ZOS

• we need a basic perf test in ZOS as primitive #1454
• node performance test  zos#2033

Need to be tested

• iperf tests run on offline nodes (should run on online nodes) zos#2111
• what are the units for iperf test results zos#2113
• CPU benckmark result doesn't tell if is it good or bad zos#2112
• Can't get the result of the public ip validation zos#2114

[ramezsaeed]

QAnet Test run:
https://app.testlodge.com/a/26076/projects/40893/runs/721107

[ramezsaeed]

TestNet run:

https://app.testlodge.com/a/26076/projects/40893/runs/722799?tab=0

[xmonader]

#1454 this needs to be tracked

[xmonader]

While this is part of 3.12 project, it's purely bizdev #1459 , please make sure that's not tracked

[ramezsaeed]

Testnet run is blocked on GPU
https://github.com/threefoldtech/tf_operations/issues/2014

[ramezsaeed]

Done with fixing and verifying all add domains feature issues.

threefoldtech/tfgrid-sdk-ts#1055

[ramezsaeed]

Work In Progress (WIP):
Testnet run iterration two is in progress:
https://app.testlodge.com/a/26076/projects/40893/runs/725177

[ramezsaeed]

Work In Progress (WIP):
Testnet run iterration two is in progress:

[ramezsaeed]

Testnet run Done here
https://app.testlodge.com/a/26076/projects/40893/runs/725177

[ramezsaeed]

waiting for 3.12 to be deployed on Mainnet

[ramezsaeed]

Test run for all the grid solutions can be followed here:

• Test Run for All the available solutions tfgrid-sdk-ts#1763

[xmonader]

@ramezsaeed please remove what's irrelevant to 3.12 from the story

[ramezsaeed]

https://app.testlodge.com/a/26076/projects/40893/runs/727876