3.14 Testplan

[ramezsaeed]

Introduction

Test plan for v 3.14.x release on TFgrid

• development project: https://github.com/orgs/threefoldtech/projects/206
• Known Issues: Known issues for 3.14.x release #1532

Schedule

• 3.14.x is devided into two Releses
  • 2.4.x expected to be on QAnet on: April 8/2024 (released on 22/5/2024)
  • 2.5.x expected to be on QAnet on: June 9/2024 (released on 11/6/2024)
• All tests and verification should be done on QAnet within one week maximum.
• After deploying on Testnet will keep it for one week before go to Mainnet in order to get tested by the community.

Scope

Test the main features in 3.14.x release:

• Multiple SSH Keys
• More application images
• ThreeFold connect app new update 3.9.1
• farmerbot updates
• pulumi support mycelium

Test Reports

• Testlodge Run:
  • QAnet:
    • QAnet 3.14 TestRun 2.4_rc1
    • QAnet 3.14 TestRun 2.4_rc2
    • Qanet testrun V2.5
    • Atef Dashboard QAnet 2.5
  • Testnet:
    • Testnet 3.14 TestRun 2.4.0
    • Testnet testrun V2.5
  • Mainnet:
    • Mainnet TestRun 2.4.2
    • Atef Dashboard 2.4.2
    • Mainnet TestRun 2.5.2
    • Atef Mainnet Dashboard V2.5.2

Roles and Responsibilities

1. verification are done on Devnet (team members verify/ Ramez can close)
2. release on QAnet done by dev team approval.
3. QA team starts new test run iteration on Qanet.
4. BAM(Bug Assessment Meeting) is needed to approve the release and address the reported issues.
5. once approved the release go to Testnet.
6. QA team starts new test run iteration on Testnet.
7. BAM is needed to approve the release and address the reported issues.
8. once approved release kept on Testnet for two weeks before go to Mainnet.
9. once release on Mainnet QA team starts new test run iteration on Mainnet.

This test plan is maintained by QA team:

@ramezsaeed
@khaledyoussef24
@A-Harby

Main Features Testing and Validation:

• Enhancement Request: Implementing Multiple SSH Keys with Enhanced Management Features tfgrid-sdk-ts#1882
  • Related issues:
    • New Ssh-Keys page enhancements tfgrid-sdk-ts#2543
    • Add a SSH key validation tfgrid-sdk-ts#2372
  • Testsuite: https://app.testlodge.com/a/26076/projects/40893/suites/234374?expand_section=430164#suite_section_430164
• More application images tf-images#195
  • Related issues:
    • integrate ubuntu 23.04 in dashbaord tfgrid-sdk-ts#2504
  • Testsuiite: TODO
• Threefold connect new app version 3.9.1
  • Related issues:
    • Test run progress for ThreeFold Connect threefold_connect#602
  • Testsuiite: https://app.testlodge.com/a/26076/projects/40893/suites/206114
• farmerbot
  • Related issues:
    • improved capacity planning to wakeup nodes Node wake priority tfgrid-sdk-go#978
    • allowing farmers to specify the priority to wakeup nodes 🐞 [Bug]: Farmerbot wake up threshold works on simple sum of all resource types tfgrid-sdk-go#804
    • https://github.com/threefoldtech/tfgrid-sdk-go/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Afarmerbot
  • Testsuiite: https://app.testlodge.com/a/26076/projects/40893/suites/221831
• pulumi
  • support mycelium Add mycelium changes pulumi-threefold#255
  • add more examples
• zos
  • supporting compressed kernels automatic vmlinuz decompression for micro VMs zos#2236 (also now supported by newer version from cloudhypervisor)
  • stricter health checks for filtering nodes to deploy on introduce diagnostics call zos#2153
  • display the right size in the microvm (not the full node disk size) Display right disk size in micro vms zos#2007
  • tools to debug images/flists if they're bootable in zos development tools to help building and debugging custom images zos#2233
  • zos able to download/mount flists over mycelium IPs Add mycelium support to zos zos#2338
  • zos to support zdb over mycelium IPs Add mycelium support to zos zos#2338
  • support HDD only nodes supporting using HDD for virtual machines workloads zos#2258

Non Release Blocking Stories:

• Automating the new playground and dashboard tests tfgrid-sdk-ts#1285

Out of QA Team Scope

• threefold cloud website #1459