Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automated install tests #226

Open
wants to merge 44 commits into
base: master
Choose a base branch
from
Open

Automated install tests #226

wants to merge 44 commits into from

Conversation

ydirson
Copy link
Contributor

@ydirson ydirson commented May 14, 2024 •
edited
Loading

Now based on top of preliminary PRs to separate generally-useful stuff:

Requires at least python 3.8 and pytest 8.2.

gduperrey
gduperrey reviewed May 14, 2024
View reviewed changes
scripts/install_xcpng.py Outdated Show resolved Hide resolved
gduperrey
gduperrey reviewed May 14, 2024
View reviewed changes
data.py-dist Outdated Show resolved Hide resolved
@ydirson
Copy link
Contributor Author

ydirson commented May 15, 2024 •
edited
Loading

Current status: install + firstboot work in a single test
Next:

  • xva export/import, split firstboot in separate test
  • include answerfile in install.img
  • answerfile generation

@ydirson ydirson force-pushed the install branch 3 times, most recently from 13282cc to 64d0e9c Compare May 23, 2024 08:16
@ydirson
Copy link
Contributor Author

ydirson commented May 23, 2024 •
edited
Loading

Current status: 2 separate tests for install and firstboot work without any manual intervention
Next steps:

  • use VM cache to gain time
  • write reinstall test
  • parametrize ISO remaster
  • write upgrade test

@ydirson ydirson force-pushed the install branch 4 times, most recently from 143c991 to 56abc4e Compare May 28, 2024 13:38
@ydirson
Copy link
Contributor Author

ydirson commented May 28, 2024 •
edited by gthvn1
Loading

Current status:

  • 3 separate tests for install, firstboot, "reinstall upgrade"
  • test dependencies declared (requires pytest-dependency, best used with pytest-order)
  • includes pytest_collection_modifyitems hook to automatically run the dependencies first

Next steps:

  • write a hook to not rerun dependencies whose output is in the vm cache, starting with Add support for a hook to override skip decision. RKrahl/pytest-dependency#77
  • start adding (explicit) parameters on tests
  • likely postoned
    • fix pytest_collection_modifyitems hook to properly register additional tests (some fixtures are missing)
    • improve hook to deal with test parameters
    • improve hook further by not ignoring dependents of tests explicitly requested (so in a->b->c->d chain with cached VMs pytest b d will also run c

@ydirson ydirson force-pushed the install branch 4 times, most recently from 1fda4a1 to 2dd8ec0 Compare June 14, 2024 08:04
@ydirson
Copy link
Contributor Author

ydirson commented Jun 14, 2024 •
edited
Loading

Current status:

  • cached VM descriptions now much less verbose, fitting in XO display
  • pytest_collection_modifyitems hook for auto-registring dependencies was moved out of scope, will be a later PR
  • start including some tests of the fixtures themselves
  • improvements to test parametrization well under way:
    • started to make fixtures more versatile, with ability to use callables taking test parameters, leading to less duplication
    • ... but bad interaction with pytest-order, to be investigated

Next steps:

  • deal with the regression on dependency ordering - may not be trivial or even doable
  • finish adding uefi/bios "firmware" parameter
  • add iso/net "package source" parameter
  • add ability to upgrade from XS
  • add ability to upgrade from 8.0-8.1 to 8.3
  • add ability to upgrade from 7.5-7.6 to 8.3
  • add "restore" test following "upgrade"

@ydirson ydirson force-pushed the install branch 3 times, most recently from b75d3ed to ebe6700 Compare June 19, 2024 16:03
@ydirson ydirson force-pushed the install branch 4 times, most recently from 358055b to 18e0ef8 Compare June 25, 2024 15:14
ydirson and others added 9 commits October 10, 2024 14:29
@ydirson
install: adjust host IP, name, UUIDs in firstboot data before booting
1349fb5 
Uses a helper VM to modify firstboot data in installed disk

Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
Add local_sr parameter to test_install
7a6b454 
Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
import_vm: add clone:// and clone+start:// URIs
559277e 
clone+start:// will be used to implement--hosts=cache://...

clone:// itself is not yet used directly, but as the "base" protocol upon
which clone+start build, it seems logical (and basically free) to implement.

Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
hosts: make setup_host a nested func
5211c04 
Will allow it to access host_list through closure.

Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
Add support for --nest=... --hosts=cache://...
6735aac 
Special care taken for making sure a created VM is destroyed if
setup_host fails further down the line.

Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@gthvn1 @ydirson
Add support for netinstall
3e3c746 
Add a new parameter to allow booting using netinstall. If an ISO only
supports netinstall, it is specified it in the data.py by setting the
`net-only`option to True. This option is set to False by default.

Signed-off-by: Guillaume <guillaume.thouvenin@vates.tech>
Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
New test: pool_join
aeef1d4 
Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
firstboot: reordering cleanup
ab1f2c2 
Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@ydirson
firstboot: move _upg and _rst tests to group them with upgrade and re…
2354c4e 
…store

Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
@stormi stormi removed the request for review from tescande October 15, 2024 13:46
@stormi
Copy link
Member

stormi commented Oct 15, 2024

@benjamreis This PR is now ready, let's review it in the next days.

benjamreis
benjamreis approved these changes Oct 16, 2024
View reviewed changes
Copy link
Contributor

@benjamreis benjamreis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ydirson
Copy link
Contributor Author

ydirson commented Oct 17, 2024

Last push fixes the check of installed OS name after upgrade

@ydirson
Copy link
Contributor Author

ydirson commented Oct 17, 2024

a few fixups:

  • better log
  • fix firstboot version check for successive-upgrade tests (otherwise not included in that branch)
  • sort checks for firstboot stamps to match their creation ordering

@ydirson
fixup! install 7/n: answerfile generation
d8a1ccf 
This gets back to DHCP, as static IPs are not needed for this part of
the job.
@ydirson
Copy link
Contributor Author

ydirson commented Oct 17, 2024

And last fixup: the long-talked-about "back to DHCP"

gthvn1
gthvn1 approved these changes Oct 18, 2024
View reviewed changes
Copy link
Contributor

@gthvn1 gthvn1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM,
hats off for the PR (french translation so I hope it makes sense :) )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gduperrey gduperrey gduperrey left review comments

@benjamreis benjamreis benjamreis approved these changes

@gthvn1 gthvn1 gthvn1 approved these changes

@stormi stormi Awaiting requested review from stormi

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ydirson @stormi @gthvn1 @benjamreis @gduperrey