Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Routine sync xs9 with master #5814

Conversation

stephenchengCloud
Copy link
Contributor

No description provided.

psafont and others added 30 commits May 24, 2024 17:31
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
It's unused outside of the repository

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…-391381

CA-391381: Avoid errors for Partial Callables in observer.py
Check feature flag in these places:
1. VM start.
2. Host evacuation. When this PR is raised, the host evacuation PR is
   still in review. So this PR doesn't include the checking for host
   evacuation. It will be included in another new PR.
3. Create VM group.
4. VM.set_groups. Adding VMs to a group and removing VMs from a group
   are all forbidden. If customers need to remove VMs from a group,
   just destroy the group.
5. Send VM anti-affinity alerts.
Also, based on our discussion, the name of feature is changed from
`VM_anti_affinity` to `VM_group`.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
…P-48011-review

CP-48011: XAPI Support anti-affinity feature check
This reverts commit 599a9c6.

Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>
Control domains, templates, and snapshots cannot be added to
VM groups.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Add a `-skip-xapi` flag that avoids connecting to XAPI.
Useful if you just want to query the list of tests,
or execute a specific test that relies on external programs
(and thus cannot be a unit test)
, but doesn't require XAPI.

This will be useful for running etcd tests with 'dune exec'.

Also add a '--' flag to be able to pass flags to alcotest directly.

E.g. `dune exec ./quicktest.exe -- -skip-xapi -- list`
The first '--' separates 'dune' arguments from 'quicktest.exe' arguments,
and the 2nd one separates legacy quicktest CLI arguments from alcotest
arguments.

To ensure this keeps working properly add a runtest that just list the tests.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Basic read/write operation, and watches with and without timeouts.

The test can be run with:
```
./quicktest.exe -skip-xapi -- test xenstore -v
```

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Remove CVM and relevant test cases

CVM had been supported but not since long time ago.

This commit is to clean up the remanent code and test cases.

Signed-off-by: Ming Lu <ming.lu@cloud.com>
…CP-49668

CP-49668: Revert definition of the old 'vm_anti_affinity' feature flag
…rom_master

Update feature/vm-anti-affinity by merging from master
Adds mising dependencies and uses correct formula for enforcing same version

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Add the missing dependency on xapi-stdext-threads

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Add func host_to_vm_count_map to be used, rename RefMap to HostMap

Signed-off-by: Gang Ji <gang.ji@citrix.com>
fixup review comments

Signed-off-by: Gang Ji <gang.ji@citrix.com>
Signed-off-by: Gang Ji <gang.ji@citrix.com>
With newer dune versions, unused bindings for binaries are also checked.

The biggest chunks of code dropped are from tests:
- the ones in stdext-encodings were missed when removing the tests, this was
  done because they were unused by xapi.
- the export in rrd was never run, it's not trivial to get working due to
  changes that have happened in the meantime, and there were tests introduced
  in that same commit to ensure the data contains sane values.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This enables more warnings and fixes opam metadata generation

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…A-393421

CA-393421: Special VMs cannot be added to VM groups
Calls `varstore-sb-state` to edit the uefi mode of a VM
Takes in input the uuid of a VM and a mode (`setup` or `user`)

Returns the output of the script calls

Signed-off-by: Benjamin Reis <benjamin.reis@vates.tech>
Host evacuation plan with anti-affinity support will be carried out in 3
phases:

1. Try to get a "spread evenly" plan for anti-affinity VMs, done if
   all the rest VMs got planned using binpack, otherwise continue.
2. Try to get a "no breach" plan for anti-affinity VMs, done if all the
   rest VMs got planned using binpack, otherwise continue.
3. Carry out a binpack plan ignoring VM anti-affinity.

Signed-off-by: Gang Ji <gang.ji@citrix.com>
Add "groups" "power_state" for VM, vm set_resident_on host

Signed-off-by: Gang Ji <gang.ji@citrix.com>
Signed-off-by: Gang Ji <gang.ji@citrix.com>
…ge_master_to_feature

Merge master to Go SDK feature branch
psafont and others added 23 commits July 8, 2024 11:08
…ov/empty-length-checking

IH-640: Eliminate unnecessary usage of List.length to check for empty lists
Signed-off-by: Andriy Sultanov <53952748+last-genius@users.noreply.github.com>
…ov/opt-refactoring

IH-633: Transition away from exception-raising Hashtbl.find and Unix.getenv
IH-621: Add IPMI host power on support and remove DRAC
…ls-config

CA-392887: `set_tls_config` immediately after enabling clustering
….getenv_opt with")

Signed-off-by: Andrii Sultanov <andrii.sultanov@cloud.com>
…ov/opt-fix

Fixes: 99c4356 ("Transition from exception-raising Unix.getenv to Sys.getenv_opt with")
The xs-trace utility is restructured to enhance its readability and, to aid
with potential future extension, its CLI is reimplemented in terms of Cmdliner.

It is hoped that the current command modes are consistent with what was already
expected by xs-trace, i.e. xs-trace (cp|mv) <src> <endpoint>

These changes should make it simpler to extend this utility with more
functionality. For example, there is an idea to add some short conversion
routines from Zipkinv2 to Google's Catapult trace format - so that single
host triaging can bypass heavy distributed tracing services and use
functionality built into Chrome (or the online Perfetto trace viewer).

Signed-off-by: Colin James <colin.barr@cloud.com>
IH-642: Restructure xs-trace to use Cmdliner
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Use the Atomic module to track whether a watcher has been created.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
This allows one to force sync the state of xapi db with the cluster
stack, useful for cluster API methods change the state of the cluster.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Previously there were two ways an alert for a cluster host join/leave
can be raised: 1. through the cluster change watcher; 2. through the api
call. These two can generate duplicate alerts as an API call can cause
the cluster change watcher to notice the change as well.

The idea of the fix here is still to let API and watcher raise alerts
separately, but now add synchronous API calls to allow API call
(cluster-host-join, etc) to call
the cluster change update code at the right time so that the cluster
change watcher won't see the change again, hence not generating duplicate
alerts.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
…ess-alert

CA-394109: Reduce number of alerts
The new fingerprint_sha256 and fingerprint_sha1 fields will be empty
when upgrading from a version without the fields. This commit checks for
this and fills them in, stopping the certificate from being needlessly
reinstalled.

Signed-off-by: Steven Woods <steven.woods@citrix.com>
In nm.ml; helpers.ml.
Expands test cases, keeps the existent exception behaviour.

Signed-off-by: Andrii Sultanov <andrii.sultanov@cloud.com>
Signed-off-by: Stephen Cheng <stephen.cheng@cloud.com>
@stephenchengCloud stephenchengCloud force-pushed the private/stephenche/sync_xs9_with_master branch from ea0966b to f0721d8 Compare July 12, 2024 05:59
Signed-off-by: Stephen Cheng <stephen.cheng@cloud.com>
@stephenchengCloud
Copy link
Contributor Author

stephenchengCloud commented Jul 12, 2024

other.yml was new created file from main.yml. So there was a conflict when doing the merge.

[stephenche@39 xen-api]$ git show f0721d8789fde8c0ba8ffefd2e9e9920536c13a1
commit f0721d8789fde8c0ba8ffefd2e9e9920536c13a1
Merge: 392124671 55963c420
Author: Stephen Cheng <stephen.cheng@cloud.com>
Date:   Fri Jul 12 06:29:07 2024 +0100

    Merge branch 'master' into private/stephenche/sync_xs9_with_master
    
    Signed-off-by: Stephen Cheng <stephen.cheng@cloud.com>

diff --cc .github/workflows/other.yml
index 000000000,57d4c1d22..3ea9bd149
mode 000000,100644..100644
--- a/.github/workflows/other.yml
+++ b/.github/workflows/other.yml
@@@ -1,0 -1,149 +1,150 @@@
+ name: Build and test (other)
+ 
+ on:
+   push:
+   pull_request:
+   schedule:
+     # run daily, this refreshes the cache
+     - cron: "13 2 * * *"
+ 
+ concurrency: # On new push, cancel old workflows from the same PR, branch or tag:
+   group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+   cancel-in-progress: true
+ 
+ jobs:
+   python-test:
+     name: Python tests
+     runs-on: ubuntu-22.04
+     strategy:
+       fail-fast: false
+       matrix:
+         python-version: ["2.7", "3.11"]
+     steps:
+       - name: Checkout code
+         uses: actions/checkout@v4
+         with:
+           fetch-depth: 0 # To check which files changed: origin/master..HEAD
+       - uses: LizardByte/setup-python-action@master
+         with:
+           python-version: ${{matrix.python-version}}
+ 
+       - uses: actions/cache@v4
+         name: Setup cache for running pre-commit fast
+         with:
+           path: ~/.cache/pre-commit
+           key: pre-commit|${{ env.pythonLocation }}|${{ hashFiles('.pre-commit-config.yaml') }}
+ 
+       - run: echo "::add-matcher::.github/workflows/python-warning-matcher.json"
+         name: "Setup GitHub for reporting Python warnings as annotations in pull request code review"
+ 
+       - uses: pre-commit/action@v3.0.1
+         name: Run pre-commit checks (no spaces at end of lines, etc)
+         if: ${{ matrix.python-version != '2.7' }}
+         with:
+           extra_args: --all-files --verbose --hook-stage commit
+         env:
+           SKIP: no-commit-to-branch
+ 
+       - name: Install dependencies only needed for python 2
+         if: ${{ matrix.python-version == '2.7' }}
+         run: pip install enum
+ 
+       - name: Install dependencies only needed for python 3
+         if: ${{ matrix.python-version != '2.7' }}
+         run: pip install opentelemetry-api opentelemetry-exporter-zipkin-json opentelemetry-sdk pandas pytype toml wrapt
+ 
+       - name: Install common dependencies for Python ${{matrix.python-version}}
+         run: pip install future mock pytest-coverage pytest-mock
+ 
+       - name: Run Pytest for python 2 and get code coverage
+         if: ${{ matrix.python-version == '2.7' }}
+         run: >
+           pytest
+           --cov=scripts --cov=ocaml/xcp-rrdd
+           scripts/ ocaml/xcp-rrdd -vv -rA
+           --junitxml=.git/pytest${{matrix.python-version}}.xml
+           --cov-report term-missing
+           --cov-report xml:.git/coverage${{matrix.python-version}}.xml
+         env:
+           PYTHONDEVMODE: yes
+ 
+       - name: Run Pytest for python 3 and get code coverage
+         if: ${{ matrix.python-version != '2.7' }}
+         run: >
+           pytest
+           --cov=scripts --cov=ocaml/xcp-rrdd --cov=python3/
+           scripts/ ocaml/xcp-rrdd python3/ -vv -rA
+           --junitxml=.git/pytest${{matrix.python-version}}.xml
+           --cov-report term-missing
+           --cov-report xml:.git/coverage${{matrix.python-version}}.xml
+         env:
+           PYTHONDEVMODE: yes
++          PYTHONPATH: "python3:python3/tests/stubs"
+ 
+       - name: Upload coverage report to Coveralls
+         uses: coverallsapp/github-action@v2
+         with:
+           format: cobertura
+           files: .git/coverage${{matrix.python-version}}.xml
+           flag-name: python${{matrix.python-version}}
+           parallel: true
+ 
+       - uses: dciborow/action-pylint@0.1.0
+         if: ${{ matrix.python-version != '2.7' }}
+         with:
+           reporter: github-pr-review
+           level: warning
+           # To be customized to cover remaining Python scripts:
+           glob_pattern: "**/*.py"
+         continue-on-error: true
+ 
+       - name: Run pytype checks
+         if: ${{ matrix.python-version != '2.7' }}
+         run: ./pytype_reporter.py
+         env:
+           PR_NUMBER: ${{ github.event.number }}
+           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+           PYTYPE_REPORTER_DEBUG: True
+ 
+       - name: pyflakes
+         uses: reviewdog/action-pyflakes@v1
+         with:
+           github_token: ${{ secrets.github_token }}
+         continue-on-error: true
+ 
+   # For coverage of 2.7 and 3.11 we upload to Coveralls in parallel mode.
+   # To view the Coveralls results of the PR, click on the "Details" link to the right
+   # of the Coveralls Logo in the Checks section of the PR.
+   finish-parallel-coveralls-upload:
+     name: Finish coverage upload
+     needs: python-test  # run after the python-test has completed uploading coverages
+     runs-on: ubuntu-latest
+     steps:
+       - name: Finish the parallel coverage upload to Coveralls
+         uses: coverallsapp/github-action@v2
+         with:
+           parallel-finished: true
+         continue-on-error: true  # Do not fail CI if this step fails
+ 
+   deprecation-test:
+     name: Deprecation tests
+     runs-on: ubuntu-22.04
+ 
+     steps:
+       - name: Checkout code
+         uses: actions/checkout@v4
+ 
+       - name: Generate empty configuration for make to be happy
+         run: touch config.mk
+ 
+       - name: quality-gate
+         run: make quality-gate
+ 
+   test-sdk-builds:
+     name: Test SDK builds
+     uses: ./.github/workflows/generate-and-build-sdks.yml
+     with:
+       # Ensure you also update ocaml-tests
+       # when changing this value, to keep builds
+       # consistent
+       xapi_version: "v0.0.0"

@gangj gangj merged commit 9bf0e46 into xapi-project:feature/xs9 Jul 12, 2024
15 checks passed
@stephenchengCloud stephenchengCloud deleted the private/stephenche/sync_xs9_with_master branch July 31, 2024 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.