Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge master to feature/py3 #5665

Conversation

stephenchengCloud
Copy link
Contributor

Sync with the lastest master branch.

freddy77 and others added 30 commits April 27, 2024 08:10
fail is always used in combination with Printf.sprintf so combine the two.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
More compact code.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
"exe" variable is compute in mutiple functions, not very expensive to
compute once.
Reduce code.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Make sure we release the "sock" file descriptors in all cases.
Add test trying to reproduce the issue passing an invalid file descriptor; not
a perfect reproduction but failure in this function can happen for instance if
daemon is restarted.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
We just want information about the current process.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
It is not safe to access a global hashtable from multiple threads, even if the operations are read-only
(it may be concurrently changed by another thread, which then may result in errors in the racing thread).

This means we must always take the mutex, and because OCaml doesn't have a reader-writer mutex, we need to take the exclusive mutex.

Eventually we should use a better datastructure here (immutable maps, or lock-free datastructures), but for now
fix the datastructure that we currently use to be thread-safe.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
In preparation for OCaml 5, on OCaml 4 they'd be equivalent.

Note that adding Atomic doesn't make operations on these values always atomic: that is
the responsibility of surrounding code.
E.g. Atomic.get + Atomic.set is not atomic, because another domain might've raced and changed the value inbetween
(so in that case Atomic.compare_and_set should be used).

However for global flags that are read multiple times, but set from a central place this isn't a problem.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Sets `observe` to `false` until at least one tracer provider is enabled.
If all tracer providers are disabled we set it back to `false`.

Prior to this `observe` seemed to be always `true` (apart from unit
tests). This would cause `with_tracing` to spam the logs with warnings
`No provider found...` until at least one tracer provider is enabled.

By setting `observe` to `false` as default and updating it depending on
the state of the tracer providers, `with_tracing` should now execute no
extra operations. Therefore, we avoid spamming the logs unnecessarily.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Create a new unit test file `test_tracing.ml` for the `tracing` library.
Add tests for `create`/`set`/`destroy` tracer providers.

Now that we change the library to have `observe` modes based on whether
or not we have `tracer providers` enabled, we want to make sure that
functions on applied on tracer providers set the correct mode for the
library.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Removes code duplication in `storage_mux.ml` by using the
already existing `with_dbg` implementation from `debuginfo` module.

This should lower the chances of unintentionally introducing bugs by
having to maintain two version of the same functions. e.g. Not using the
no op when tracing is disabled and generating unwanted warning messages.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Moves the following:

- `create` under `TracerProvider`;
- `set` under `TracerProvider`;
- `destroy` under `TracerProvider;
- `get_tracer_providers` unde `TracerProvider`;
- `get_tracer` under `Tracer`.

Adds documentation for `TracerProvider` module.

It kept being confusing of what `Tracing.set` does unless I was going through
the implementation again and again. Therefore, I moved some of the
functions so that their functionality becomes (hopefully) more intuitive.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
This is mostly copied from the jekyll version at xapi-project.github.io and
amended for Hugo. The main difference is that all XenAPI pages have now been
integrated with the main menu in the left side bar, including the menu of the
XenAPI classes.

Some automation is left to do in order to take the `doc/data` files from a
build of `ocaml/idl` and update the class pages using the script
`doc/make-class-pages.py`. Also the overview of xapi releases is still missing,
as well as some of the API guides.

Finally, there is some overlap with markdown files in `ocaml/doc`, which needs
to be sorted out.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
From Xen 4.19 onwards the legacy paths disappeared and the only valid path for
pygrub is /usr/libexec/xen/bin/pygrub. This path has always been preferred, but
now it's mandatory.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Add a new field `cluster_stack_version` to the cluster datamodel to
track the version of corosync currently in use. This version will
always be set to 3. Also add logic to switch corosync binary and
associated library versions when a cluster is created, if needed.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
This is purely for testing purpose. Normal user is not allowed to create
a corosync2 cluster.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
New XAPI conf option: `disable-webserver` (default: false)
If true, all requests for the fileserver will get a 403 response.

Signed-off-by: Benjamin Reis <benjamin.reis@vates.tech>
…ewing certificates in PS 5.1 and connecting to the server.

Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
In case of getting the list of files we called Unix.readlink twice for every file
descriptor.
In case we wanted just the count we computed the list anyway.
Factor out code to avoid calling function twice and do not compute the list
is not needed.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
The VM power state should be preserved on a suspended VM import.

In commit ebb58a8, the power state and suspend VDI in `vm_record` were
reset on importing a suspended VM. This meant to facilitate the
following `Client.VM.create_from_record`. But this gets the restore of
power state and suspend VDI broken as it requires the data in
`vm_record`.

This commit preserves the data in `vm_record` and reset it just before
it being passed to `Client.VM.create_from_record`.

Signed-off-by: Ming Lu <ming.lu@cloud.com>
CA-392930: Fixed exception handling which prevents connection
Instruments `xapi_session.ml` to create span around session logins.

Adds new attribute for spans:

- `xs.xapi.session.originator`

where available.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
…orosync3-basic

CP-48027: Corosync upgrade add `cluster_stack_version` datamodel change
…392836

CA-392836,CA-392847: Lost the power state on suspended VM import
This forces the failure on a host that is trying to perform corosync
upgrade. There are ways to recover: if the failure happens early, before
the cluster is created in the DB, then a recreate ought to fix the
problem. This happens when the corosync upgrade fails on the
coordinator.

If the failure happens after the cluster is created on a pool member,
then a `pool-resync` should help retry this upgrade.

Hopefully this can simulate some of the failure paths, but is by no
means exhaustive. Other more complicated failures are not easily
recoverable and therefore not simulated for now.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Refactor the representation of Kerberos Domain Controller (KDC) to better
support IPv6. It became apparent that the current implementation assumes
standard port 88 being used by a KDC - which we should relax in the
future.

Capture a KDC configuration as KDC.t and provide functions to create,
serialise, log a value. Represent the IP address as Ipaddr.t (which can
be IPv6 or IPv4).

The existing implementation does not properly use a domain controller's
port number that it obtains from a `net lookup kdc`:

* the port number is not passed to other net commands that query the
  domain controller

* the port number is not stored in the xapi database (along the IP
  address).

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
Vincent-lau and others added 26 commits May 22, 2024 17:40
…orosync3-upgrade-fist

CP-49635: Add FIST point for corosync upgrade
This unblocks the Storage: BST to use distributed tracing as the sr_uuid
parameter is sometimes not given.

Signed-off-by: Steven Woods <steven.woods@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
It's unused outside of the repository

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…-391381

CA-391381: Avoid errors for Partial Callables in observer.py
This reverts commit 599a9c6.

Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>
Remove CVM and relevant test cases

CVM had been supported but not since long time ago.

This commit is to clean up the remanent code and test cases.

Signed-off-by: Ming Lu <ming.lu@cloud.com>
…CP-49668

CP-49668: Revert definition of the old 'vm_anti_affinity' feature flag
Adds mising dependencies and uses correct formula for enforcing same version

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Add the missing dependency on xapi-stdext-threads

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
With newer dune versions, unused bindings for binaries are also checked.

The biggest chunks of code dropped are from tests:
- the ones in stdext-encodings were missed when removing the tests, this was
  done because they were unused by xapi.
- the export in rrd was never run, it's not trivial to get working due to
  changes that have happened in the meantime, and there were tests introduced
  in that same commit to ensure the data contains sane values.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This enables more warnings and fixes opam metadata generation

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Calls `varstore-sb-state` to edit the uefi mode of a VM
Takes in input the uuid of a VM and a mode (`setup` or `user`)

Returns the output of the script calls

Signed-off-by: Benjamin Reis <benjamin.reis@vates.tech>
Returns the SecureBoot status of a VM:
- `not_supported`: VM's firmware is not UEFI
- `disabled`: Secureboot is disabled on this VM
- `first_boot`: Secured boot is enabled on this VM and its NVRAM.EFI-variables is empty
- `ready`: Secured boot is enabled on this VM and PK, KEK, db and dbx are defined in its EFI variables
- `ready_no_dbx`: Secured boot is enabled on this VM and PK, KEK, db but not dbx are defined in its EFI variables
- `setup_mode`: Secured boot is enabled on this VM and PK is not defined in its EFI variables
- `certs_incomplete`: Secured boot is enabled on this VM and the certificates defines in its EFI variables are incomplete

Signed-off-by: Benjamin Reis <benjamin.reis@vates.tech>
Returns a pool's state for guest SecureBoot:
- `ready`: the active pool UEFI certificates (custom ones first, default ones if no custom ones) contain PK, KEK, db and dbx
- `ready_no_dbx`: the active pool UEFI certificates contain PK, KEK and db but not dbx
- `not_ready`: otherwise

Signed-off-by: Benjamin Reis <benjamin.reis@vates.tech>
Currently, the storage layer tracks two extra SR health statuses that XAPI is
unaware of. These are "unreachable" and "unavailable". At present, the storage
side simply maps these to "recovering" when sending information to the
toolstack. This change will allow the storage layer to track more accurate SR
health statuses.

Signed-off-by: Colin James <colin.barr@cloud.com>
@liulinC liulinC merged commit 3810bca into xapi-project:feature/py3 Jun 3, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.