Merge master to feature/non-cdn-update
#5889
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Lin Liu <lin.liu@citrix.com>
CA-389241: import-update-key compat with xs8 and xs9
* CP-46944: Update yum plugins to dnf plugins Signed-off-by: Lin Liu <lin.liu@citrix.com> * CP-46944: Update unittest to make github CI happy github CI has issues with - ubuntu-22.04 container - python11 - python3-dnf Instead of patching the upstreams, we just provide a stub for the fix Signed-off-by: Lin Liu <lin.liu@citrix.com> --------- Signed-off-by: Lin Liu <lin.liu@citrix.com>
* Feature branch sync
* CP-45921: Use dnf as package manager for XS9 Given XS9 has updated to dnf and no yum is available, xapi will choose package manager basing on following - If dnf exists, use dnf - otherwise, fallback to yum xapi just presume dnf or yum is available in the system. Because xapi decides to use dnf or yum according to the running environment, this commit is compatible with yum/xs8 and dnf/xs9 Signed-off-by: Lin Liu <lin.liu@citrix.com> * CP-45921: Code refine for - Move test code to its own suite - Replace `active` with `manager`, which is static - Add cmd_line record type to abstract cmd line type - Refine doc comment and license --------- Signed-off-by: Lin Liu <lin.liu@citrix.com>
The default gpg public keyring database updated - pubring.kbx for new gpg on XS9 - pubring.gpg for old gpg on XS8 Detect whether pubring.kbx exists and fallback to old one for XS8 Signed-off-by: Lin Liu <lin.liu@citrix.com>
…project#5564) * CP-48221: Update plugin detect package manager dynamically Given XS9 has deplicated yum with dnf, to be compatible with XS8, - Use dnf if dnf is detected - Fallback to yum otherwise Signed-off-by: Lin Liu <lin.liu@citrix.com> * CP-48221: Make CI happy with various scans --------- Signed-off-by: Lin Liu <lin.liu@citrix.com> Co-authored-by: Luca Zhang <feiya.zhang@cloud.com>
Routine feature branch sync
Prior to this commit, the xapi on the coordinator host records the 'Unix.gettimeofday' as the timestamps of the heartbeat with other pool supporter hosts. When the system clock is updated with a huge jump forward, the timestamps would be far back into the past. This would cause the xapi assumes that the hosts are offline as long time no heartbeats. In this commit, the timestamps are changed to get from a monotonic clock. In this way, the system clock changes will not impact the heartbeats' timestamps any more. Additionally, Host_metrics.last_updated is only set when the object is created. It's useless in check_host_liveness at all. Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Stephen Cheng <stephen.cheng@cloud.com>
Signed-off-by: Stephen Cheng <stephen.cheng@cloud.com>
…tephenche/sync_xs9_with_master Routine sync xs9 with master
Also move the last function to Xapi_stdext. Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Note we are only checking for default of new parameters in messages that already have default parameters. It lacks checks for new parameters added to messages with no default parameters, but there is no easy way to achieve that because the generator cannot tell what is new and what is not, moreover, there are existing messages with non-default parameters introduced in different releases. Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
It forced users to define how the elements were concatenated, allowing usage to be inconsistent. Fortunately this was not the case, but it's better to define the concatenation in a single place and reduce possible mistakes. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
It forced users to define how the elements were concatenated, allowing usage to be inconsistent. Fortunately this was not the case, but it's better to define the concatenation in a single place and reduce possible mistakes. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Lin Liu <Lin.Liu01@cloud.com>
Concatenates the query value back after Uri.query splits the string separated by commas. Signed-off-by: Andrii Sultanov <andrii.sultanov@cloud.com>
) * CP-50078: Check if SMAPI observation is enabled Modifies xapi-storage-script to determine if SMAPI observation is enabled by checking whether a configuration file exists for it within the file system. Internally, xapi queries the database to determine if a specific component should be observed but as xapi-storage-script is an external daemon, we rely on the file system. To avoid modifying much of the monadic structure, a configuration record is used to record whether observation is enabled. It is hoped this less invasive change will make it simpler to rework xapi-storage-script in the future. The path of the observer configuration directory is relocated to xapi-consts. Signed-off-by: Colin James <colin.barr@cloud.com> * CP-50078: Augment RPC transport to use observer.py Modifies the RPC transport (fork_exec_rpc) to take a "dbg" parameter that can encode a traceparent. If observation is enabled and a traceparent can be parsed from the dbg parameter, the storage plugin's script will be invoked via observer.py (for automatic instrumentation). This change does not yet seed each of the transports with the dbg parameter; the volume, datapath, etc. interfaces - defined in xapi-storage - use the convention of having dbg as the first parameter for each RPC function. The act of supplying this dbg parameter to the transport could be done implicitly, but this would be a contract that is hard to enforce, despite vastly reducing the pervasiveness of changes to come; each handler will need to partially apply its received dbg parameter to the RPC transport (e.g. volume_rpc ~dbg) at each usage site. The python3 path is relocated from xapi_globs.ml to become a constant within xapi-consts. Signed-off-by: Colin James <colin.barr@cloud.com> * CP-50078: Augment RPC transports for tracing This change is rather pervasive but it seeds the volume and datapath RPC transport with the received "dbg" parameter at each usage site. The change could be done implicitly if we could be promised that "dbg" always occurs as a first parameter - we would then just modify the transport to be aware of this contract and not have to partially apply it at each usage site. Signed-off-by: Colin James <colin.barr@cloud.com> --------- Signed-off-by: Colin James <colin.barr@cloud.com>
Use `finally` to execute `complete_tracing` on the context when we are done and not at the beginning. This allows for instrumentation and correct trace display of functions such as `assert_op_valid`. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Update `with_tracing` to record the error and finish the span if the instrumented function fails. Add `session.track.id` and `task.id` attributes to all spans inside a trace that is following a task. This will result in spans being finished even if the function has raised an exception. Plus, better debuggability in the case that something fails. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Instruments task relatated functions. This will allow for better debuggablity and understanding of issues related with tasks. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
`Date.now` is preffered for synchronisation between host as it uses `ptime`. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
This signature is completely unused. We could instead generate a client.mli, but that is more complicated, currently the client.mli it'd generate wouldn't be polymorphic enough. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Server.ml takes a while to compile, but most unit tests don't actually need it.
Reorganize Api_server into Api_server+Api_server_common, where the latter
suffices for unit tests.
'dune runtest' times are improved:
```
hyperfine --min-runs 2 'dune clean; dune runtest --cache=disabled' 'cd ../scm-prev; dune clean; dune runtest --cache=disabled'
Benchmark 1: dune clean; dune runtest --cache=disabled
Time (mean ± σ): 103.491 s ± 1.596 s [User: 374.464 s, System: 125.957 s]
Range (min … max): 102.363 s … 104.620 s 2 runs
Benchmark 2: cd ../scm-prev; dune clean; dune runtest --cache=disabled
Time (mean ± σ): 114.158 s ± 2.980 s [User: 380.638 s, System: 134.558 s]
Range (min … max): 112.051 s … 116.266 s 2 runs
Summary
dune clean; dune runtest --cache=disabled ran
1.10 ± 0.03 times faster than cd ../scm-prev; dune clean; dune runtest --cache=disabled
```
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
`sexpr` is now fully thread safe without having to use locks, doesn't need to depend on threadext. `gen_api_main` can use the external `uuidm` module directly, without waiting for internal one to be built. `dune-build-info` is only needed by xapi_version. `xapi-stdext-unix` is not needed in `xapi-idl` The sexplib ppx runtime also doesn't need to be linked in some libraries that do not use it anymore, and where it is used it'll be automatically linked. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Xapi_version depends on Build_info which can change on every commit. It is better to remove it from the dependencies of gen_api_main, especially that gen_api_main is on the critical path for discovering more dependencies. The 'xapi_user_agent' constant got moved to Xapi_version. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
It'll be slower, but it can run a lot earlier in the build process. Compiling the datamodels takes time, but compiling them for bytecode is faster. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Introduce a _minimal library, so we can start compiling server.ml earlier. Build time reduced from 80s to: ``` Benchmark 1: dune clean; dune build --cache=disabled Time (mean ± σ): 67.081 s ± 0.190 s [User: 326.847 s, System: 103.668 s Range (min … max): 66.946 s … 67.215 s 2 runs ``` Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Remove most sleeps, and reduce test duration from 5 seconds to 1. (If we do want to run a performance test we can increase these again) Run just a basic test for 0.1 seconds instead of a performance test for 5s by default. (can still be tweaked by overriding SECS) Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Instead of every 5s. Speeds up testing, and may speed up startup somewhat. And a connection try once every 0.5s won't create a lot of load on the system. (If needed we could implement some form of exponential backoff here). ``` Benchmark 1: dune clean; dune runtest --cache=disabled Time (mean ± σ): 97.642 s ± 0.933 s [User: 354.132 s, System: 113.436 s] Range (min … max): 96.982 s … 98.302 s 2 runsi ``` Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…aintenance2 Reduce build/test times
These are errors in dune 3.15 and don't seem to be problematic Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Also make dune generate the opam metadata Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Only tests need it to generate crypto keys, but it's needed to create the serial when signing certificates. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
BengangY
force-pushed
the
private/bengangy/merge-master-to-non-cdn-update
branch
from
b4cc389
to
a03b3da
Compare
gangj
reviewed
Jul 29, 2024
There was a problem hiding this comment.
Let's see if reviewable is able to review this merging commit
Reviewable status: 0 of 287 files reviewed, all discussions resolved
gangj
reviewed
Jul 29, 2024
There was a problem hiding this comment.
Seems no
Reviewable status: 0 of 287 files reviewed, all discussions resolved
gangj
approved these changes
Jul 29, 2024
There was a problem hiding this comment.
Reviewed 286 of 287 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on @BengangY)
psafont
approved these changes
Jul 29, 2024
gangj
merged commit d7682dc
into
xapi-project:feature/non-cdn-update
14 of 15 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolved conflicts:
This change is