Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{CI} Add Test Rpm Package Mariner #25747

Merged
merged 3 commits into from
Mar 13, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -683,6 +683,53 @@ jobs:
TargetPath: $(Build.ArtifactStagingDirectory)
ArtifactName: $(artifact)

- job: TestRpmPackageMariner
displayName: Test Rpm Package Mariner
timeoutInMinutes: 120
dependsOn:
- BuildRpmPackageMariner
- ExtractMetadata
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI', 'Manual', 'Schedule'))
pool:
name: $(pool)
strategy:
matrix:
2.0 AMD64:
image: mcr.microsoft.com/cbl-mariner/base/core:2.0
artifact: rpm-mariner2.0-amd64
pool: ${{ variables.ubuntu_pool }}
2.0 ARM64:
image: mcr.microsoft.com/cbl-mariner/base/core:2.0
artifact: rpm-mariner2.0-arm64
pool: ${{ variables.ubuntu_arm64_pool }}
steps:
- task: DownloadPipelineArtifact@1
displayName: 'Download Metadata'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/metadata'
artifactName: metadata

- task: DownloadPipelineArtifact@1
displayName: 'Download Build Artifacts'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/rpm'
artifactName: $(artifact)

- bash: ./scripts/ci/install_docker.sh
displayName: Install Docker

- bash: |
set -ex

CLI_VERSION=`cat $SYSTEM_ARTIFACTSDIRECTORY/metadata/version`
RPM_NAME=$(find $SYSTEM_ARTIFACTSDIRECTORY/rpm/ -type f -name "azure-cli-$CLI_VERSION-1.cm2.*.rpm" -printf '%f\n')

echo "== Test rpm package on ${IMAGE} =="
docker pull $IMAGE
docker run --rm -e RPM_NAME=$RPM_NAME -v $SYSTEM_ARTIFACTSDIRECTORY/rpm:/mnt/rpm -v $(pwd):/azure-cli $IMAGE /bin/bash "/azure-cli/scripts/release/rpm/test_rpm_in_docker.sh"

displayName: 'Test Rpm Package Mariner'

# TODO: rpmbuild on Red Hat UBI 8 is slow for unknown reason. Still working with Red Hat to investigate.
- job: BuildRpmPackages
displayName: Build Rpm Packages
Expand Down
32 changes: 32 additions & 0 deletions scripts/release/rpm/test_mariner_in_docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env bash

# This script should be run in a centos7 docker.
set -exv

export USERNAME=azureuser

tdnf --nogpgcheck install /mnt/rpm/$RPM_NAME -y

tdnf install git gcc python3-devel python3-pip findutils -y
Copy link
Contributor Author

@bebound bebound Mar 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some differences with test_rpm_in_docker.sh

  1. Mariner does not install pip when install python3-devel.
  2. dnf becomes tdnf.
  3. pip is also in /usr/bin/ by default.

I don't want to add the fifth env variables, so I create a new script for Mariner.

Comment on lines +1 to +10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this script be reused from test_rpm_in_docker.sh by introducing some parameters?

Copy link
Contributor Author

@bebound bebound Mar 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some differences between Mariner and UBI, see #25747 (comment).

Using same script decreases the duplicate code but increases the complexity of script.
We've used different dockerfile for Mariner and UBI. We can also use different test scripts.


ln -s -f /usr/bin/python3 /usr/bin/python
time az self-test
time az --version

cd /azure-cli/
pip install wheel
./scripts/ci/build.sh

# From Fedora36, when using `pip install --prefix` with root privileges, the package is installed into `{prefix}/local/lib`.
# In order to keep the original installation path, I have to set RPM_BUILD_ROOT
# Ref https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/developers/Development_Python/#_pipsetup_py_installation_with_prefix
export RPM_BUILD_ROOT=/

pip install pytest --prefix /usr/lib64/az
pip install pytest-xdist --prefix /usr/lib64/az
pip install pytest-forked --prefix /usr/lib64/az

find /azure-cli/artifacts/build -name "azure_cli_testsdk*" | xargs pip install --prefix /usr/lib64/az --upgrade --ignore-installed
find /azure-cli/artifacts/build -name "azure_cli_fulltest*" | xargs pip install --prefix /usr/lib64/az --upgrade --ignore-installed --no-deps

python /azure-cli/scripts/release/rpm/test_rpm_package.py