Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: print packages' timestamp in job 'install' #2674

Merged
merged 3 commits into from
May 23, 2022
Merged

ci: print packages' timestamp in job 'install' #2674

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d19329e
ci: print packages' timestamp in job 'install'
umarcor May 19, 2022
7b6109c
ci/scripts/install: format
umarcor May 19, 2022
e3f409f
ci/install: print timestamp and hash as step summary content
umarcor May 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 40 additions & 38 deletions .github/ci/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash

INSTALL_DIR="$(pwd)/install"
mkdir -p $INSTALL_DIR
Expand All @@ -8,18 +8,18 @@ source $(dirname "$0")/setup-and-activate.sh

heading "Installing gsutil"
(
apt -qqy update && apt -qqy install curl
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
apt -qqy update && apt -qqy install google-cloud-cli
apt -qqy update && apt -qqy install curl
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
apt -qqy update && apt -qqy install google-cloud-cli
)
echo "----------------------------------------"

heading "Set environment variables for F4PGA CLI utils"
{
export F4PGA_FAM=xc7
export F4PGA_ENV_BIN="$(cd $(dirname "$0"); pwd)/../../env/conda/envs/symbiflow_arch_def_base/bin"
export F4PGA_ENV_SHARE="$(cd $(dirname "$0"); pwd)/../../install/share/symbiflow"
export F4PGA_FAM=xc7
export F4PGA_ENV_BIN="$(cd $(dirname "$0"); pwd)/../../env/conda/envs/symbiflow_arch_def_base/bin"
export F4PGA_ENV_SHARE="$(cd $(dirname "$0"); pwd)/../../install/share/symbiflow"
}

echo "----------------------------------------"
Expand All @@ -34,47 +34,49 @@ echo "----------------------------------------"

heading "Running installed toolchain tests"
(
pushd build
export VPR_NUM_WORKERS=${MAX_CORES}
export CTEST_OUTPUT_ON_FAILURE=1
ctest -R binary_toolchain_test_xc7* -j${MAX_CORES}
popd
pushd build
export VPR_NUM_WORKERS=${MAX_CORES}
export CTEST_OUTPUT_ON_FAILURE=1
ctest -R binary_toolchain_test_xc7* -j${MAX_CORES}
popd
)
echo "----------------------------------------"

heading "Compressing install dir (creating packages)"
(
rm -rf build

# Remove symbolic links and copy content of the linked files
for file in $(find install -type l)
do cp --remove-destination $(readlink $file) $file
done

du -ah install
export GIT_HASH=$(git rev-parse --short HEAD)
tar -I "pixz" -cvf symbiflow-arch-defs-install-${GIT_HASH}.tar.xz -C install share/symbiflow/techmaps share/symbiflow/scripts environment.yml
tar -I "pixz" -cvf symbiflow-arch-defs-benchmarks-${GIT_HASH}.tar.xz -C install benchmarks
for device in $(ls install/share/symbiflow/arch)
do
tar -I "pixz" -cvf symbiflow-arch-defs-$device-${GIT_HASH}.tar.xz -C install share/symbiflow/arch/$device
done
rm -rf install
rm -rf build

# Remove symbolic links and copy content of the linked files
for file in $(find install -type l)
do cp --remove-destination $(readlink $file) $file
done

du -ah install
export GIT_HASH=$(git rev-parse --short HEAD)
tar -I "pixz" -cvf symbiflow-arch-defs-install-${GIT_HASH}.tar.xz -C install share/symbiflow/techmaps share/symbiflow/scripts environment.yml
tar -I "pixz" -cvf symbiflow-arch-defs-benchmarks-${GIT_HASH}.tar.xz -C install benchmarks
for device in $(ls install/share/symbiflow/arch)
do
tar -I "pixz" -cvf symbiflow-arch-defs-$device-${GIT_HASH}.tar.xz -C install share/symbiflow/arch/$device
done
rm -rf install
)
echo "----------------------------------------"

GCP_PATH=symbiflow-arch-defs/artifacts/prod/foss-fpga-tools/symbiflow-arch-defs/continuous/install

heading "Uploading packages"
(
if [ "$UPLOAD_PACKAGES" = "true" ]; then
TIMESTAMP=$(date +'%Y%m%d-%H%M%S')
for package in $(ls *.tar.xz)
do
gsutil cp ${package} gs://${GCP_PATH}/${TIMESTAMP}/
done
else
echo "Not uploading packages as not requested by the CI"
fi
if [ "$UPLOAD_PACKAGES" != "true" ]; then
echo "Not uploading packages as not requested by the CI"
exit 0
fi
TIMESTAMP=$(date +'%Y%m%d-%H%M%S')
echo "> Timestamp: $TIMESTAMP"

echo 'Timestamp: $TIMESTAMP' >> $GITHUB_STEP_SUMMARY
echo 'Hash: '"$(echo ${package} | sed 's/.*-\(.*\)\.tar\.xz/\1/')" >> $GITHUB_STEP_SUMMARY

for package in $(ls *.tar.xz); do gsutil cp ${package} gs://${GCP_PATH}/${TIMESTAMP}/; done
)
echo "----------------------------------------"