Skip to content

Commit

Permalink
Merge pull request #35 from 130s/add/dockerbased_prerelease
Browse files Browse the repository at this point in the history
Add docker-based ROS prerelease test.
  • Loading branch information
130s committed May 5, 2016
2 parents e56dc51 + d179d25 commit e68bf5f
Show file tree
Hide file tree
Showing 5 changed files with 228 additions and 45 deletions.
17 changes: 5 additions & 12 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,23 +12,16 @@ compiler:
# Whoever receives success and/or failure via email
env:
matrix:
# - ROS_DISTRO=hydro AFTER_INSTALL=""
# - ROS_DISTRO=hydro USE_CATKIN_MAKE=true BEFORE_SCRIPT="pwd; git clone http://github.com/jsk-ros-pkg/jsk_common_msgs" NOT_TEST_INSTALL=true
# - ROS_DISTRO=hydro AFTER_INSTALL="" TEST_PKGS=" "
# - ROS_DISTRO=hydro AFTER_INSTALL="" ROS_REPOSITORY_PATH=http://packages.ros.org/ros/ubuntu
# - ROS_DISTRO=hydro AFTER_INSTALL="mongodb-clients"
# - ROS_DISTRO=hydro BEFORE_SCRIPT="pwd; git clone https://github.com/ros-industrial/industrial_core"
- ROS_DISTRO=indigo BUILDER='not-compile'
- ROS_DISTRO=indigo BUILDER='not-compile' TRAVIS_REPO_SLUG=ros-industrial/industrial_ci
- ROS_DISTRO=indigo BUILDER='not-compile' TRAVIS_REPO_SLUG=ros-industrial/industrial_ci TRAVIS_BRANCH=master TRAVIS_COMMIT='' TRAVIS_PULL_REQUEST=''
- ROS_DISTRO=indigo NOT_TEST_BUILD='true'
- ROS_DISTRO=indigo NOT_TEST_INSTALL='true'
- ROS_DISTRO=indigo CATKIN_PARALLEL_JOBS='-p1' ROS_PARALLEL_JOBS='-j1' # Intend build on low-power platform
# - env: ROS_DISTRO=indigo PRERELEASE=true ## Comment out because this is meaningless for always failing without prerelease testable contents in industrial_ci.
- ROS_DISTRO=indigo PRERELEASE=true PRERELEASE_REPONAME=industrial_core PRERELEASE_DOWNSTREAM_DEPTH=0
- ROS_DISTRO=jade PRERELEASE=true PRERELEASE_REPONAME=warehouse_ros PRERELEASE_DOWNSTREAM_DEPTH=1
matrix:
allow_failures:
- env: ROS_DISTRO=jade BUILDER='not-compile'
- env: ROS_DISTRO=jade BUILDER='not-compile' TRAVIS_REPO_SLUG=ros-industrial/industrial_ci
- env: ROS_DISTRO=jade BUILDER='not-compile' TRAVIS_REPO_SLUG=ros-industrial/industrial_ci TRAVIS_BRANCH=master TRAVIS_COMMIT='' TRAVIS_PULL_REQUEST=''
- env: ROS_DISTRO=indigo PRERELEASE=true PRERELEASE_REPONAME=industrial_core PRERELEASE_DOWNSTREAM_DEPTH=0
- env: ROS_DISTRO=jade PRERELEASE=true PRERELEASE_REPONAME=warehouse_ros PRERELEASE_DOWNSTREAM_DEPTH=1 # Intended to fail (as of Apr 2016), to test the capability of capturing Prerelease Test failure.
before_script:
- CI_DIR=.ci_config
- mkdir $CI_DIR; cp * $CI_DIR # Need to copy, since directory starting from dot is ignoreed by catkin build. Also this folder name is used for testing both this repo itself and the "client" repositories
Expand Down
59 changes: 57 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,8 @@ You can configure the behavior in `.travis.yml` in your client repository.

Required environment variables:

* `ROS_REPOSITORY_PATH`: Location of ROS' binary repositories where depended packages get installed from (typically both standard repo and `"Shadow-Fixed" repository <http://wiki.ros.org/ShadowRepository>`_)
* `ROS_DISTRO`: Version of ROS (Indigo, Jade etc.).
* `ROS_REPOSITORY_PATH`: Location of ROS' binary repositories where depended packages get installed from (typically both standard repo (`http://packages.ros.org/ros/ubuntu`) and `"Shadow-Fixed" repository <http://wiki.ros.org/ShadowRepository>`_ (`http://packages.ros.org/ros-shadow-fixed/ubuntu`))
* `ROS_DISTRO`: Version of ROS in all lower case. E.g.: `indigo` / `jade`

Optional environment variables
++++++++++++++++++++++++++++++++
Expand All @@ -118,6 +118,9 @@ Note that some of these currently tied only to a single option, but we still lea
* `CI_PARENT_DIR` (default: .ci_config): (NOT recommended to specify) This is the folder name that is used in downstream repositories in order to point to this repo.
* `NOT_TEST_BUILD` (default: not set): If true, tests in build space won't be run.
* `NOT_TEST_INSTALL` (default: not set): If true, tests in `install` space won't be run.
* `PRERELEASE` (default: false): If `true`, run `prerelease test on docker that emulates ROS buildfarm <http://wiki.ros.org/bloom/Tutorials/PrereleaseTest/>`_. The usage of Prerelease Test feature is `explained more in this section <https://github.com/130s/industrial_ci/blob/add/dockerbased_prerelease/README.rst#optional-run-ros-prerelease-test>`_.
* `PRERELEASE_DOWNSTREAM_DEPTH` (0 to 4, default: 0): Number of the levels of the packages dependecies the Prerelease Test targets at. Range of the level is defined by ROS buildfarm (`<http://prerelease.ros.org>`_). Note that you might as well use `0` for the packages that are depended on many packages, because the job simply won't finish within Travis CI's time limit (50 minutes).
* `PRERELEASE_REPONAME` (default: not set): The target of prerelease test (that you select at `<http://prerelease.ros.org/indigo>`_, `<http://prerelease.ros.org/kinetic>`_ etc.) If not set then it tests the package of the repository's name. You can specify this by your ROS package name format (with underscore e.g. `industrial_core`), not Debian package name format.
* `PKGS_DOWNSTREAM` (default: explained): Packages in downstream to be tested. By default, `TARGET_PKGS` is used if set, if not then `BUILD_PKGS` is used.
* `ROS_PARALLEL_JOBS` (default: -j8): Maximum number of packages to be built in parallel by the underlining build tool. As of Jan 2016, this is only enabled with `catkin_tools` (with `make` as an underlining builder).
* `ROS_PARALLEL_TEST_JOBS` (default: -j8): Maximum number of packages which could be examined in parallel during the test run by the underlining build tool. If not set it's filled by `ROS_PARALLEL_JOBS`. As of Jan 2016, this is only enabled with `catkin_tools` (with `make` as an underlining builder).
Expand Down Expand Up @@ -193,6 +196,58 @@ Maintainers of client repos are responsible for applying the changes that happen

2. Don't forget to commit the changes the command above makes.

(Optional) Run ROS Prerelease Test
-------------------------------------------------------------------------------------

Running `docker-based ROS Prerelease Test <http://wiki.ros.org/bloom/Tutorials/PrereleaseTest/>`_ is strongly recommended when you make a release. There are, however, some inconvenience (requires host computer setup, runs on your local host, etc. Detail discussed in `a ticket <https://github.com/ros-industrial/industrial_ci/pull/35#issue-150581346>`_). `industrial_ci` provides a way to run it on your `Travis CI` test.

To do so, add a single line to your Travis config (eg. `.travis.yml`):

::

ROS_DISTRO=indigo PRERELEASE=true

Or with more configuration:

::

ROS_DISTRO=indigo PRERELEASE=true PRERELEASE_REPONAME=industrial_core PRERELEASE_DOWNSTREAM_DEPTH=0

NOTE: A job that runs Prerelease Test does not run the checks that are defined in `travis.sh <https://github.com/ros-industrial/industrial_ci/blob/master/travis.sh>`_. To run both, use `matrix` in Travis config.

See the usage sample in `.travis in indusrial_ci repository <https://github.com/ros-industrial/industrial_ci/blob/master/.travis.yml>`_.

The following is some tips to be shared for running Prerelease Test on Travis CI using `industrial_ci`.

(Workaround) Prerelease Test job on Travis always passes
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

As of April 2016, because the Prerelease Test job on Travis CI always passes regardless the test result (due to the limitaiton discussed in `a pull request <https://github.com/ros-industrial/industrial_ci/pull/35#issuecomment-214678922>`_), you should not use this Prerelease Test-based job as a criteria for Travis CI check.

Recommended way is to put the line in `allow_failures` matrix. E.g.:

::

:
matrix:
allow_failures:
- env: ROS_DISTRO=indigo PRERELEASE=true
:

(Workaround) Don't want to always run Prerelease Test
+++++++++++++++++++++++++++++++++++++++++++++++++++++

Running Prerelease Test may usually take longer than the tests commonly defined, which can result in longer time for the Travis check to finish. This is usually okay, as developers who are concerned with PRs might not wait for the Travis result that eagerly (besides that, Travis CI limits the maximum run time as 50 minutes so there can't be very long run). If you're concerned, however, then you may want to separately run the Prerelease Test. An example way to do this is to create a branch specifically for Prerelease Test where `.travis.yml` only defines a check entry with `PRERELEASE` turned on. E.g.:

::

:
env:
matrix:
- ROS_DISTRO=indigo PRERELEASE=true
:

Then open a pull request using this branch against the branch that the change is subject to be merged. You do not want to actually merge this branch no matter what the Travis result is. This branch is solely for Prerelease Test purpose.

For maintainers of industrial_ci repository
================================================
Expand Down
79 changes: 79 additions & 0 deletions ros_pre-release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#!/bin/bash

# Software License Agreement (BSD License)
#
# Copyright (c) 2016, Isaac I. Y. Saito
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of the Isaac I. Y. Saito, nor the names
# of its contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
## Author: Isaac I. Y. Saito, Mathias L«ädtke

set -e
set -x

source ${CI_SOURCE_PATH}/$CI_PARENT_DIR/util.sh

if [ ! "$PRERELEASE_DOWNSTREAM_DEPTH" ]; then export PRERELEASE_DOWNSTREAM_DEPTH="0"; fi
if [ ! "$PRERELEASE_REPONAME" ]; then PRERELEASE_REPONAME=$(echo $TRAVIS_REPO_SLUG | cut -d'/' -f 2); fi
echo "PRERELEASE_REPONAME = ${PRERELEASE_REPONAME}"

travis_time_start install_for_display_testresult
# Set apt repo
sudo -E sh -c 'echo "deb $ROS_REPOSITORY_PATH `lsb_release -cs` main" > /etc/apt/sources.list.d/ros-latest.list'
# Common ROS install preparation
wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
sudo apt-get -qq install -y ros-$ROS_DISTRO-catkin && source /opt/ros/$ROS_DISTRO/setup.bash || (echo 'ros-latest.list content: \n'; cat /etc/apt/sources.list.d/ros-latest.list; error;)
travis_time_end # install_for_display_testresult

travis_time_start setup_docker

sudo usermod -aG docker ubuntu
# ROS Buildfarm for prerelease http://wiki.ros.org/regression_tests#How_do_I_setup_my_system_to_run_a_prerelease.3F
sudo -E sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo -E apt-key adv --keyserver hkp://pool.sks-keyservers.net --recv-key 0xB01FA116
# Buildfarm workaround for Python3 http://wiki.ros.org/regression_tests#How_do_I_setup_my_system_to_run_a_prerelease.3F
sudo -E apt-get update && sudo -E apt-get install python3 python3-pip python-ros-buildfarm
sudo python3 -m pip install -U EmPy

travis_time_end # setup_docker

travis_time_start setup_prerelease_scripts

mkdir -p /tmp/prerelease_job; cd /tmp/prerelease_job; generate_prerelease_script.py https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml $ROS_DISTRO default ubuntu trusty amd64 ${PRERELEASE_REPONAME} --level $PRERELEASE_DOWNSTREAM_DEPTH --output-dir ./

travis_time_end # setup_prerelease_scripts

travis_time_start run_prerelease
./prerelease.sh;
travis_time_end # run_prerelease

travis_time_start show_testresult
catkin_test_results --verbose && (echo 'ROS Prerelease Test went successful.'; exit 0) || error
travis_time_end # show_testresult


cd $TRAVIS_BUILD_DIR # cd back to the repository's home directory with travis
pwd
55 changes: 24 additions & 31 deletions travis.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,44 +29,21 @@
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
## Greatly inspired by JSK travis with additional comments for maintainability: https://github.com/jsk-ros-pkg/jsk_travis
## Greatly inspired by JSK travis https://github.com/jsk-ros-pkg/jsk_travis
## Author: Isaac I. Y. Saito

## This is a "common" script that can be run on travis CI at a downstream github repository.
## See ./README.rst for the detailed usage.

set -e
set -x

function travis_time_start {
set +x
TRAVIS_START_TIME=$(date +%s%N)
TRAVIS_TIME_ID=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)
TRAVIS_FOLD_NAME=$1
echo -e "\e[0Ktravis_fold:start:$TRAVIS_FOLD_NAME"
echo -e "\e[0Ktravis_time:start:$TRAVIS_TIME_ID\e[34m>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\e[0m"
set -x
}

function travis_time_end {
set +x
_COLOR=${1:-32}
TRAVIS_END_TIME=$(date +%s%N)
TIME_ELAPSED_SECONDS=$(( ($TRAVIS_END_TIME - $TRAVIS_START_TIME)/1000000000 ))
echo -e "travis_time:end:$TRAVIS_TIME_ID:start=$TRAVIS_START_TIME,finish=$TRAVIS_END_TIME,duration=$(($TRAVIS_END_TIME - $TRAVIS_START_TIME))\e[0K"
echo -e "travis_fold:end:$TRAVIS_FOLD_NAME\e[${_COLOR}m<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\e[0m"
echo -e "\e[0K\e[${_COLOR}mFunction $TRAVIS_FOLD_NAME took $(( $TIME_ELAPSED_SECONDS / 60 )) min $(( $TIME_ELAPSED_SECONDS % 60 )) sec\e[0m"
set -x
}

function error {
travis_time_end 31
trap - ERR
exit 1
}

export CI_SOURCE_PATH=$(pwd)
export CI_PARENT_DIR=.ci_config # This is the folder name that is used in downstream repositories in order to point to this repo.
BUILDER=catkin
ROSWS=wstool
CI_PARENT_DIR=.ci_config # This is the folder name that is used in downstream repositories in order to point to this repo.

source ${CI_SOURCE_PATH}/$CI_PARENT_DIR/util.sh

trap error ERR

Expand All @@ -76,7 +53,6 @@ if [ "`git diff origin/master FETCH_HEAD $CI_PARENT_DIR`" != "" ] ; then DIFF=`g
travis_time_start setup_ros

# Define some config vars
export CI_SOURCE_PATH=$(pwd)
export DOWNSTREAM_REPO_NAME=${PWD##*/}
if [ ! "$CATKIN_PARALLEL_JOBS" ]; then export CATKIN_PARALLEL_JOBS="-p4"; fi
if [ ! "$CATKIN_PARALLEL_TEST_JOBS" ]; then export CATKIN_PARALLEL_TEST_JOBS="$CATKIN_PARALLEL_JOBS"; fi
Expand All @@ -102,6 +78,15 @@ if [ $HAVE_MONGO_DB == 0 ]; then
fi

travis_time_end # setup_ros

# Start prerelease, and once it finishs then finish this script too.
# This block needs to be here because catkin_test_results isn't available until up to this point.
travis_time_start prerelease_from_travis_sh
if [ "$PRERELEASE" == true ] && [ -e ${CI_SOURCE_PATH}/$CI_PARENT_DIR/ros_pre-release.sh ]; then
${CI_SOURCE_PATH}/$CI_PARENT_DIR/ros_pre-release.sh && exit 0 || exit 1;
fi
travis_time_end # prerelease_from_travis_sh

travis_time_start setup_rosdep

# Setup rosdep
Expand Down Expand Up @@ -184,8 +169,16 @@ fi

travis_time_end # rosdep_install

travis_time_start wstool_info
# Start prerelease, and once it finishs then finish this script too.
# This block needs to be here because catkin_test_results isn't available until up to this point.
travis_time_start prerelease_from_travis_sh
if [ "$PRERELEASE" == true ] && [ -e ${CI_SOURCE_PATH}/$CI_PARENT_DIR/ros_pre-release.sh ]; then
${CI_SOURCE_PATH}/$CI_PARENT_DIR/ros_pre-release.sh
catkin_test_results build && (echo 'ROS Prerelease Test went successful.'; exit 0) || error
fi
travis_time_end # prerelease_from_travis_sh

travis_time_start wstool_info
$ROSWS --version
$ROSWS info -t .
cd ../
Expand Down
63 changes: 63 additions & 0 deletions util.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/bin/bash

# Originally developed in JSK travis package https://github.com/jsk-ros-pkg/jsk_travis

# Software License Agreement (BSD License)
#
# Copyright (c) 2016, Isaac I. Y. Saito
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of the Isaac I. Y. Saito, nor the names
# of its contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

## util.sh
## This is a script where the functions commonly used within the industrial_ci repo are defined.

function travis_time_start {
set +x
TRAVIS_START_TIME=$(date +%s%N)
TRAVIS_TIME_ID=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)
TRAVIS_FOLD_NAME=$1
echo -e "\e[0Ktravis_fold:start:$TRAVIS_FOLD_NAME"
echo -e "\e[0Ktravis_time:start:$TRAVIS_TIME_ID\e[34m>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\e[0m"
set -x
}

function travis_time_end {
set +x
if [ -z $TRAVIS_START_TIME ]; then echo '[travis_time_end] var TRAVIS_START_TIME is not set. You need to call `travis_time_start` in advance. Rerutning.'; return; fi
_COLOR=${1:-32}
TRAVIS_END_TIME=$(date +%s%N)
TIME_ELAPSED_SECONDS=$(( ($TRAVIS_END_TIME - $TRAVIS_START_TIME)/1000000000 ))
echo -e "travis_time:end:$TRAVIS_TIME_ID:start=$TRAVIS_START_TIME,finish=$TRAVIS_END_TIME,duration=$(($TRAVIS_END_TIME - $TRAVIS_START_TIME))\e[0K"
echo -e "travis_fold:end:$TRAVIS_FOLD_NAME\e[${_COLOR}m<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\e[0m"
echo -e "\e[0K\e[${_COLOR}mFunction $TRAVIS_FOLD_NAME took $(( $TIME_ELAPSED_SECONDS / 60 )) min $(( $TIME_ELAPSED_SECONDS % 60 )) sec\e[0m"
set -x
}

function error {
travis_time_end 31
trap - ERR
exit 1
}

0 comments on commit e68bf5f

Please sign in to comment.