Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added 'import' tests and github actions for docker images #1090

Merged
merged 8 commits into from
Aug 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/build.space.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Free disk space on Linux
#sudo swapoff /swapfile
#sudo rm -rf /swapfile
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc
sudo apt-get remove php* ruby-* subversion mongodb-org -yq >/dev/null 2>&1
sudo apt-get autoremove -y >/dev/null 2>&1
sudo apt-get autoclean -y >/dev/null 2>&1
sudo rm -rf /usr/local/lib/android >/dev/null 2>&1
docker rmi $(docker image ls -aq) >/dev/null 2>&1
#sudo swapoff /swapfile
#sudo rm -rf /swapfile
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc
sudo apt-get remove php* ruby-* subversion mongodb-org -yq >/dev/null 2>&1
sudo apt-get autoremove -y >/dev/null 2>&1
sudo apt-get autoclean -y >/dev/null 2>&1
sudo rm -rf /usr/local/lib/android >/dev/null 2>&1
docker rmi $(docker image ls -aq) >/dev/null 2>&1
15 changes: 15 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,16 @@ jobs:
set -x -e
python --version
bash -x -e .github/workflows/build.wheel.sh python
- name: Test docker images on macOS
run: |
set -x -e
mkdir -p ~/.docker/machine/cache
curl -Lo ~/.docker/machine/cache/boot2docker.iso https://github.com/boot2docker/boot2docker/releases/download/v19.03.12/boot2docker.iso
brew install docker docker-machine
docker-machine create --driver virtualbox default
docker-machine env default
eval "$(docker-machine env default)"
bash -x -e tools/docker/tests/dockerfile_cpu_test.sh

linux-bazel:
name: Bazel Linux
Expand Down Expand Up @@ -304,6 +314,11 @@ jobs:
docker run -i --rm -v $PWD:/v -w /v --net=host \
buildpack-deps:$(echo ${{ matrix.os }} | awk -F- '{print $2}') \
bash -x -e .github/workflows/build.wheel.sh python${{ matrix.python }}
- name: Test docker images on Linux
run: |
set -x -e
bash -x -e tools/docker/tests/dockerfile_cpu_test.sh
bash -x -e tools/docker/tests/dockerfile_devel_test.sh

windows-bazel:
name: Bazel Windows
Expand Down
27 changes: 27 additions & 0 deletions tools/docker/tests/bazel_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

echo "Configuring the environment for build ..."
./configure.sh

echo "Cleaning up existing bazel build files ..."
rm -rf bazel-*

echo "Building the tensorflow-io package ..."
bazel build -j 8 --copt=-msse4.2 --copt=-mavx --compilation_mode=opt --verbose_failures --test_output=errors --crosstool_top=//third_party/toolchains/gcc7_manylinux2010:toolchain //tensorflow_io/...

echo "Validating import ..."
python -c "import tensorflow_io as tfio; print(tfio.__version__)"
24 changes: 24 additions & 0 deletions tools/docker/tests/dockerfile_cpu_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

IMAGE_NAME="tfio-cpu"
export PYTHON_BIN_PATH=$(which python3.7)

echo "Build the docker image ..."
docker build -f tools/docker/cpu.Dockerfile -t ${IMAGE_NAME} .

echo "Starting the docker container from image: ${IMAGE_NAME} and validating import ..."
docker run -t --rm ${IMAGE_NAME} python -c "import tensorflow_io as tfio; print(tfio.__version__)"
24 changes: 24 additions & 0 deletions tools/docker/tests/dockerfile_devel_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

IMAGE_NAME="tfio-dev"
export PYTHON_BIN_PATH=$(which python3)

echo "Build the docker image ..."
docker build -f tools/docker/devel.Dockerfile -t ${IMAGE_NAME} .

echo "Starting the docker container from image: ${IMAGE_NAME} and building the package ..."
docker run -t --rm --net=host -v ${PWD}:/v -w /v tfio-dev bash tools/docker/tests/bazel_build.sh