Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Python Linux wheel release scripts #700

Merged
merged 2 commits into from
Jan 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 7 additions & 33 deletions .github/tools/release_linux.sh
Original file line number Diff line number Diff line change
@@ -1,41 +1,15 @@
#!/usr/bin/env bash
set -e -x

ln -sf /usr/bin/python3.5 /usr/bin/python3 # Py36 has issues with add-apt
add-apt-repository -y ppa:deadsnakes/ppa

apt-get -y -qq update

apt-get -y -qq install python${PYTHON_VERSION}-dev --no-install-recommends
apt-get -y -qq install python${PYTHON_VERSION}-distutils --no-install-recommends || true
ln -sf /usr/bin/python${PYTHON_VERSION} /usr/bin/python
ln -sf /usr/bin/python${PYTHON_VERSION} /usr/bin/python3

ls /usr/include/x86_64-linux-gnu/
ls /dt7/usr/include/x86_64-linux-gnu/

# This really shouldn't be required to get 3.7 builds working
ln -sf /usr/include/x86_64-linux-gnu/python3.7m /dt7/usr/include/x86_64-linux-gnu/python3.7m || true
ln -sf /usr/include/x86_64-linux-gnu/python3.7m /dt8/usr/include/x86_64-linux-gnu/python3.7m || true
ln -sf /usr/include/x86_64-linux-gnu/python3.8 /dt7/usr/include/x86_64-linux-gnu/python3.8 || true
ln -sf /usr/include/x86_64-linux-gnu/python3.8 /dt8/usr/include/x86_64-linux-gnu/python3.8 || true
ln -sf /usr/include/x86_64-linux-gnu/python3.9 /dt7/usr/include/x86_64-linux-gnu/python3.9 || true
ln -sf /usr/include/x86_64-linux-gnu/python3.9 /dt8/usr/include/x86_64-linux-gnu/python3.9 || true

curl https://bootstrap.pypa.io/get-pip.py | python
python --version
python -m pip --version

curl -L https://github.com/bazelbuild/bazelisk/releases/download/v1.8.0/bazelisk-linux-amd64 > /usr/bin/bazelisk
chmod +x /usr/bin/bazelisk

python -m pip install numpy six --no-cache-dir

export MANYLINUX2010=1
./configure.py
python configure.py

# Build
bazelisk build :build_pip_pkg --copt=-fvisibility=hidden --copt=-mavx --distinct_host_configuration=false --discard_analysis_cache --notrack_incremental_state --nokeep_state_after_build --nouse_action_cache --local_ram_resources=4096
bazel build :build_pip_pkg \
--copt=-fvisibility=hidden \
--copt=-mavx \
--distinct_host_configuration=false \
--verbose_failures \
--crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda11.2:toolchain

# Package Whl
bazel-bin/build_pip_pkg artifacts
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ jobs:
runs-on: macos-latest
strategy:
matrix:
python-version: [3.6, 3.7, 3.8, 3.9]
python-version: [3.7, 3.8, 3.9]
fail-fast: false
steps:
- uses: actions/checkout@v2
Expand Down Expand Up @@ -199,7 +199,7 @@ jobs:
runs-on: ubuntu-18.04
strategy:
matrix:
python-version: [3.6, 3.7, 3.8, 3.9]
python-version: [3.7, 3.8, 3.9]
CNugteren marked this conversation as resolved.
Show resolved Hide resolved
fail-fast: false
steps:
- uses: actions/checkout@v2
Expand All @@ -211,7 +211,7 @@ jobs:
docker run -e PYTHON_VERSION=${{ matrix.python-version }} \
-e LCE_RELEASE_VERSION=${{ github.event.inputs.version }} \
-v ${PWD}:/compute-engine -w /compute-engine \
tensorflow/tensorflow:2.2.0-custom-op-ubuntu16 \
tensorflow/build:latest-python${{ matrix.python-version }} \
.github/tools/release_linux.sh

sudo apt-get -y -qq install patchelf --no-install-recommends
Expand All @@ -232,7 +232,7 @@ jobs:
runs-on: windows-latest
strategy:
matrix:
python-version: [3.6, 3.7, 3.8, 3.9]
python-version: [3.7, 3.8, 3.9]
fail-fast: false
steps:
- name: Configure Pagefile
Expand Down
23 changes: 0 additions & 23 deletions configure.py
Original file line number Diff line number Diff line change
Expand Up @@ -411,26 +411,6 @@ def set_cc_opt_flags(environ_cp):
write_to_bazelrc("build:opt --host_copt=%s" % opt)


def maybe_set_manylinux_toolchain(environ_cp):
write_to_bazelrc(
"build:manylinux2010 --crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda11.2:toolchain"
)
if get_var(
environ_cp,
var_name="MANYLINUX2010",
query_item="manylinux2010-compatible pip package",
enabled_by_default=False,
question=(
"Are you trying to build a manylinux2010-compatible pip package "
"in the tensorflow:custom-op-ubuntu16 Docker container?"
),
yes_reply="Building manylinux2010-compatible pip package.",
no_reply="Not building manylinux2010-compatible pip package.",
):
write_to_bazelrc("build --config=manylinux2010")
write_to_bazelrc("test --config=manylinux2010")


def set_windows_build_flags(environ_cp):
"""Set Windows specific build options."""

Expand Down Expand Up @@ -638,9 +618,6 @@ def main():
if is_windows():
set_windows_build_flags(environ_cp)

if is_linux():
maybe_set_manylinux_toolchain(environ_cp)

if get_var(
environ_cp,
"LCE_SET_ANDROID_WORKSPACE",
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def get_version_number(default):
setup(
name="larq-compute-engine",
version=get_version_number(default="0.6.2"),
python_requires=">=3.6",
python_requires=">=3.7",
description="Highly optimized inference engine for binarized neural networks.",
long_description=readme(),
long_description_content_type="text/markdown",
Expand Down