Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update java version to 17 on ubuntu #1738

Merged
merged 4 commits into from
Jul 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion benchmarks/install_dependencies.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ sudo apt-get install -y \
python3-dev \
python-psutil \
python3-pip \
openjdk-11-jdk \
openjdk-17-jdk \
g++ \
curl \
vim \
Expand Down
14 changes: 8 additions & 6 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
#
# This file can build images for cpu and gpu env. By default it builds image for CPU.
# Use following option to build image for cuda/GPU: --build-arg BASE_IMAGE=nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
# Here is complete command for GPU/cuda -
# Here is complete command for GPU/cuda -
# $ DOCKER_BUILDKIT=1 docker build --file Dockerfile --build-arg BASE_IMAGE=nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 -t torchserve:latest .
#
# Following comments have been shamelessly copied from https://github.com/pytorch/pytorch/blob/master/Dockerfile
#
#
# NOTE: To build this you will need a docker version > 18.06 with
# experimental enabled and DOCKER_BUILDKIT=1
#
# If you do not use buildkit you are not going to have a good time
#
# For reference:
# For reference:
# https://docs.docker.com/develop/develop-images/build_enhancements/


Expand All @@ -34,15 +34,15 @@ RUN --mount=type=cache,id=apt-dev,target=/var/cache/apt \
python3.8-distutils \
python3.8-venv \
python3-venv \
openjdk-11-jre-headless \
openjdk-17-jdk \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please add comments (eg. link of the jdk headless ca-certificate issue) at here so that we can follow later?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

curl \
&& rm -rf /var/lib/apt/lists/* \
&& cd /tmp \
&& curl -O https://bootstrap.pypa.io/get-pip.py \
&& python3.8 get-pip.py


RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.8 1 \
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.8 1 \
&& update-alternatives --install /usr/local/bin/pip pip /usr/local/bin/pip3.8 1

RUN python3.8 -m venv /home/venv
Expand Down Expand Up @@ -84,7 +84,9 @@ RUN --mount=type=cache,target=/var/cache/apt \
python3.8 \
python3.8-distutils \
python3.8-dev \
openjdk-11-jre-headless \
# using openjdk-17-jdk due to circular dependency(ca-certificates) bug in openjdk-17-jre-headless debian package
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009905
openjdk-17-jdk \
build-essential \
&& rm -rf /var/lib/apt/lists/* \
&& cd /tmp
Expand Down
10 changes: 5 additions & 5 deletions docker/Dockerfile.dev
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ RUN --mount=type=cache,id=apt-dev,target=/var/cache/apt \
python3.8-distutils \
# python3-venv \
build-essential \
openjdk-11-jdk \
openjdk-17-jdk \
curl \
vim \
numactl \
Expand Down Expand Up @@ -84,10 +84,10 @@ CMD ["serve"]

# Build CodeBuild Image
FROM compile-image AS codebuild-image
ENV JAVA_VERSION=11 \
JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64" \
JDK_HOME="/usr/lib/jvm/java-11-openjdk-amd64" \
JRE_HOME="/usr/lib/jvm/java-11-openjdk-amd64" \
ENV JAVA_VERSION=17 \
JAVA_HOME="/usr/lib/jvm/java-17-openjdk-amd64" \
JDK_HOME="/usr/lib/jvm/java-17-openjdk-amd64" \
JRE_HOME="/usr/lib/jvm/java-17-openjdk-amd64" \
ANT_VERSION=1.10.3 \
MAVEN_HOME="/opt/maven" \
MAVEN_VERSION=3.5.4 \
Expand Down
2 changes: 1 addition & 1 deletion frontend/gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-6.4-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.3-bin.zip
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
1 change: 1 addition & 0 deletions frontend/server/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ jar {
attributes 'Main-Class': 'org.pytorch.serve.ModelServer'
}
includeEmptyDirs = false
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
from configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }

exclude "META-INF/maven/**"
Expand Down
59 changes: 35 additions & 24 deletions ts_scripts/install_dependencies.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import argparse
import os
import platform
import argparse
import sys
from pathlib import Path

from print_env_info import run_and_parse_first_match

REPO_ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..")
Expand All @@ -11,11 +11,10 @@
from ts_scripts.utils import check_python_version


class Common():

class Common:
def __init__(self):
self.torch_stable_url = "https://download.pytorch.org/whl/torch_stable.html"
self.sudo_cmd = 'sudo '
self.sudo_cmd = "sudo "

def install_java(self):
pass
Expand Down Expand Up @@ -70,25 +69,25 @@ def install_wget(self):


class Linux(Common):

def __init__(self):
super().__init__()
# Skip 'sudo ' when the user is root
self.sudo_cmd = '' if os.geteuid() == 0 else self.sudo_cmd
self.sudo_cmd = "" if os.geteuid() == 0 else self.sudo_cmd

if args.force:
os.system(f"{self.sudo_cmd}apt-get update")

def install_java(self):
if os.system("javac --version") != 0 or args.force:
os.system(f"{self.sudo_cmd}apt-get install -y openjdk-11-jdk")
os.system(f"{self.sudo_cmd}apt-get install -y openjdk-17-jdk")

def install_nodejs(self):
if os.system("node -v") != 0 or args.force:
os.system(f"{self.sudo_cmd}curl -sL https://deb.nodesource.com/setup_14.x | {self.sudo_cmd}bash -")
os.system(
f"{self.sudo_cmd}curl -sL https://deb.nodesource.com/setup_14.x | {self.sudo_cmd}bash -"
)
os.system(f"{self.sudo_cmd}apt-get install -y nodejs")


def install_wget(self):
if os.system("wget --version") != 0 or args.force:
os.system(f"{self.sudo_cmd}apt-get install -y wget")
Expand All @@ -98,20 +97,17 @@ def install_libgit2(self):
f"wget https://github.com/libgit2/libgit2/archive/refs/tags/v1.3.0.tar.gz -O libgit2-1.3.0.tar.gz"
)
os.system(f"tar xzf libgit2-1.3.0.tar.gz")
os.system(
f"cd libgit2-1.3.0 && cmake . && make && sudo make install && cd .."
)
os.system(f"cd libgit2-1.3.0 && cmake . && make && sudo make install && cd ..")
os.system(f"rm -rf libgit2-1.3.0 && rm libgit2-1.3.0.tar.gz")

def install_maven(self):
os.system(f"{self.sudo_cmd}apt-get install -y maven")


class Windows(Common):

def __init__(self):
super().__init__()
self.sudo_cmd = ''
self.sudo_cmd = ""

def install_java(self):
pass
Expand All @@ -124,7 +120,6 @@ def install_wget(self):


class Darwin(Common):

def __init__(self):
super().__init__()

Expand Down Expand Up @@ -169,23 +164,39 @@ def install_dependencies(cuda_version=None):
# Sequence of installation to be maintained
system.install_java()
requirements_file_path = "requirements/" + (
"production.txt" if args.environment == "prod" else "developer.txt")
"production.txt" if args.environment == "prod" else "developer.txt"
)
system.install_python_packages(cuda_version, requirements_file_path)


def get_brew_version():
"""Returns `brew --version` output. """
"""Returns `brew --version` output."""

return run_and_parse_first_match("brew --version", r'Homebrew (.*)')
return run_and_parse_first_match("brew --version", r"Homebrew (.*)")


if __name__ == "__main__":
check_python_version()
parser = argparse.ArgumentParser(description="Install various build and test dependencies of TorchServe")
parser.add_argument('--cuda', default=None, choices=['cu92', 'cu101', 'cu102', 'cu111', 'cu113'], help="CUDA version for torch")
parser.add_argument('--environment', default='prod', choices=['prod', 'dev'],
help="environment(production or developer) on which dependencies will be installed")
parser.add_argument("--force", action='store_true', help="force reinstall dependencies wget, node, java and apt-update")
parser = argparse.ArgumentParser(
description="Install various build and test dependencies of TorchServe"
)
parser.add_argument(
"--cuda",
default=None,
choices=["cu92", "cu101", "cu102", "cu111", "cu113"],
help="CUDA version for torch",
)
parser.add_argument(
"--environment",
default="prod",
choices=["prod", "dev"],
help="environment(production or developer) on which dependencies will be installed",
)
parser.add_argument(
"--force",
action="store_true",
help="force reinstall dependencies wget, node, java and apt-update",
)
args = parser.parse_args()

install_dependencies(cuda_version=args.cuda)
2 changes: 1 addition & 1 deletion ts_scripts/install_utils
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ install_java_deps()
brew tap AdoptOpenJDK/openjdk
brew cask install adoptopenjdk11
else
sudo apt-get install -y openjdk-11-jdk
sudo apt-get install -y openjdk-17-jdk
fi
fi
set -e
Expand Down