Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: 1.20.0 release(fix cuda dependency) #69

Merged
merged 1 commit into from
Nov 4, 2024
Merged

ci: 1.20.0 release(fix cuda dependency) #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,11 +158,11 @@ sudo cmake --install build --prefix /usr/local/onnxruntime-server
# Docker

- Docker hub: [kibaes/onnxruntime-server](https://hub.docker.com/r/kibaes/onnxruntime-server)
- [`1.20.0-linux-cuda11`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cuda11.dockerfile) amd64(CUDA 11.x, cuDNN 8.x)
- [`1.20.0-linux-cuda12`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cuda12.dockerfile) amd64(CUDA 12.x, cuDNN 9.x)
- [`1.20.0-linux-cpu`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cpu.dockerfile) amd64, arm64

```shell
DOCKER_IMAGE=kibae/onnxruntime-server:1.20.0-linux-cuda11 # or kibae/onnxruntime-server:1.20.0-linux-cpu
DOCKER_IMAGE=kibae/onnxruntime-server:1.20.0-linux-cuda12 # or kibae/onnxruntime-server:1.20.0-linux-cpu

docker pull ${DOCKER_IMAGE}

Expand Down
18 changes: 9 additions & 9 deletions deploy/build-docker/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,19 @@ if [ "$1" != "--target=cpu" ]; then
# | | | | | | | | | | / /_\ \ |______> < | '_ \ |__ _|
# | `----.| `--' | | '--' | / _____ \ / . \ | (_) | | |
# \______| \______/ |_______/ /__/ \__\ /__/ \__\ \___/ |_|
POSTFIX=linux-cuda11
IMAGE_NAME=${IMAGE_PREFIX}:${VERSION}-${POSTFIX}

docker buildx build --platform linux/amd64 -t ${IMAGE_NAME} -f deploy/build-docker/${POSTFIX}.dockerfile --load . || exit 1
./deploy/build-docker/docker-image-test.sh ${IMAGE_NAME} 1 || exit 1
docker buildx build --platform linux/amd64 -t ${IMAGE_NAME} -f deploy/build-docker/${POSTFIX}.dockerfile --push . || exit 1


# POSTFIX=linux-cuda12
# POSTFIX=linux-cuda11
# IMAGE_NAME=${IMAGE_PREFIX}:${VERSION}-${POSTFIX}
#
# docker buildx build --platform linux/amd64 -t ${IMAGE_NAME} -f deploy/build-docker/${POSTFIX}.dockerfile --load . || exit 1
# ./deploy/build-docker/docker-image-test.sh ${IMAGE_NAME} 1 || exit 1
# docker buildx build --platform linux/amd64 -t ${IMAGE_NAME} -f deploy/build-docker/${POSTFIX}.dockerfile --push . || exit 1


POSTFIX=linux-cuda12
IMAGE_NAME=${IMAGE_PREFIX}:${VERSION}-${POSTFIX}

docker buildx build --platform linux/amd64 -t ${IMAGE_NAME} -f deploy/build-docker/${POSTFIX}.dockerfile --load . || exit 1
./deploy/build-docker/docker-image-test.sh ${IMAGE_NAME} 1 || exit 1
docker buildx build --platform linux/amd64 -t ${IMAGE_NAME} -f deploy/build-docker/${POSTFIX}.dockerfile --push . || exit 1
fi

4 changes: 2 additions & 2 deletions deploy/build-docker/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ services:
onnxruntime_server_simple:
# After the docker container is up, you can use the REST API (http://localhost:8080).
# API documentation will be available at http://localhost:8080/api-docs.
image: kibaes/onnxruntime-server:1.20.0-linux-cuda11
image: kibaes/onnxruntime-server:1.20.0-linux-cuda12
ports:
- "8080:80" # for http backend
volumes:
Expand All @@ -29,7 +29,7 @@ services:
onnxruntime_server_advanced:
# After the docker container is up, you can use the REST API (http://localhost, https://localhost).
# API documentation will be available at http://localhost/api-docs.
image: kibaes/onnxruntime-server:1.20.0-linux-cuda11
image: kibaes/onnxruntime-server:1.20.0-linux-cuda12
ports:
- "80:80" # for http backend
- "443:443" # for https backend
Expand Down
8 changes: 4 additions & 4 deletions deploy/build-docker/linux-cuda12.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM nvidia/cuda:12.3.2-cudnn9-devel-ubuntu22.04 AS builder
FROM nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04 AS builder

RUN apt update && apt install -y curl wget git build-essential cmake pkg-config libboost-all-dev libssl-dev
RUN mkdir -p /app/source
Expand All @@ -15,18 +15,18 @@ RUN case ${TARGETPLATFORM} in \
"linux/amd64") ./download-onnxruntime.sh linux x64-gpu ;; \
esac

RUN cmake -DCUDA_SDK_ROOT_DIR=/usr/local/cuda -DBoost_USE_STATIC_LIBS=ON -DOPENSSL_USE_STATIC_LIBS=ON -B build -S . -DCMAKE_BUILD_TYPE=Release
RUN cmake -DCUDA_SDK_ROOT_DIR=/usr/local/cuda-12 -DBoost_USE_STATIC_LIBS=ON -DOPENSSL_USE_STATIC_LIBS=ON -B build -S . -DCMAKE_BUILD_TYPE=Release
RUN cmake --build build --parallel 4 --target onnxruntime_server_standalone
RUN cmake --install build --prefix /app/onnxruntime-server

# target
FROM nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04 AS target
FROM nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04 AS target
COPY --from=builder /app/onnxruntime-server /app
COPY --from=builder /usr/local/onnxruntime /usr/local/onnxruntime

WORKDIR /app
RUN mkdir -p models logs certs

ENV ONNX_SERVER_CONFIG_PRIORITY=env
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/onnxruntime/lib
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-12/lib64:/usr/local/onnxruntime/lib
ENTRYPOINT ["/app/bin/onnxruntime_server", "--model-dir", "models", "--log-file", "logs/app.log", "--access-log-file", "logs/access.log", "--tcp-port", "6432", "--http-port", "80"]
8 changes: 4 additions & 4 deletions docs/docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

# Supported tags and respective Dockerfile links

- [`1.20.0-linux-cuda11`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cuda11.dockerfile) amd64(CUDA 11.x, cuDNN 8.x)
- [`1.20.0-linux-cuda12`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cuda12.dockerfile) amd64(CUDA 12.x, cuDNN 9.x)
- [`1.20.0-linux-cpu`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cpu.dockerfile) amd64, arm64

# How to use this image
Expand All @@ -28,7 +28,7 @@
- API documentation will be available at http://localhost/api-docs.

```shell
DOCKER_IMAGE=kibae/onnxruntime-server:1.20.0-linux-cuda11 # or kibae/onnxruntime-server:1.20.0-linux-cpu
DOCKER_IMAGE=kibae/onnxruntime-server:1.20.0-linux-cuda12 # or kibae/onnxruntime-server:1.20.0-linux-cpu

docker pull ${DOCKER_IMAGE}

Expand Down Expand Up @@ -69,7 +69,7 @@ services:
onnxruntime_server_simple:
# After the docker container is up, you can use the REST API (http://localhost:8080).
# API documentation will be available at http://localhost:8080/api-docs.
image: kibaes/onnxruntime-server:1.20.0-linux-cuda11
image: kibaes/onnxruntime-server:1.20.0-linux-cuda12
ports:
- "8080:80" # for http backend
volumes:
Expand Down Expand Up @@ -101,7 +101,7 @@ services:
onnxruntime_server_advanced:
# After the docker container is up, you can use the REST API (http://localhost, https://localhost).
# API documentation will be available at http://localhost/api-docs.
image: kibaes/onnxruntime-server:1.20.0-linux-cuda11
image: kibaes/onnxruntime-server:1.20.0-linux-cuda12
ports:
- "80:80" # for http backend
- "443:443" # for https backend
Expand Down
Loading