Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to speedup dockerfile build time #2264

Merged
merged 31 commits into from
Aug 4, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
a59a597
Try to speedup dockerfile build time
ValarDragon Jul 31, 2022
f4e5b98
retry
ValarDragon Jul 31, 2022
c7916f7
Remove from tag
ValarDragon Jul 31, 2022
8860015
Reorder cosmwasm steps to be before copying whole directory
ValarDragon Jul 31, 2022
74e1702
Try cache
ValarDragon Jul 31, 2022
918fa37
add -v flag to go build
ValarDragon Jul 31, 2022
c0cfb5c
verbose flag try 2
ValarDragon Jul 31, 2022
67c4b61
Try cache again
ValarDragon Jul 31, 2022
8a8c028
try changing Github actions caching
ValarDragon Jul 31, 2022
cf6e38e
Try cache fixing
ValarDragon Jul 31, 2022
28e5ed3
Fix cache
ValarDragon Jul 31, 2022
e176580
See if failure was a fluke
ValarDragon Jul 31, 2022
5424f8b
Move-cache to right spot
ValarDragon Jul 31, 2022
71a31b1
Try using absolute path
ValarDragon Jul 31, 2022
33417f9
Try the buildkit cache workflow
ValarDragon Jul 31, 2022
d6c4dda
tryfix buildkit cache
ValarDragon Jul 31, 2022
6c7645f
More buildkit tries
ValarDragon Aug 1, 2022
0ec6e0c
Check CI re-run
ValarDragon Aug 1, 2022
df16a62
See if this makes cache get found again
ValarDragon Aug 1, 2022
d1d2f0d
Retry
ValarDragon Aug 1, 2022
debfa7c
retry
ValarDragon Aug 1, 2022
ecd9dd7
retry
ValarDragon Aug 1, 2022
0b10041
revert back to main .github folder
ValarDragon Aug 1, 2022
8f8df84
Rename tests CI job
ValarDragon Aug 1, 2022
542e379
Undo -v flag in install
ValarDragon Aug 1, 2022
caeeb4e
Add go mod download earlier in build process
ValarDragon Aug 1, 2022
35aa0b7
Try staging download better
ValarDragon Aug 1, 2022
52e5dc6
Accidental local change
ValarDragon Aug 1, 2022
88c477e
correct filename
ValarDragon Aug 1, 2022
6029639
Try further layer reduction, use cache in go mod download
ValarDragon Aug 1, 2022
3481524
See speed on cache re-run
ValarDragon Aug 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Tests & Code Coverage
name: Tests

on:
pull_request:
Expand Down Expand Up @@ -29,7 +29,7 @@ jobs:
name: Skipping test
run: echo Should I skip tests? ${{ steps.skip_check.outputs.should_skip }}

go_test:
go:
needs: should_run_go_test
if: ${{ needs.should_run_test.outputs.should_skip != 'true' }}
runs-on: ubuntu-latest
Expand Down Expand Up @@ -66,7 +66,7 @@ jobs:
name: Run all tests
run: make test-unit

e2e_test:
e2e:
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
Expand Down
37 changes: 24 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,35 @@ ARG BASE_IMG_TAG=nonroot

FROM golang:1.18.2-alpine3.15 as build

RUN set -eux; apk add --no-cache ca-certificates build-base;
RUN apk add git
# Needed by github.com/zondax/hid
RUN apk add linux-headers
# linux-headers needed by github.com/zondax/hid
RUN set -eux; apk add --no-cache ca-certificates build-base; apk add git linux-headers

# CosmWasm: see https://github.com/CosmWasm/wasmvm/releases
# 1) Add the releases into /lib/
ADD https://github.com/CosmWasm/wasmvm/releases/download/v1.0.0/libwasmvm_muslc.aarch64.a \
https://github.com/CosmWasm/wasmvm/releases/download/v1.0.0/libwasmvm_muslc.x86_64.a \
/lib/
# 2) Verify their hashes
RUN sha256sum /lib/libwasmvm_muslc.aarch64.a | grep 7d2239e9f25e96d0d4daba982ce92367aacf0cbd95d2facb8442268f2b1cc1fc; \
sha256sum /lib/libwasmvm_muslc.x86_64.a | grep f6282df732a13dec836cda1f399dd874b1e3163504dbd9607c6af915b2740479
# Copy the right library according to architecture. The final location will be found by the linker flag `-lwasmvm_muslc`
RUN cp /lib/libwasmvm_muslc.$(uname -m).a /lib/libwasmvm_muslc.a

# Get go dependencies first
COPY go.mod go.sum /osmosis/
WORKDIR /osmosis
COPY . /osmosis
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/root/go/pkg/mod \
go mod download

# CosmWasm: see https://github.com/CosmWasm/wasmvm/releases
ADD https://github.com/CosmWasm/wasmvm/releases/download/v1.0.0/libwasmvm_muslc.aarch64.a /lib/libwasmvm_muslc.aarch64.a
ADD https://github.com/CosmWasm/wasmvm/releases/download/v1.0.0/libwasmvm_muslc.x86_64.a /lib/libwasmvm_muslc.x86_64.a
RUN sha256sum /lib/libwasmvm_muslc.aarch64.a | grep 7d2239e9f25e96d0d4daba982ce92367aacf0cbd95d2facb8442268f2b1cc1fc
RUN sha256sum /lib/libwasmvm_muslc.x86_64.a | grep f6282df732a13dec836cda1f399dd874b1e3163504dbd9607c6af915b2740479
# Copy all files into our docker repo
COPY . /osmosis

# CosmWasm: copy the right library according to architecture. The final location will be found by the linker flag `-lwasmvm_muslc`
RUN cp /lib/libwasmvm_muslc.$(uname -m).a /lib/libwasmvm_muslc.a
# build

RUN BUILD_TAGS=muslc LINK_STATICALLY=true make build
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/root/go/pkg/mod \
BUILD_TAGS=muslc LINK_STATICALLY=true make build

# --------------------------------------------------------
# Runner
Expand Down
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -248,13 +248,13 @@ build-e2e-script:
go build -mod=readonly $(BUILD_FLAGS) -o $(BUILDDIR)/ ./tests/e2e/initialization/$(E2E_SCRIPT_NAME)

docker-build-debug:
@docker build -t osmosis:debug --build-arg BASE_IMG_TAG=debug -f Dockerfile .
@DOCKER_BUILDKIT=1 docker build -t osmosis:debug --build-arg BASE_IMG_TAG=debug -f Dockerfile .

docker-build-e2e-init-chain:
@docker build -t osmosis-e2e-init-chain:debug --build-arg E2E_SCRIPT_NAME=chain -f tests/e2e/initialization/init.Dockerfile .
@DOCKER_BUILDKIT=1 docker build -t osmosis-e2e-init-chain:debug --build-arg E2E_SCRIPT_NAME=chain -f tests/e2e/initialization/init.Dockerfile .

docker-build-e2e-init-node:
@docker build -t osmosis-e2e-init-node:debug --build-arg E2E_SCRIPT_NAME=node -f tests/e2e/initialization/init.Dockerfile .
@DOCKER_BUILDKIT=1 docker build -t osmosis-e2e-init-node:debug --build-arg E2E_SCRIPT_NAME=node -f tests/e2e/initialization/init.Dockerfile .

.PHONY: test-mutation

Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Conceptually, we can split the e2e setup into 2 parts:

1. Chain Initialization

The chain can either be initailized off of the current branch, or off the prior mainnet release and then upgraded to the current branch.
The chain can either be initialized off of the current branch, or off the prior mainnet release and then upgraded to the current branch.

If current, we run chain initialization off of the current Git branch
by calling `chain.Init(...)` method in the `configurer/current.go`.
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/configurer/chain/node.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ func (n *NodeConfig) Run() error {
n.t.Logf("started node container: %s", n.Name)
return true
},
5*time.Minute,
2*time.Minute,
time.Second,
"Osmosis node failed to produce blocks",
)
Expand Down