Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails with "bad record MAC" error #1942

Closed
tonyhopkins994 opened this issue Sep 11, 2024 · 3 comments
Closed

Build fails with "bad record MAC" error #1942

tonyhopkins994 opened this issue Sep 11, 2024 · 3 comments

Comments

@tonyhopkins994
Copy link

tonyhopkins994 commented Sep 11, 2024

Follow up from #1925 where image builds were getting stuck.

After the fix from @8W9aG yesterday, the pyenv Illegal instruction no longer occurs but there's a new error that's interrupting builds.

------
 > [1/5] FROM r8.im/cog-base:cuda11.8-python3.9@sha256:b467e2cc3f460447163c62166482937c974a3e31d5452197860941e69fc8b04c:
------
Dockerfile:2
--------------------
   1 |     #syntax=docker/dockerfile:1.4
   2 | >>> FROM r8.im/cog-base:cuda11.8-python3.9
   3 |     COPY .cog/tmp/build20240911181105.0228062875176495/requirements.txt /tmp/requirements.txt
   4 |     ENV CFLAGS="-O3 -funroll-loops -fno-strict-aliasing -flto -S"
--------------------
ERROR: failed to solve: failed to copy: local error: tls: bad record MAC
ⅹ Failed to build Docker image: exit status 1

Edit: Using the --use-cog-base-image=false tag does not prevent the error

Here's the .yaml being used for the build for reference:

# Configuration for Cog ⚙️
# Reference: https://github.com/replicate/cog/blob/main/docs/yaml.md

build:
  gpu: true
  cuda: "11.8"
  python_version: "3.9"

  python_packages:
    - torch
    - torchvision
    - torchaudio
    - torchsde
    - transformers>=4.28.1
    - safetensors>=0.3.0
    - accelerate
    - Pillow
    - scipy
    - tqdm
    - diffusers>=0.30.0


predict: "predict.py:Predictor"

@8W9aG
Copy link
Contributor

8W9aG commented Sep 12, 2024

This is a docker issue and during my reading I think it may be a local networking issue:
microsoft/WSL#4658

Could you give this a go on a different network / build machine and tell me if you see the same thing?

@tonyhopkins994
Copy link
Author

This is a docker issue and during my reading I think it may be a local networking issue: microsoft/WSL#4658

Could you give this a go on a different network / build machine and tell me if you see the same thing?

Changing the active network when building did not make any difference to the output. I'm unable to check this on another system at the moment.

Tried updating my docker from v4.26.1 to v4.33.1 to no avail. Same error message.

@tonyhopkins994
Copy link
Author

Update: Was able to get past this error with a change of the network adapter. Swapped out a TP-Link Archer T2U adapter for a TP-Link Wireless USB Adapter.
My best guess is that the network on the T2U was too unstable for big downloads (although that's never been a problem out of WSL2).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants