Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fly deploy nil pointer dereference: machine not found #4031

Open
technillogue opened this issue Oct 24, 2024 · 0 comments
Open

fly deploy nil pointer dereference: machine not found #4031

technillogue opened this issue Oct 24, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@technillogue
Copy link
Contributor

Describe the bug
flyctl deploy segfaults, maybe related to removing one machine and adding a volume

  • Operating system: ubuntu
  • fly version

** Paste your fly.toml

app = 'llama-3-8b-trt'
primary_region = 'ord'
[build]
  image = 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e'
  entrypoint = "bash"
[http_service]
  internal_port = 5000
  force_https = true
  auto_stop_machines = false
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']
[[vm]]
  size = 'l40s'
[mounts]
  source="models"
  destination="/weights"

** Command output: **

fly deploy --local-only
==> Verifying app config
Validating /home/sylv/r8/tmp/fly-mistral/fly.toml
✓ Configuration is valid
--> Verified app config
==> Building image
Searching for image 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e' locally...
Searching for image 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e' remotely...
image found: img_8rlxp2nw7o97p3jq

Watch your deployment at https://fly.io/apps/llama-3-8b-trt/monitoring

-------
Updating existing machines in 'llama-3-8b-trt' with rolling strategy
Failed to update machines: failed to update machine 148e54e9a34448: failed to launch VM: insufficient resources to create new machine with existing volume 'vol_4yj1lm1k9n66ow14' Retrying...
WARN error refreshing lease for machine 148e54e9a34448: failed to get lease on VM 148e54e9a34448: machine not found
-------
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x20e9a9e]
 ✖ failed to launch VM: insufficient resources to create new machine with existing volume 'vol_4yj1lm1k9n66ow14'
goroutine 87 [running]:
github.com/superfly/flyctl/internal/command/deploy.(*machineDeployment).updateProcessGroup.func1()
        /home/runner/work/flyctl/flyctl/internal/command/deploy/plan.go:339 +0x1e
golang.org/x/sync/errgroup.(*Group).Go.func1()
        /home/runner/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 86
        /home/runner/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96
@technillogue technillogue added the bug Something isn't working label Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant