Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when formatting s3 buckets in us-east1 #4649

Closed
mwqgithub opened this issue Apr 7, 2024 · 2 comments
Closed

Panic when formatting s3 buckets in us-east1 #4649

mwqgithub opened this issue Apr 7, 2024 · 2 comments
Assignees

Comments

@mwqgithub
Copy link

What happened:

[root@ip-172-31-45-198 ~]# juicefs format sqlite3://myjfs.db myjfs --storage s3 --bucket juicefs-us-east-1
2024/04/07 03:11:20.228937 juicefs[13600] : Meta address: sqlite3://myjfs.db [interface.go:497]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1ea69e9]

goroutine 1 [running]:
github.com/juicedata/juicefs/pkg/object.autoS3Region({0xc000112788, 0x11}, {0x0?, 0x1?}, {0x0?, 0xc000db2fa8?})
/go/src/github.com/juicedata/juicefs/pkg/object/s3.go:394 +0x8a9
github.com/juicedata/juicefs/pkg/object.newS3({0xc000844018, 0x11}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0})
/go/src/github.com/juicedata/juicefs/pkg/object/s3.go:468 +0x32b
github.com/juicedata/juicefs/pkg/object.CreateStorage({0xc00005212e, 0x2}, {0xc000844018, 0x11}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0})
/go/src/github.com/juicedata/juicefs/pkg/object/object_storage.go:172 +0x175
github.com/juicedata/juicefs/cmd.createStorage({{0xc000052120, 0x5}, {0xc00084c360, 0x24}, {0xc00005212e, 0x2}, {0x0, 0x0}, {0xc000844018, 0x11}, ...})
/go/src/github.com/juicedata/juicefs/cmd/format.go:239 +0x50a
github.com/juicedata/juicefs/cmd.format(0xc0002b7180)
/go/src/github.com/juicedata/juicefs/cmd/format.go:467 +0xf25
github.com/urfave/cli/v2.(*Command).Run(0xc0010ee120, 0xc001135c80)
/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/command.go:177 +0x719
github.com/urfave/cli/v2.(*App).RunContext(0xc000aeba40, {0x3874230?, 0xc000052088}, {0xc000eca200, 0x8, 0x8})
/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:389 +0x1035
github.com/urfave/cli/v2.(*App).Run(...)
/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:254
github.com/juicedata/juicefs/cmd.Main({0xc000eca000?, 0x8?, 0x8?})
/go/src/github.com/juicedata/juicefs/cmd/main.go:94 +0x354c
main.main()
/go/src/github.com/juicedata/juicefs/main.go:29 +0x2e
[root@ip-172-31-45-198 ~]#

What you expected to happen:

Format a volume with the bucket juicefs-us-east-1.

How to reproduce it (as minimally and precisely as possible):

Format a volume with a s3 bucket in us-east-1

Anything else we need to know?

GetBucketLocation will return nil in us-east-1.
image

Environment:

  • JuiceFS version (use juicefs --version) or Hadoop Java SDK version:

[root@ip-172-31-45-198 ~]# juicefs --version
juicefs version 1.1.2+2024-02-04.8dbd89a

  • Cloud provider or hardware configuration running JuiceFS:

Cloud provider: aws
Instance Type: c5.xlarge

  • OS (e.g cat /etc/os-release):

[root@ip-172-31-45-198 ~]# cat /etc/os-release
NAME="Rocky Linux"
VERSION="9.3 (Blue Onyx)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.3"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Rocky Linux 9.3 (Blue Onyx)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:9::baseos"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2032-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9"
ROCKY_SUPPORT_PRODUCT_VERSION="9.3"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.3"

  • Kernel (e.g. uname -a):

[root@ip-172-31-45-198 ~]# uname -a
Linux ip-172-31-45-198.ec2.internal 5.14.0-362.8.1.el9_3.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov 8 17:36:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

  • Object storage (cloud provider and region, or self maintained):

s3

  • Metadata engine info (version, cloud provider managed or self maintained):

sqlite

  • Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):
  • Others:
@mwqgithub mwqgithub added the kind/bug Something isn't working label Apr 7, 2024
@zhijian-pro zhijian-pro self-assigned this Apr 7, 2024
@zhijian-pro
Copy link
Contributor

juicefs format \
    --storage minio \
    --bucket http://127.0.0.1:9000/<bucket> \
    --access-key minioadmin \
    --secret-key minioadmin \
    ... \
    myjfs

Your bucket URL format may not be correct, please refer to the document below to set the bucket URL correctly.
https://juicefs.com/docs/community/reference/how_to_set_up_object_storage/#minio

@zhijian-pro zhijian-pro removed the kind/bug Something isn't working label Apr 7, 2024
@mwqgithub
Copy link
Author

juicefs format \
    --storage minio \
    --bucket http://127.0.0.1:9000/<bucket> \
    --access-key minioadmin \
    --secret-key minioadmin \
    ... \
    myjfs

Your bucket URL format may not be correct, please refer to the document below to set the bucket URL correctly. https://juicefs.com/docs/community/reference/how_to_set_up_object_storage/#minio

Change bucket URL worked. Thanks for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants