I/O error with third party application

[kocane]

So I'm trying to have a third pary tool (Acronis Backup) to include a Goofy mounted S3 bucket in the backup, but it seems to fail. Of course I'm not gonna blame Goofy, but was hoping that maybe I could do something to fix it.

When the backup tool tries to get a hold of the files, I see this error:

2020/06/04 16:36:31.515029 fuse.DEBUG Op 0x00000007        connection.go:408] <- OpenDir (inode 1)
2020/06/04 16:36:31.515080 fuse.DEBUG OpenDir 1  []
2020/06/04 16:36:31.515100 fuse.DEBUG Op 0x00000007        connection.go:491] -> OK ()
2020/06/04 16:36:31.515258 fuse.DEBUG Op 0x00000008        connection.go:408] <- ListXattr (inode 1)
2020/06/04 16:36:31.515282 fuse.DEBUG ListXattr 1  []
2020/06/04 16:36:31.515323 s3.DEBUG DEBUG: Validate Request s3/HeadObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2020/06/04 16:36:31.515337 s3.DEBUG DEBUG: Build Request s3/HeadObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2020/06/04 16:36:31.515344 s3.DEBUG DEBUG: Sign Request s3/HeadObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2020/06/04 16:36:31.515358 s3.ERROR code=InvalidParameter msg=1 validation error(s) found., err=InvalidParameter: 1 validation error(s) found.
caused by: ParamMinLenError: minimum field size of 1, HeadObjectInput.Key.

2020/06/04 16:36:31.515369 s3.ERROR code=InvalidParameter msg=1 validation error(s) found., err=InvalidParameter: 1 validation error(s) found.
caused by: ParamMinLenError: minimum field size of 1, HeadObjectInput.Key.

2020/06/04 16:36:31.515385 fuse.DEBUG Op 0x00000008        connection.go:493] -> Error: "InvalidParameter: 1 validation error(s) found.\n- minimum field size of 1, HeadObjectInput.Key.\n"
2020/06/04 16:36:31.515398 fuse.ERROR *fuseops.ListXattrOp error: InvalidParameter: 1 validation error(s) found.
2020/06/04 16:36:31.515405 fuse.ERROR - minimum field size of 1, HeadObjectInput.Key.
2020/06/04 16:36:31.515578 fuse.DEBUG Op 0x00000009        connection.go:408] <- ReleaseDirHandle
2020/06/04 16:36:31.515607 fuse.DEBUG ReleaseDirHandle
2020/06/04 16:36:31.515621 fuse.DEBUG Op 0x00000009        connection.go:491] -> OK ()

So something is pointed at xattr. In old issues I see that it's not supported but in the latest version (v0.24.0 - which I use) I see something about xattr, which makes me unsure.

Is there anything I can do.

[agates]

I am seeing this with Peertube also.

[dotslash]

I think this is a goofys bug. After a quick scan, I think the bug is this

goofys sends HeadBlob("") to s3 when xattrs are being fetched on the root inode. s3 client is rejecting the api because it expects a non empty path for the head request. Goofys should return empty list when it gets xattr request results in HeadBlob("")

[djmaze]

@dotslash Thanks for the hint. I had the same problem and made the following change (1 is the root inode):

diff --git a/internal/goofys.go b/internal/goofys.go
index b78d215..bc90187 100644
--- a/internal/goofys.go
+++ b/internal/goofys.go
@@ -462,11 +462,15 @@ func (fs *Goofys) GetXattr(ctx context.Context,
 
 func (fs *Goofys) ListXattr(ctx context.Context,
        op *fuseops.ListXattrOp) (err error) {
+       var xattrs []string
+
        fs.mu.RLock()
        inode := fs.getInodeOrDie(op.Inode)
        fs.mu.RUnlock()
 
-       xattrs, err := inode.ListXattr()
+       if inode.Id != 1 {
+               xattrs, err = inode.ListXattr()
+       }
 
        ncopied := 0

Not sure if it is the 100% correct way of doing this. But with this change, I am finally able to take backups with restic from a goofys mount.

[btakita]

I'm getting the same issue using borg backup.

See https://help.nextcloud.com/t/nextcloud-aio-backup-on-goofys-mount-extended-stat-xattrs-errno-5-i-o-error-fd-7/165499/7

Most notably getting an Errno 5. Full error coming from borg backup is:

nextcloud_aio_volumes/nextcloud_aio_nextcloud_data: extended stat (xattrs): [Errno 5] I/O error: '<FD 7>'

Debugging the mount, I got the following error:

2023/07/06 05:55:43.896653 s3.DEBUG DEBUG: Validate Request s3/HeadObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2023/07/06 05:55:43.896702 s3.DEBUG DEBUG: Build Request s3/HeadObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2023/07/06 05:55:43.896739 s3.DEBUG DEBUG: Sign Request s3/HeadObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2023/07/06 05:55:43.896761 s3.ERROR code=InvalidParameter msg=1 validation error(s) found., err=InvalidParameter: 1 validation error(s) found.
caused by: ParamMinLenError: minimum field size of 1, HeadObjectInput.Key.

2023/07/06 05:55:43.896780 s3.ERROR code=InvalidParameter msg=1 validation error(s) found., err=InvalidParameter: 1 validation error(s) found.
caused by: ParamMinLenError: minimum field size of 1, HeadObjectInput.Key.

2023/07/06 05:55:43.896806 fuse.DEBUG Op 0x00000036        connection.go:493] -> Error: "InvalidParameter: 1 validation error(s) found.\n- minimum field size of 1, HeadObjectInput.Key.\n"
2023/07/06 05:55:43.896832 fuse.ERROR *fuseops.ListXattrOp error: InvalidParameter: 1 validation error(s) found.
2023/07/06 05:55:43.896845 fuse.ERROR - minimum field size of 1, HeadObjectInput.Key.
2023/07/06 05:55:43.897283 fuse.DEBUG Op 0x00000038        connection.go:408] <- ReleaseDirHandle
2023/07/06 05:55:43.897340 fuse.DEBUG ReleaseDirHandle
2023/07/06 05:55:43.897372 fuse.DEBUG Op 0x00000038        connection.go:491] -> OK ()
2023/07/06 05:56:33.796088 fuse.DEBUG Op 0x0000003a        connection.go:408] <- LookUpInode (parent 1, name "skip.update")
2023/07/06 05:56:33.796419 fuse.DEBUG Ref 15 skip.update [1]