-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: amplify io is too large to hold in fuse buffer #1311
fix: amplify io is too large to hold in fuse buffer #1311
Conversation
@ccx1024cc , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/77059 |
@ccx1024cc , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/77064 |
@ccx1024cc , The CI test is completed, please check result:
Congratulations, your test job passed! |
@ccx1024cc , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/77074 |
@ccx1024cc , The CI test is completed, please check result:
Sorry, your test job failed. Please get the details in the link. |
@ccx1024cc , The CI test is completed, please check result:
Congratulations, your test job passed! |
@ccx1024cc , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/77083 |
@ccx1024cc , The CI test is completed, please check result:
Congratulations, your test job passed! |
@ccx1024cc , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/77111 |
@ccx1024cc , The CI test is completed, please check result:
Congratulations, your test job passed! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned. Reproduction: run nydusd with 3MB amplify_io error from random io: reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" } Details: size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header) let amplify_io = min(user_specified, fuseWriter.available_bytes()) Resolution: This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs"). In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135"). Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com>
bd2d244
to
6099fda
Compare
@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/84091 |
@ccx1024cc , The CI test is completed, please check result:
Sorry, your test job failed. Please get the details in the link. |
Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com>
6099fda
to
fa3c6a6
Compare
@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/84094 |
@ccx1024cc , The CI test is completed, please check result:
Congratulations, your test job passed! |
* fix: amplify io is too large to hold in fuse buffer Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned. Reproduction: run nydusd with 3MB amplify_io error from random io: reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" } Details: size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header) let amplify_io = min(user_specified, fuseWriter.available_bytes()) Resolution: This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs"). In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135"). Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> * feat: e2e for amplify io larger than fuse buffer Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> --------- Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> Co-authored-by: 泰友 <cuichengxu.ccx@antgroup.com>
* fix: amplify io is too large to hold in fuse buffer Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned. Reproduction: run nydusd with 3MB amplify_io error from random io: reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" } Details: size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header) let amplify_io = min(user_specified, fuseWriter.available_bytes()) Resolution: This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs"). In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135"). Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> * feat: e2e for amplify io larger than fuse buffer Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> --------- Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> Co-authored-by: 泰友 <cuichengxu.ccx@antgroup.com>
* fix: amplify io is too large to hold in fuse buffer Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned. Reproduction: run nydusd with 3MB amplify_io error from random io: reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" } Details: size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header) let amplify_io = min(user_specified, fuseWriter.available_bytes()) Resolution: This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs"). In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135"). Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> * feat: e2e for amplify io larger than fuse buffer Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> --------- Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> Co-authored-by: 泰友 <cuichengxu.ccx@antgroup.com>
backport the logic of master commit c8a39c8 * fix: amplify io is too large to hold in fuse buffer Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned. Reproduction: run nydusd with 3MB amplify_io error from random io: reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" } Details: size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header) let amplify_io = min(user_specified, fuseWriter.available_bytes()) Resolution: This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs"). In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135"). Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> * feat: e2e for amplify io larger than fuse buffer Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> --------- Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> Co-authored-by: 泰友 <cuichengxu.ccx@antgroup.com> Signed-off-by: Peng Tao <bergwolf@hyper.sh>
backport the logic of master commit c8a39c8 * fix: amplify io is too large to hold in fuse buffer Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned. Reproduction: run nydusd with 3MB amplify_io error from random io: reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" } Details: size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header) let amplify_io = min(user_specified, fuseWriter.available_bytes()) Resolution: This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs"). In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135"). Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> * feat: e2e for amplify io larger than fuse buffer Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> --------- Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com> Co-authored-by: 泰友 <cuichengxu.ccx@antgroup.com> Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Fuse request buffer is fixed by
FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE
. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned.Reproduction
Details
Resolution
Evalucation of amplify_io will be replaced with ZeroCopyWriter.available_bytes().
Relevant Issue
#1270
ci: https://github.com/dragonflyoss/image-service/actions/runs/4931748472/jobs/8814099799
#1312