Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fancy and more efficient syscalls to copy (sparse) files on Linux #218

Merged
merged 1 commit into from
May 20, 2024

Conversation

vpodzime
Copy link
Contributor

The FICLONE ioctl() is the most efficient way to copy a file inside one file system if available and supported. copy_file_range() is an alternative that can provide similar efficiency, but also only works inside one file system and requires us to take care of holes in files. sendfile() is very similar, but unline copy_file_range() it doesn't support reflinks.

Ticket: CFE-4380
Changelog: FileSparseCopy() now uses FICLONE ioctl(),
copy_file_range() or sendfile() on Linux
platforms (if available)

@vpodzime vpodzime requested a review from larsewi April 30, 2024 15:33
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from 0bfb82d to 08935a5 Compare April 30, 2024 18:23
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime requested a review from olehermanse May 2, 2024 07:39
larsewi
larsewi previously approved these changes May 2, 2024
Copy link
Contributor

@larsewi larsewi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work 🚀

libutils/file_lib.c Show resolved Hide resolved
libutils/file_lib.c Show resolved Hide resolved
libutils/file_lib.c Show resolved Hide resolved
@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from 08935a5 to 2cf2738 Compare May 3, 2024 06:17
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

larsewi
larsewi previously approved these changes May 3, 2024
@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from 2cf2738 to cae11d6 Compare May 3, 2024 08:17
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from cae11d6 to d26863d Compare May 6, 2024 06:16
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

olehermanse
olehermanse previously approved these changes May 6, 2024
Copy link
Contributor

@olehermanse olehermanse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, a few smaller comments.

libutils/file_lib.c Outdated Show resolved Hide resolved
libutils/file_lib.c Show resolved Hide resolved
libutils/file_lib.c Show resolved Hide resolved
libutils/file_lib.c Show resolved Hide resolved
libutils/file_lib.c Outdated Show resolved Hide resolved
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from 55d0157 to b23d9f4 Compare May 6, 2024 15:16
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from b23d9f4 to ee77078 Compare May 6, 2024 22:59
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from ee77078 to a6d6640 Compare May 7, 2024 06:23
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from a6d6640 to 568d3a3 Compare May 17, 2024 10:50
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from 568d3a3 to 9988c09 Compare May 17, 2024 10:52
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

The FICLONE ioctl() is the most efficient way to copy a file
inside one file system if available and supported.
copy_file_range() is an alternative that can provide similar
efficiency, but also only works inside one file system and
requires us to take care of holes in files. sendfile() is very
similar, but unline copy_file_range() it doesn't support
reflinks.

Ticket: CFE-4380
Changelog: FileSparseCopy() now uses FICLONE ioctl(),
           copy_file_range() or sendfile() on Linux
           platforms (if available)
@vpodzime vpodzime force-pushed the master-efficient_data_copy branch from 9988c09 to e644c0c Compare May 17, 2024 13:25
@mender-test-bot
Copy link

There was an error running your pipeline, see logs for details.

@vpodzime vpodzime merged commit 5f195b2 into NorthernTechHQ:master May 20, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants