Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

implement migration v2 #746

Merged
merged 42 commits into from
Sep 15, 2023
Merged

implement migration v2 #746

merged 42 commits into from
Sep 15, 2023

Conversation

w-henderson
Copy link
Contributor

@w-henderson w-henderson commented Jul 12, 2023 •
edited
Loading

This commit adds the core implementation of migration v2 according to the spec.

@w-henderson
implement migration v2 in libvfio-user
3d6d384 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: implementation fixes from comments
a70cbe9 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: more implementation changes for comments
2898d38 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: more fixes and responses to comments
3aaaa70 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
tmakatos
tmakatos reviewed Jul 14, 2023
View reviewed changes
include/libvfio-user.h Outdated Show resolved Hide resolved
include/libvfio-user.h Show resolved Hide resolved
include/libvfio-user.h Outdated Show resolved Hide resolved
include/libvfio-user.h Outdated Show resolved Hide resolved
include/vfio-user.h Show resolved Hide resolved
include/vfio-user.h Show resolved Hide resolved
lib/libvfio-user.c Show resolved Hide resolved
lib/libvfio-user.c Outdated Show resolved Hide resolved
@w-henderson
fix: make migration data size 32-bit
de676cf 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson w-henderson force-pushed the w-henderson/migration-v2-implementation branch from 89ed6fa to de676cf Compare July 14, 2023 11:53
@w-henderson
fix: further fixes in response to comments
d7d7122 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
feat: begin implementing DMA dirty page logging (WIP)
453f673 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: bugs in dirty page logging
d317b9a 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: dirty page logging bug fixes
1ce8931 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: wrong size for dirty page bitmap
29f91e6 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: prevent uninitialized return value
e781a19 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: don't try to log non-mapped regions
3a8f13f 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: set argsz when getting dirty bitmap
1476fa2 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: conflict with kernel definitions
d5ecd25 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson w-henderson force-pushed the w-henderson/migration-v2-implementation branch from 12c2779 to d5ecd25 Compare August 1, 2023 09:39
@w-henderson
fix: crash when not logging all DMA regions
5d46895 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson w-henderson mentioned this pull request Aug 2, 2023
Closed
5 tasks
@w-henderson
fix: prevent uninitialised bytes being sent on error
3e3f04b 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
Base automatically changed from w-henderson/migration-v2-spec to master-migv2 August 3, 2023 08:35
@w-henderson
fix: migration FSM transitions
49701eb 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: error returns incorrectly not using ERROR_INT
c2cc2e4 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: make state transitions idempotent
7b2a8cf 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
fix: disallow STOP_COPY to PRE_COPY transition
73765a7 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson w-henderson requested a review from jlevon August 7, 2023 10:35
@w-henderson
fix: out of bounds read
c40d16f 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
jlevon
jlevon reviewed Aug 8, 2023
View reviewed changes
include/libvfio-user.h Outdated Show resolved Hide resolved
include/libvfio-user.h Outdated Show resolved Hide resolved
include/vfio-user.h Outdated Show resolved Hide resolved
include/vfio-user.h Outdated Show resolved Hide resolved
include/vfio-user.h Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
@w-henderson
refactor: respond to John's review
cea84e7 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
refactor: move transition definitions to migration.c
96ef311 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@jlevon
Copy link
Collaborator

jlevon commented Aug 29, 2023

Is it too much to move the dirty page logging changes into a separate PR?

That doesn't seem worth it presuming we're going to merge the whole dev branch in a single change into main later.

tmakatos
tmakatos reviewed Aug 29, 2023
View reviewed changes
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
@w-henderson
changes from Thanos's review
977211f 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
tmakatos
tmakatos reviewed Aug 30, 2023
View reviewed changes
include/libvfio-user.h Show resolved Hide resolved
include/libvfio-user.h Outdated Show resolved Hide resolved
include/vfio-user.h Outdated Show resolved Hide resolved
lib/common.h Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
@w-henderson
improve variable naming and documentation
3c9dcab 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson w-henderson force-pushed the w-henderson/migration-v2-implementation branch from 88b64a9 to 3c9dcab Compare August 30, 2023 09:37
tmakatos
tmakatos reviewed Aug 30, 2023
View reviewed changes
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/libvfio-user.c Outdated Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
tmakatos
tmakatos reviewed Aug 30, 2023
View reviewed changes
lib/dma.c Outdated Show resolved Hide resolved
jlevon
jlevon reviewed Aug 30, 2023
View reviewed changes
lib/dma.c Outdated Show resolved Hide resolved
jlevon
jlevon reviewed Aug 30, 2023
View reviewed changes
lib/dma.h Outdated Show resolved Hide resolved
@w-henderson
more fixes from Thanos's review
1507e56 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson
refactor dirty page get
f1927fa 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
tmakatos
tmakatos reviewed Aug 30, 2023
View reviewed changes
Copy link
Member

@tmakatos tmakatos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new DMA reporting code is vastly easier to understand, good job!

lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Show resolved Hide resolved
lib/dma.c Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
lib/migration.c Outdated Show resolved Hide resolved
@w-henderson
respond to more of Thanos's review
2c98f1d 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
jlevon
jlevon reviewed Aug 30, 2023
View reviewed changes
include/libvfio-user.h Show resolved Hide resolved
include/libvfio-user.h Show resolved Hide resolved
lib/common.h Outdated Show resolved Hide resolved
lib/dma.c Show resolved Hide resolved
lib/dma.c Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/dma.c Outdated Show resolved Hide resolved
lib/libvfio-user.c Outdated Show resolved Hide resolved
lib/migration.c Show resolved Hide resolved
@w-henderson
minor changes from John's review
9ce4988 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
tmakatos
tmakatos previously approved these changes Aug 31, 2023
View reviewed changes
@tmakatos tmakatos dismissed their stale review August 31, 2023 11:26

removing approval due to pending discussion and CI errors

@w-henderson
introduce iov_free
fde85d6 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@w-henderson w-henderson mentioned this pull request Aug 31, 2023
Open
jlevon
jlevon approved these changes Aug 31, 2023
View reviewed changes
lib/libvfio-user.c Show resolved Hide resolved
@tmakatos
Copy link
Member

@w-henderson let me know once you've fixed the CI errors

@tmakatos
Copy link
Member

@w-henderson let me know once you've fixed the CI errors

My bad, updating samples and tests and in next PRs, CI will never work in this PR.

@w-henderson
fix: migration quiesce and short write
9ba1d7e 
Signed-off-by: William Henderson <william.henderson@nutanix.com>
@jlevon jlevon merged commit 5562eb7 into master-migv2 Sep 15, 2023
1 of 5 checks passed
@jlevon jlevon deleted the w-henderson/migration-v2-implementation branch September 15, 2023 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tmakatos tmakatos tmakatos approved these changes

@jlevon jlevon jlevon approved these changes

Assignees
No one assigned
Labels
migration-v2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@w-henderson @jlevon @tmakatos