Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Drop versionize support, substitute with serde #4230

Merged
merged 11 commits into from
Jan 15, 2024
Merged

Drop versionize support, substitute with serde #4230

merged 11 commits into from
Jan 15, 2024

Commits on Jan 12, 2024

  1. snapshot: switch to using serde instead of Versionize 

    We want to simplify the way we handle Firecracker snapshots. Versionize
crate was created as a mechanism to help us keep track multiple versions
for each struct we want to (de)serialize in a Firecracker snapshot. This
allowed us to support forwards & backwards compatibility across multiple
Firecracker versions. In reality, we don't really make use of this
compatibility flexibility. Instead, we keep maintaining multiple
versions for struct fields (even for versions no longer supported),
which increases code and testing complexity.

We will revert to simply using serde for serializing Firecracker state
into a snapshot file. We will also decouple Firecracker from snapshot
versions. The snapshot format will now have an independent version which
starts from v1.0.0.

This change touches many parts of the code base, so we will do it
gradually. In this commit, we change the snapshot crate and the
snapshotting API. We only make the minimum changes needed to the rest of
the codebase to keep Firecracker working. We change `MicrovmState`, the
structure that describes the Firecracker state we serialize in a
snapshot to skip serialization of all its fields (essentially it is now
describing an empty snapshot). In subsequent commits, we will start
re-adding fields as we adapt the relevant structures from Versionize to
serde.

Integration tests will not be working for this commit.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    f39fdf2 View commit details
    Browse the repository at this point in the history

  2. snapshot: serialize VmInfo with serde 

    Derive Serialize & Deserialize for VmInfo and remove Versionize. Now we
can serialize this struct in the Firecracker snapshot file.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    557247b View commit details
    Browse the repository at this point in the history

  3. snapshot: serialize GuestMemoryState with serde 

    Derive Serialize and Deserialize for GuestMemoryState. Also add a unit
test for (de)serializing this type with the snapshot crate.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    b06caba View commit details
    Browse the repository at this point in the history

  4. snapshot: serialize VmState and VcpuState 

    Derive Serialize and Deserialize and remove Versionize support. Fix
these two structs together because both depend on kvm-bindings
implementing (De)serialize for its types, so updating to the
kvm-bindings to the version that introduces serde breaks both of these
types.

While implementing serde for VcpuState also remove the `msrs` field.
According to Versionize this was end-of-life and we are anyway breaking
compatibility with previous snapshot versions.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    32fe5d0 View commit details
    Browse the repository at this point in the history

  5. snapshot: serialize DeviceStates 

    Derive Serialize and Deserialize for DeviceStates and remove Versionnize
support. Do this for all device types at the same time, since they all
depend on VirtIO related state.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    ddfb0e7 View commit details
    Browse the repository at this point in the history

  6. snapshot: remove versionize dependencies 

    Remove versionize and versionize_derive dependencies.

Also, re-enable the rust integration tests. Now that we have all of the
state serialized with serde these should work again.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    4bf66dd View commit details
    Browse the repository at this point in the history

  7. fix(test): --describe-snapshot cmd line test 

    This test relies on the fact that the current Firecracker binary can
parse snapshots that were created with a previous Firecracker version.
1.7.0 changes the snapshot format header and its parser only understands
the new format. As a result, it can't parse previous snapshots. We fix
this by skipping the tests for Firecracker versions previous to 1.7.0.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    f67822d View commit details
    Browse the repository at this point in the history

  8. fix(test): microVM config after restore 

    We were skipping the serialization of the "boot_args" field of the
"boot-source" objectm, if kernel boot arguments were not supported. Now,
we return None. This was done so that we have consistent snapshot
formats, now that we use serde to serialize microVM state in a snapshot
file.

Amend this test to expect '"boot_args": None' in the "boot-source"
object.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    434e338 View commit details
    Browse the repository at this point in the history

  9. fix(test): use --snapshot-version for supported snapshot format 

    Snapshot versions are now independent from Firecracker binary versions.
Use Firecracker --snapshot-version flag to retrieve the supported
snapshot data format version for checking if the created snapshot is
supported from the current release.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    603d6d5 View commit details
    Browse the repository at this point in the history

  10. fix(test): remove tests that are not relevant any more 

    With the new snapshot format we break compatibility with all previous
snapshot format versions. This commits removes all the tests that were
checking functional correctness and performance of loading older
snapshots in the current Firecracker binary.

In the future, it might be that two Firecracker binaries support the
same snapshot versions. When that happens we should thing of a proper
way to test that this works, but the current tests are not fit for
purpose.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    74b29f1 View commit details
    Browse the repository at this point in the history

  11. doc: adapt snapshot versioning docs and CHANGELOG 

    Adapt documentation around snapshot versions format and compatibility to
remove mentions of Versionize and the effects its usage had to snapshot
compatibility. Also, add a CHANGELOG entry regarding the new strategy
for snapshot versioning.

Signed-off-by: Babis Chalios <bchalios@amazon.es>
    @bchalios
    bchalios committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    d58177b View commit details
    Browse the repository at this point in the history