Move to vanilla firecracker snapshots

[CuriousGeorgiy]

Summary

Closes #794

Implementation Notes ⚒️

• Replaced the tap manager with the new network manager and dropped the tap manager;
• Reworked snapshot manager to maintain one list of snapshots with a new ready feature;
• Created a device snapshot for the base container image during patch creation;
• Dropped VM offloading APIs and replaced them with snapshot creation or VM shutdown;
• Added a new network pool size option to the orchestrator;
• Updated firecracker-related binaries;
• Updated firecracker integration code;
• Added basic remote snapshot create and load tests.

Snapshot revisions

Snapshots are now identified by a function instance revision, which is naturally provided by the K_REVISION environment variable of knative¹.

External Dependencies 🍀

• Encapsulate the host interface name into the VM pool and tap manager #811 must be merged first;
• Various enhancements to the vHive function pool #817 must be merged first;
• Added rsync to system dependencies.

Breaking API Changes ⚠️

• N/A.

Footnotes

1. https://knative.dev/docs/concepts/serving-resources/revisions/ ↩

[leokondrashov]

@CuriousGeorgiy, there are several errors with lint and compiler (unused import and formatting). Please fix and push.

[CuriousGeorgiy]

@leokondrashov done.

[lrq619]

Hello George, codes on this branch cannot pass firecracker-cri tests.
Codes on the main branch can pass tests so it's not the problem of runners. You can see here: https://github.com/vhive-serverless/vHive/actions/runs/6460688210

Please check what is broken in the branch and make sure it passes the firecracker-cri tests, thank you!

[CuriousGeorgiy]

@lrq619 Hi! What you are saying is true, but at the same time:

1. If I set up the CRI environment on a fresh CloudLab node, and run the CRI tests, everything works fine.
2. I cannot see any error messages or signs of something going wrong in the logs.

So I don't know how to reproduce this issue, and I am not sure it is inherently related to my changes, it may as well be something like #781 or something related to #836.

I have already addressed this to @leokondrashov and kindly asked him to take a look.

[ustiugov]

let's postpone the gvisor upgrade and merge this PR. @leokondrashov, can you please proceed to a merge without the gvisor upgrade. in case of runner failure, please Ruiqi for help

[ustiugov]

@CuriousGeorgiy can you please rebase your branch so we can check the latest CI status? If the problem indeed persists, you have to debug it. if the problem is with the runners, we will take over (but as tests pass in main, it doesn't seem to be the case)

I cannot see any error messages or signs of something going wrong in the logs.

the logs are uploaded as artifacts, please check there.

in future, please have your branches inside the vHive upstream repo, otherwise we cannot take over tasks from you.

[leokondrashov]

The issue with PRs from forks was not connected to the accesses. It was pre-push hook that was trying to access fork's LFS which we don't need.

[CuriousGeorgiy]

@ustiugov

the logs are uploaded as artifacts, please check there.

As I said, I have checked the artifacts and haven't found any errors compared to the main branch.

I have also tried setting up the CRI environment on a fresh CloudLab node (it succeeds) and running CRI tests there (it also succeeds), so I am not sure how I can debug the issue without access to the runners.

[CuriousGeorgiy]

@leokondrashov 1 meaningful difference I see in the containerd logs between this branch and the main branch is that on this branch containerd doesn't get any more requests after the istio-proxy pod is (successfully) created.

[leokondrashov]

I managed to make the test work. This was a strange behaviour of runner.

Later the result might change because Ruiqi will be testing runners once again. But we can merge, finally.