Merged PR 4636050: update hcsshim to 43d3084

Related work items: microsoft#955, microsoft#985, microsoft#1023, microsoft#1026, microsoft#1032, microsoft#1034, microsoft#1038, microsoft#1041, microsoft#1044

.github/workflows/ci.yml

@@ -68,6 +68,7 @@ jobs:
       - run: go build ./cmd/wclayer
       - run: go build ./cmd/device-util
       - run: go build ./cmd/ncproxy
+      - run: go build ./cmd/dmverity-vhd
       - run: go build ./internal/tools/grantvmgroupaccess
       - run: go build ./internal/tools/uvmboot
       - run: go build ./internal/tools/zapdir

README.md

@@ -2,9 +2,63 @@
 
 [![Build status](https://github.com/microsoft/hcsshim/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/microsoft/hcsshim/actions?query=branch%3Amaster)
 
-This package contains the Golang interface for using the Windows [Host Compute Service](https://techcommunity.microsoft.com/t5/containers/introducing-the-host-compute-service-hcs/ba-p/382332) (HCS) to launch and manage [Windows Containers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/). It also contains other helpers and functions for managing Windows Containers such as the Golang interface for the Host Network Service (HNS).
+This package contains the Golang interface for using the Windows [Host Compute Service](https://techcommunity.microsoft.com/t5/containers/introducing-the-host-compute-service-hcs/ba-p/382332) (HCS) to launch and manage [Windows Containers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/). It also contains other helpers and functions for managing Windows Containers such as the Golang interface for the Host Network Service (HNS), as well as code for the [guest agent](./internal/guest/README.md) (commonly referred to as the GCS or Guest Compute Service in the codebase) used to support running Linux Hyper-V containers.
 
-It is primarily used in the [Moby Project](https://github.com/moby/moby), but it can be freely used by other projects as well.
+It is primarily used in the [Moby](https://github.com/moby/moby) and [Containerd](https://github.com/containerd/containerd) projects, but it can be freely used by other projects as well.
+
+## Building
+
+While this repository can be used as a library of sorts to call the HCS apis, there are a couple binaries built out of the repository as well. The main ones being the Linux guest agent, and an implementation of the [runtime v2 containerd shim api](https://github.com/containerd/containerd/blob/master/runtime/v2/README.md).
+### Linux Hyper-V Container Guest Agent
+
+To build the Linux guest agent itself all that's needed is to set your GOOS to "Linux" and build out of ./cmd/gcs.
+```powershell
+C:\> $env:GOOS="linux"
+C:\> go build .\cmd\gcs\
+```
+
+or on a Linux machine
+```sh
+> go build ./cmd/gcs
+```
+
+If you want it to be packaged inside of a rootfs to boot with alongside all of the other tools then you'll need to provide a rootfs that it can be packaged inside of. An easy way is to export the rootfs of a container.
+
+```sh
+docker pull busybox
+docker run --name base_image_container busybox
+docker export base_image_container | gzip > base.tar.gz
+BASE=./base.tar.gz
+make all
+```
+
+If the build is successful, in the `./out` folder you should see:
+```sh
+> ls ./out/
+delta.tar.gz  initrd.img  rootfs.tar.gz
+```
+
+### Containerd Shim
+For info on the Runtime V2 API: https://github.com/containerd/containerd/blob/master/runtime/v2/README.md.
+
+Contrary to the typical Linux architecture of shim -> runc, the runhcs shim is used both to launch and manage the lifetime of containers.
+
+```powershell
+C:\> $env:GOOS="windows"
+C:\> go build .\cmd\containerd-shim-runhcs-v1
+```
+
+Then place the binary in the same directory that Containerd is located at in your environment. A default Containerd configuration file can be generated by running:
+```powershell
+.\containerd.exe config default | Out-File "C:\Program Files\containerd\config.toml" -Encoding ascii
+```
+
+This config file will already have the shim set as the default runtime for cri interactions.
+
+To trial using the shim out with ctr.exe:
+```powershell
+C:\> ctr.exe run --runtime io.containerd.runhcs.v1 --rm mcr.microsoft.com/windows/nanoserver:2004 windows-test cmd /c "echo Hello World!"
+```
 
 ## Contributing
 
@@ -16,7 +70,7 @@ When you submit a pull request, a CLA-bot will automatically determine whether y
 a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions
 provided by the bot. You will only need to do this once across all repos using our CLA.
 
-We also ask that contributors [sign their commits](https://git-scm.com/docs/git-commit) using `git commit -s` or `git commit --signoff` to certify they either authored the work themselves or otherwise have permission to use it in this project. 
+We also ask that contributors [sign their commits](https://git-scm.com/docs/git-commit) using `git commit -s` or `git commit --signoff` to certify they either authored the work themselves or otherwise have permission to use it in this project.
 
 
 ## Code of Conduct

cmd/containerd-shim-runhcs-v1/delete.go

@@ -99,31 +99,6 @@ The delete command will be executed in the container's bundle as its cwd.
 			}
 		}
 
-		// Determine if the config file was a POD and if so kill the whole POD.
-		if s, err := getSpecAnnotations(bundleFlag); err != nil {
-			if !os.IsNotExist(err) {
-				return err
-			}
-		} else {
-			if containerType := s["io.kubernetes.cri.container-type"]; containerType == "container" {
-				if sandboxID := s["io.kubernetes.cri.sandbox-id"]; sandboxID != "" {
-					if sys, _ := hcs.OpenComputeSystem(ctx, sandboxID); sys != nil {
-						if err := sys.Terminate(ctx); err != nil {
-							fmt.Fprintf(os.Stderr, "failed to terminate '%s': %v", idFlag, err)
-						} else if err := sys.Wait(); err != nil {
-							fmt.Fprintf(os.Stderr, "failed to wait for '%s' to terminate: %v", idFlag, err)
-						}
-						sys.Close()
-					}
-				}
-			}
-		}
-
-		// Remove the bundle on disk
-		if err := os.RemoveAll(bundleFlag); err != nil && !os.IsNotExist(err) {
-			return err
-		}
-
 		if data, err := proto.Marshal(&task.DeleteResponse{
 			ExitedAt:   time.Now(),
 			ExitStatus: 255,

cmd/dmverity-vhd/README.md

@@ -0,0 +1,33 @@
+# dmverity-vhd
+
+Takes an OCI image locator and an output directory and converts the layers that
+make up the image into a series of VHDs in the output directory. One VHD will
+be created per image layer.
+
+VHDs are named with the name of the layer SHA.
+
+Each layer contains
+[dm-verity](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/verity.html)
+information that can be used to ensure the integrity of the created ext4
+filesystem. All VHDs have a layout of:
+
+- ext4 filesystem
+- dm-verity superblock
+- dm-verity merkle tree
+- VHD footer
+
+The output is deterministic except for the UUIDs embedded in the VHD footer and
+the dm-verity superblock. Both UUIDs are currently seeded using a random number
+generator.
+
+## Example usage
+
+Create VHDs:
+```bash
+dmverity-vhd create -i alpine:3.12 -o alpine_3_12_layers
+```
+
+Compute root hashes:
+```bash
+dmverity-vhd roothash -i alpine:3.12
+```