Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

Support persistent memory volumes #2515

Merged
merged 8 commits into from
Mar 26, 2020

Conversation

devimc
Copy link

@devimc devimc commented Mar 6, 2020

A persistent memory volume MUST meet the following conditions:

  • A loop device must be mounted in the directory passed as volume
  • The loop device must have a backing file
  • The backing file must have the PFN signature at offset 4k [1][2]

The backing file is used as backend file for a NVDIMM device in the guest

fixes #2262

[1] - https://github.com/kata-containers/osbuilder/blob/master/image-builder
/nsdax.gpl.c
[2] - https://github.com/torvalds/linux/blob/master/drivers/nvdimm/pfn.h

cc @pohly

Signed-off-by: Julio Montes julio.montes@intel.com

@devimc devimc force-pushed the topic/pmem-CSI branch 2 times, most recently from 42f14fc to 58eba1f Compare March 6, 2020 22:14
@devimc
Copy link
Author

devimc commented Mar 6, 2020

/test

@codecov
Copy link

codecov bot commented Mar 6, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@aa0d4ee). Click here to learn what that means.
The diff coverage is 43.8%.

@@            Coverage Diff            @@
##             master    #2515   +/-   ##
=========================================
  Coverage          ?   50.35%           
=========================================
  Files             ?      118           
  Lines             ?    16947           
  Branches          ?        0           
=========================================
  Hits              ?     8533           
  Misses            ?     7380           
  Partials          ?     1034

@devimc
Copy link
Author

devimc commented Mar 11, 2020

/test-ubuntu

@devimc
Copy link
Author

devimc commented Mar 12, 2020

/test-ubuntu

@devimc
Copy link
Author

devimc commented Mar 12, 2020

@jodh-intel @amshinde @pohly @bergwolf @lifupan please take a look

Copy link
Contributor

@jodh-intel jodh-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @devimc ! Just a logger fix to add I think.

// getBackingFile is used to fetch the backing file for the device.
func getBackingFile(devInfo DeviceInfo) (string, error) {
backingFilePath := filepath.Join(getSysDevPath(devInfo), "loop", "backing_file")
if _, err := os.Stat(backingFilePath); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could remove the Stat by checking the error from ReadFile() using os.IsNotExist(err).

Julio Montes added 8 commits March 20, 2020 13:45
bring `pmem` option to pmem/nvdimm devices

shortlog:
qemu: add pmem flag to memory-backend-file

Signed-off-by: Julio Montes <julio.montes@intel.com>
`GetDevicePathAndFsType` is a function to get the path and filesystem type
of a mount point from `/proc/mounts`.
Move `GetDevicePathAndFsType` to utils_linux since it's linux specific
and that way it can be used in other subpackages.

Signed-off-by: Julio Montes <julio.montes@intel.com>
Implement function the get the backing file from a loop device.
The backing file can be used as backend file for a NVDIMM device in the guest

Signed-off-by: Julio Montes <julio.montes@intel.com>
Implement function to get the pmem `DeviceInfo` from a volume.
`PmemDeviceInfo` return a new `DeviceInfo` object if a volume has a loop device
as backend and the backing file for such loop device contains the PFN signature,
needed to enable DAX in the guest.

Signed-off-by: Julio Montes <julio.montes@intel.com>
A `BlockDrive` can be used as pmem device, since they both are similar and
can be mounted in the same way in the guest. The `Pmem` attribute helps kata
to identify a pmem device and how it has to be hotplugged in the guest.

Signed-off-by: Julio Montes <julio.montes@intel.com>
Reimplement `createBlockDevices` to identify possible volumes that can be
used as pmem devices

Signed-off-by: Julio Montes <julio.montes@intel.com>
hotplug as NVDIMM devices the block drives that can be used as pmem devices
(`Pmem=true`), the host path to such devices is a raw file that contains
the PFN signature.

Signed-off-by: Julio Montes <julio.montes@intel.com>
A persistent memory volume MUST meet the following conditions:
* A loop device must be mounted in the directory passed as volume
* The loop device must have a backing file
* The backing file must have the PFN signature at offset 4k [1][2]

The backing file is used as backend file for a NVDIMM device in the guest

fixes kata-containers#2262

[1] - https://github.com/kata-containers/osbuilder/blob/master/image-builder
/nsdax.gpl.c
[2] - https://github.com/torvalds/linux/blob/master/drivers/nvdimm/pfn.h

Signed-off-by: Julio Montes <julio.montes@intel.com>
@devimc
Copy link
Author

devimc commented Mar 20, 2020

@jodh-intel change applied, thanks

@devimc
Copy link
Author

devimc commented Mar 20, 2020

/test

Copy link
Contributor

@jodh-intel jodh-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @devimc.

lgtm

@devimc
Copy link
Author

devimc commented Mar 20, 2020

can we merge this?

cc @amshinde @pohly

@pohly
Copy link

pohly commented Mar 26, 2020

This all looks reasonable to me, with the caveat of course that I don't know the code base.

For user-facing documentation we can use the PMEM-CSI project, because that is what users will have to start with to use the new feature. I can also add end-to-end testing there once there is release of Kata Containers with this feature.

Are you doing canary builds, i.e. once this is merged, can I directly install a pre-release of Kata Containers which supports PMEM?

@devimc
Copy link
Author

devimc commented Mar 26, 2020

@pohly sure, you can download the static tarball form here https://github.com/kata-containers/runtime/releases/

@jcvenegas jcvenegas merged commit 11c998b into kata-containers:master Mar 26, 2020
@pohly
Copy link

pohly commented Mar 27, 2020

sure, you can download the static tarball form here https://github.com/kata-containers/runtime/releases/

Can you ping me when there's an alpha with this feature included?

@devimc
Copy link
Author

devimc commented Mar 27, 2020

@pohly

Can you ping me when there's an alpha with this feature included?

sure thing 👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support PMEM inside Kata Containers when running under Kubernetes
4 participants