a generic way to handle remote snapshotter

[xujihui1985]

notice recently overlaybd was merged by this PR the #603, and the code looks ugly and lack of design, I think it's time to consider a generic way to handle remote snapshot not just hard code for some specific one.

[AkihiroSuda]

I guess snapshotter plugins should expose some common annotations in plugin.Exports
https://github.com/containerd/containerd/blob/7fa44fc98f4bde30fa154c361e11609f742aaccb/api/services/introspection/v1/introspection.proto#L66

cc @BigVan @ktock

[xujihui1985]

@AkihiroSuda does plugin.Exports just for in-tree plugin? what about remote snapshotter?

[changweige]

I guess snapshotter plugins should expose some common annotations in plugin.Exports https://github.com/containerd/containerd/blob/7fa44fc98f4bde30fa154c361e11609f742aaccb/api/services/introspection/v1/introspection.proto#L66

cc @BigVan @ktock

@AkihiroSuda
Thank you for the potential solution for adding more crucial stuff to downloaded image by nerdctl . Those extra metadata is
essential for a remote snapshotter especially for those with lazily read capability e.g. estargz and overlaybd. Solutions for how to accelerate container startup are flourishing these days and in fact we are working on another one called nydus. We are also pursuing close and efficient method to integrate with containerd as a remote snapshotter. So better all remote snapshotters can have a fair and common interface to pass the bits they need.

Hi @ktock , do you think it's possible we can do something on this? Thanks ahead. ;-)

[fuweid]

@xujihui1985 @changweige

Community always welcome the proposal design and PR about enhancement.
And also welcome to get involved into the code review of prs which you are
interested in.

There are still some issues about how to integrate remote snapshotter with
containerd, which @ktock are trying to provide several good proposals to solve.
Since those issues are known to each remote snapshotters, I proposed to setup
discuss group or working group to fix that, which mentioned in Kubecon 2021
China - containerd office hours. We can add the agenda to talk about this in
next containerd community meeting (currently, scheduled at Jan 13, 2022).
It will be a lot of fun in 2022 :)

[xujihui1985]

@fuweid thanks，nice to know about the working group, we are exciting to be part of it.

[changweige]

@xujihui1985 @changweige

Community always welcome the proposal design and PR about enhancement. And also welcome to get involved into the code review of prs which you are interested in.

There are still some issues about how to integrate remote snapshotter with containerd, which @ktock are trying to provide several good proposals to solve. Since those issues are known to each remote snapshotters, I proposed to setup discuss group or working group to fix that, which mentioned in Kubecon 2021 China - containerd office hours. We can add the agenda to talk about this in next containerd community meeting (currently, scheduled at Jan 13, 2022). It will be a lot of fun in 2022 :)

@fuweid
Setting up a working group to address those concerns from snapshotter vendors sound very cool.
Can you share us a way to get involved in the community activity?

[BigVan]

I agree with u about the working group. For remote image support in containerd is preliminary now, sth like the design specification and authentication on lazy pulling are also needed more discussion. @fuweid

[fuweid]

@xujihui1985 @changweige
Community always welcome the proposal design and PR about enhancement. And also welcome to get involved into the code review of prs which you are interested in.
There are still some issues about how to integrate remote snapshotter with containerd, which @ktock are trying to provide several good proposals to solve. Since those issues are known to each remote snapshotters, I proposed to setup discuss group or working group to fix that, which mentioned in Kubecon 2021 China - containerd office hours. We can add the agenda to talk about this in next containerd community meeting (currently, scheduled at Jan 13, 2022). It will be a lot of fun in 2022 :)

@fuweid Setting up a working group to address those concerns from snapshotter vendors sound very cool. Can you share us a way to get involved in the community activity?

Currently, we use https://www.cncf.io/calendar/ to do schedule. And the doc is here https://docs.google.com/document/d/1Q8KyVJd26oAQ3MafbnkVBgJCopPl2Bw45H9L9br9Vus/edit. Hope it can help.

[xujihui1985]

@fuweid thanks. As further discussion will be going in a dedicated working group and we will participate in this discussion to make this feature better. I'm close this issue for now.