API: Add support for fetching images across all projects (from Incus) #14260

kadinsayani · 2024-10-10T23:24:18Z

Resolves #14126.

This PR adds support for fetching images across all projects by adding a query parameter all-projects.

github-actions · 2024-10-10T23:24:31Z

Heads up @mionaalex - the "Documentation" label was applied to this issue.

lxd/images.go

tomponline

The all-projects parameter should only change which resources are returned, it should not change the output format.

kadinsayani · 2024-11-06T14:26:27Z

This PR should be good to go. It adds support for the --all-projects flag when running lxc image list. I'm planning to open separate PR's for --all-projects support for network zones, profiles, networks, network-acls, and storage buckets (Incus cherry picks).

markylaing

Can you please add something to reduce the number of images returned by the database when the caller is not authenticated?

Other than that I just have a few questions :)

lxd/images.go

client/interfaces.go

lxc/image.go

markylaing

Just a few nits that should be easy to fix :)

lxd/images.go

client/lxd_images.go

kadinsayani · 2024-11-11T14:21:18Z

Just a few nits that should be easy to fix :)

Thanks for the review @markylaing :) I think I've addressed everything now.

markylaing

Just a couple of last bits. Thanks :)

lxd/images.go

kadinsayani · 2024-11-21T22:59:32Z

Rebased and good to go 👍

doc/api-extensions.md

markylaing

Please double check the API extension commit and cherry-pick if it was added to Incus (splitting the doc change if necessary). Other than that, LGTM. cc @tomponline.

tomponline

There are no tests, we especially need tests to check access control of images.

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit 09576c289f57dc657ad99ef4a9a2fc5630d78007) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit 087e0b3701787e9ab237d57321aff329b706295b) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit a3544074d69dd2438986311d2996a408deb5ad2e) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit 34ae1bd35a1ea79fb78b06dd0de7eb5f70ccb4a6) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit f5c5a8170106e94ed314f5761b4b91d9621acc65) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

…ojects` This commit adds tests to `test/suites/auth.sh` for fetching images across all projects. Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

kadinsayani · 2024-11-22T20:10:54Z

There are no tests, we especially need tests to check access control of images.

Tests added! 😄

tomponline

thanks!

changes made

Fixes #14561 introduced with #14260. This PR fixes a regression introduced with #14260 by adding `GetImagesAllProjects` and `GetImagesAllProjectsWithFilter` functions to `client/simplestreams_images.go`, and moving their method signatures to the `ImageServer` interface. While adding support for fetching images across all projects, we changed the logic in the `lxc image list` command to call different functions based on whether the `--all-projects` flag is passed in or not: 1fae6e6 This is fine, but we needed to add `simplestreams` protocol support as well to facilitate listing public images from remotes such as `images` and `ubuntu-minimal-daily`. The actual bug comes from the following LOC: https://github.com/canonical/lxd/blob/4a3c4a343e4a9490105bd3da893f9892b86e01d8/lxc/image.go#L1356-L1359 Fetching from an instance server isn't applicable for `simplestreams` servers, so `GetImagesAllProjects` is never reached due to the error raised by `c.global.conf.GetInstanceServer`. Instead, we should fetch an image server before fetching images, hence why moving the new client functions method signatures to the `ImageServer` interface fixes the regression.

kadinsayani requested review from tomponline and markylaing October 10, 2024 23:24

github-actions bot added Documentation Documentation needs updating API Changes to the REST API labels Oct 10, 2024

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch from 80348bf to 9c8ceab Compare October 11, 2024 00:07

tomponline reviewed Oct 14, 2024

View reviewed changes

lxd/images.go Outdated Show resolved Hide resolved

tomponline requested changes Oct 15, 2024

View reviewed changes

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch from 9c8ceab to a949a09 Compare October 28, 2024 16:26

kadinsayani marked this pull request as draft October 28, 2024 16:27

kadinsayani changed the title ~~Add support for fetching images across all projects~~ WIP: Add support for fetching images across all projects (from Incus) Oct 28, 2024

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch from a949a09 to 5308878 Compare October 29, 2024 09:45

kadinsayani changed the title ~~WIP: Add support for fetching images across all projects (from Incus)~~ Add support for fetching images across all projects (from Incus) Oct 29, 2024

kadinsayani marked this pull request as ready for review October 29, 2024 09:49

kadinsayani requested a review from tomponline October 30, 2024 09:24

markylaing requested changes Nov 6, 2024

View reviewed changes

lxd/images.go Outdated Show resolved Hide resolved

client/interfaces.go Outdated Show resolved Hide resolved

lxc/image.go Outdated Show resolved Hide resolved

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch 4 times, most recently from 53c78dc to 4b4ae6b Compare November 8, 2024 16:23

kadinsayani requested a review from markylaing November 8, 2024 16:37

markylaing requested changes Nov 8, 2024

View reviewed changes

lxd/images.go Outdated Show resolved Hide resolved

lxd/images.go Outdated Show resolved Hide resolved

client/lxd_images.go Outdated Show resolved Hide resolved

client/lxd_images.go Outdated Show resolved Hide resolved

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch 3 times, most recently from 79816ce to 3722085 Compare November 8, 2024 18:53

kadinsayani requested a review from markylaing November 11, 2024 13:45

markylaing previously requested changes Nov 11, 2024

View reviewed changes

lxd/images.go Show resolved Hide resolved

lxd/images.go Outdated Show resolved Hide resolved

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch from 3722085 to 3a1f0a2 Compare November 11, 2024 18:38

markylaing reviewed Nov 22, 2024

View reviewed changes

doc/api-extensions.md Outdated Show resolved Hide resolved

markylaing reviewed Nov 22, 2024

View reviewed changes

tomponline requested changes Nov 22, 2024

View reviewed changes

markylaing mentioned this pull request Nov 22, 2024

Revisit public image handling #14509

Open

MaheshPunjabi and others added 7 commits November 22, 2024 11:14

api: Add images_all_projects extension

60f0c5c

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit 09576c289f57dc657ad99ef4a9a2fc5630d78007) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

shared/api: Add project property to Image

4f51b94

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit 087e0b3701787e9ab237d57321aff329b706295b) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

lxd/images: Add support for all_projects

99e45ee

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit a3544074d69dd2438986311d2996a408deb5ad2e) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

api: Run make update-api

d7e95fb

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

lxc/client: Add GetImagesAllProjects function

7dc7739

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit 34ae1bd35a1ea79fb78b06dd0de7eb5f70ccb4a6) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

lxc/client: Add GetImagesAllProjectsWithFilter function

35a5ab5

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

lxc/image: Add --all-projects flag to list

1fae6e6

Signed-off-by: Mahesh Punjabi <mahesh.punjabi86@gmail.com> (cherry picked from commit f5c5a8170106e94ed314f5761b4b91d9621acc65) Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com> License: Apache-2.0

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch 4 times, most recently from c202961 to 7ad4209 Compare November 22, 2024 18:51

kadinsayani added 3 commits November 22, 2024 12:10

test: Add tests to test/suites/auth.sh for `lxc image list --all-pr…

404d80c

…ojects` This commit adds tests to `test/suites/auth.sh` for fetching images across all projects. Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

i18n: Update translation templates.

5e67eaa

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

i18n: Update translations.

0a73c3e

Signed-off-by: Kadin Sayani <kadin.sayani@canonical.com>

kadinsayani force-pushed the 14126-add-all-projects-support-to-fetch-images branch from 7ad4209 to 0a73c3e Compare November 22, 2024 19:10

kadinsayani requested a review from tomponline November 25, 2024 19:44

kadinsayani changed the title ~~Add support for fetching images across all projects (from Incus)~~ API: Add support for fetching images across all projects (from Incus) Nov 26, 2024

tomponline approved these changes Nov 29, 2024

View reviewed changes

tomponline merged commit f8ea6e0 into canonical:main Nov 29, 2024
25 of 26 checks passed

kadinsayani deleted the 14126-add-all-projects-support-to-fetch-images branch November 29, 2024 15:39

This was referenced Nov 29, 2024

[regression]: lxc image list <OFFICIAL REMOTE>: broken on latest/edge #14561

Closed

LXC: Fix lxc image list regression #14563

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API: Add support for fetching images across all projects (from Incus) #14260

API: Add support for fetching images across all projects (from Incus) #14260

kadinsayani commented Oct 10, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024

tomponline left a comment

kadinsayani commented Nov 6, 2024

markylaing left a comment

markylaing left a comment

kadinsayani commented Nov 11, 2024

markylaing left a comment

kadinsayani commented Nov 21, 2024

markylaing left a comment

tomponline left a comment

kadinsayani commented Nov 22, 2024

tomponline left a comment

API: Add support for fetching images across all projects (from Incus) #14260

API: Add support for fetching images across all projects (from Incus) #14260

Conversation

kadinsayani commented Oct 10, 2024 • edited Loading

github-actions bot commented Oct 10, 2024

tomponline left a comment

Choose a reason for hiding this comment

kadinsayani commented Nov 6, 2024

markylaing left a comment

Choose a reason for hiding this comment

markylaing left a comment

Choose a reason for hiding this comment

kadinsayani commented Nov 11, 2024

markylaing left a comment

Choose a reason for hiding this comment

kadinsayani commented Nov 21, 2024

markylaing left a comment

Choose a reason for hiding this comment

tomponline left a comment

Choose a reason for hiding this comment

kadinsayani commented Nov 22, 2024

tomponline left a comment

Choose a reason for hiding this comment

kadinsayani commented Oct 10, 2024 •

edited

Loading