[BUG] opensearch-dashboards-plugin remove securityDashboards sometimes does not work

[mnasiadka]

Describe the bug

Running opensearch-dashboards-plugin remove securityDashboards does not remove the plugin in some cases since 2.13.0

To Reproduce
Steps to reproduce the behavior:

1. /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin list
2. /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin remove securityDashboards
3. /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin list (shows up securityDashboards again)

Expected behavior
securityDashboards plugin to be removed

Logs from OpenStack Kolla-Ansible deployment using self-built OpenSearch images here: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d9d/915981/3/check/kolla-ansible-rocky9-prometheus-opensearch/d9d5f2a/primary/logs/container_logs/opensearch_dashboards.txt

OpenSearch Version
2.13

Dashboards Version
2.13

Plugins

Please list all plugins currently enabled.

Screenshots

If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• Rocky Linux 9.3, Ubuntu 22.04
• Browser and version [e.g. 22]

Additional context

Add any other context about the problem here.

[mnasiadka]

Actually tracked that down to wrong permissions in the RPM/DEB package - in 2.12 it was installed with opensearch-dashboards user permissions - now it's root:root all over the place

[ananzh]

@peterzhuamazon @bbarani could you help to check this?

@opensearch-project/admin could we redirect this the build team?

[peterzhuamazon]

RPM/DEB package need to use sudo for the full removal.
Without sudo it looks like it is removed but it is not.
Root/Root is an expected change in opensearch-project/opensearch-build#4034.

Thanks.

[mnasiadka]

When you use sudo, you get:
OpenSearch Dashboards should not be run as root. Use --allow-root to continue.

First of all - that change should be more visible in docs/release notes/changelog/whatever.
Second of all - the command should fail if it's unable to remove the plugin, shouldn't it?
Not counting the message that it should not be run as root...

[N0zz]

+1

Designated user can't install or remove plugins.

Running opensearch-dasboards-plugin install fails, due to insufficient permissions to $product_dir:

Plugin installation was unsuccessful due to error "EACCES: permission denied, mkdir '/usr/share/opensearch-dashboards/plugins/.plugin.installing'"

Running opensearch-dasboards-plugin remove goes succesfully, logs that plugin has been removed, but files are untouched and plugin is still present.

You can try to install/remove plugins as root, but it requires --allow-root to work, as mentioned.

I've compared deb packages for 2.11.1 and 2.14.0, there are no chowns for $product_dir in postinst of the latter package.

[rishabh6788]

@peterzhuamazon Is this something that needs to be fixed or documented properly?

[peterzhuamazon]

This is the behavior of OSD core, not build repo in the packaging.
Since @kavilla is already assigned here.
I will transfer the issue to OSD core. Thanks.