[BUG][Security Plugin] Can't upload configuration #96

ng-bsy · 2022-09-15T14:05:13Z

Describe the bug

ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml (No such file or directory)
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml (No such file or directory)
ERR: cannot upload configuration, see errors above

This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 10.10.1.77:9200 ... done
Connected as *****************"
OpenSearch Version: 2.3.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: *****
Clusterstate: GREEN
Number of nodes: 5
Number of data nodes: 5
.opendistro_security index already exists, so we do not need to create one.
Populate config from /usr/share/opensearch/plugins/opensearch-security/securityconfig/
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml (No such file or directory)
Will update '/roles' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/action_groups.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/action_groups.yml (No such file or directory)
Will update '/tenants' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml (No such file or directory)
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml (No such file or directory)
ERR: cannot upload configuration, see errors above

To Reproduce
Steps to reproduce the behavior:

Clone https://github.com/opensearch-project/ansible-playbook.git
Customize Variables in inventories/opensearch/hosts and inventories/opensearch/group_vars/all/all.yml
2.1 Fix various other problems with the Playbook, already mentioned in the Issues, which hinder the Playbook getting this far
Run ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become
Get stuck on Security Plugin configuration | Initialize the opensearch security index in opensearch

Expected behavior
I expect the Playbook to run successfully and not into errors.

Playbook Name
roles/linux/opensearch/tasks/security.yml

Host/Environment (please complete the following information):

Freshly installed linux systems
OpenSearch & Dashboards v2.3.0
Playbook a2a7f28

The text was updated successfully, but these errors were encountered:

ng-bsy · 2022-09-15T14:42:03Z

~~Can't~~ Finally can edit my own issues it seems...
Wanted to add, that it's a fresh installation of OpenSearch

ng-bsy · 2022-09-15T15:37:42Z

It seems the Playbook isn't supplying needed configuration files:
/usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml
/usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml

/usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml is also missing, but this seems by design, when not using auth_type 'oidc'

ng-bsy · 2022-09-15T16:06:04Z

Where does the securityconfig/config.yml come from, by the way, when not using auth_type 'oidc' ?

https://github.com/opensearch-project/ansible-playbook/blob/main/roles/linux/opensearch/tasks/security.yml#L149-L158

ng-bsy · 2022-09-15T16:09:53Z

This bug was introduced in 436136d

peterzhuamazon · 2022-09-29T18:43:59Z

@ssi444 ^ Would you mind provide some context to the issue @ng-bsy is facing due to the commit?
Thanks!

ng-bsy · 2022-09-30T07:53:37Z

@peterzhuamazon @ssi444 I had to apply the following change (reverting a part of mentioned commit) for the playbook to work, when configured with internal authentication:

I suppose there should be be added some kind of "else" clause to the "when ... 'oidc'..." to handle both configuration cases (internal & oidc)

ssi444 · 2022-09-30T08:00:31Z

@peterzhuamazon @ng-bsy

-cd {{os_sec_plugin_conf_path }} - applies all files
-f {{os_sec_plugin_conf_path }}/internal_users.yml - applies only 1 file

I checked on version 1.3, I haven't looked at version 2 yet. I'll see how the time is and I'll write.

ssi444 · 2022-09-30T15:19:11Z

Where does the securityconfig/config.yml come from, by the way, when not using auth_type 'oidc' ?

https://github.com/opensearch-project/ansible-playbook/blob/main/roles/linux/opensearch/tasks/security.yml#L149-L158

This file is in the OpenSearch package. With auth_type 'internal', this file is simply not overwritten and that's it

ssi444 · 2022-09-30T15:57:36Z

@peterzhuamazon @ng-bsy

In version 2.x, the security plugin files were moved from the {os_home}/plugins/opensearch-security/securityconfig directory to the {os_home}/config/opensearch-security directory. To work correctly, you need to change the value of the os_sec_plugin_conf_path variable in the inventories/opensearch/group_vars/all/all.yml file

I had this

os_home: /OS/opensearch
os_conf_dir: /OS/opensearch/config
os_plugin_bin_path: /OS/opensearch/bin/opensearch-plugin
os_sec_plugin_conf_path: /OS/opensearch/plugins/opensearch-security/securityconfig
os_sec_plugin_tools_path: /OS/opensearch/plugins/opensearch-security/tools

os_dashboards_home: /OS/opensearch-dashboards
os_dashboards_conf_dir: /OS/opensearch-dashboards/config
os_dashboards_plugin_bin_path: /OS/opensearch-dashboards/bin/opensearch-dashboards-plugin

It became so

os_home: /OS/opensearch
os_conf_dir: /OS/opensearch/config
os_plugin_bin_path: /OS/opensearch/bin/opensearch-plugin
os_sec_plugin_conf_path: /OS/opensearch/config/opensearch-security
os_sec_plugin_tools_path: /OS/opensearch/plugins/opensearch-security/tools

os_dashboards_home: /OS/opensearch-dashboards
os_dashboards_conf_dir: /OS/opensearch-dashboards/config
os_dashboards_plugin_bin_path: /OS/opensearch-dashboards/bin/opensearch-dashboards-plugin

ssi444 · 2022-09-30T16:02:56Z

Thus, I made only 3 changes to my configuration in order to deploy a cluster of version 2.3.0 instead of a cluster of version 1.3.2

os_version: "1.3.2" -> "2.3.0"
os_dashboards_version: "1.3.2" -> "2.3.0"
os_sec_plugin_conf_path: "/OS/opensearch/plugins/opensearch-security/securityconfig" -> "/OS/opensearch/configuration/opensearch-security"

ssi444 · 2022-09-30T16:44:10Z

There is another pitfall associated with --become. This task will also be performed with elevated privileges, but on the local computer and if this cannot be done, an error will occur.
I started connecting to the OS nodes by the root user and there was no problem, but if another user connects, for example, ubuntu, an error will occur.
If you add become: no to this task, then it does not help. We need to think about it...

- name: Security Plugin configuration | Check that the files/internal_users.yml exists
  stat:
    path: files/internal_users.yml
  register: custom_users_result
  delegate_to: localhost
  run_once: true

and

- name: Security Plugin configuration | Check for a custom configuration for internal users and hash passwords for them

ng-bsy · 2022-10-07T17:59:56Z

3. os_sec_plugin_conf_path: "/OS/opensearch/plugins/opensearch-security/securityconfig" -> "/OS/opensearch/configuration/opensearch-security"

@ssi444 This helped a lot, thanks!

Edit: Although I edited it in roles/linux/opensearch/defaults/main.yml

peterzhuamazon · 2022-10-07T18:02:29Z

Fixed in #80

ng-bsy added bug Something isn't working untriaged labels Sep 15, 2022

peterzhuamazon removed the untriaged label Sep 29, 2022

ng-bsy closed this as completed Oct 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG][Security Plugin] Can't upload configuration #96

[BUG][Security Plugin] Can't upload configuration #96

ng-bsy commented Sep 15, 2022 •

edited

Loading

ng-bsy commented Sep 15, 2022 •

edited

Loading

ng-bsy commented Sep 15, 2022

ng-bsy commented Sep 15, 2022

ng-bsy commented Sep 15, 2022

peterzhuamazon commented Sep 29, 2022

ng-bsy commented Sep 30, 2022

ssi444 commented Sep 30, 2022

ssi444 commented Sep 30, 2022

ssi444 commented Sep 30, 2022 •

edited

Loading

ssi444 commented Sep 30, 2022

ssi444 commented Sep 30, 2022

ng-bsy commented Oct 7, 2022 •

edited

Loading

peterzhuamazon commented Oct 7, 2022

[BUG][Security Plugin] Can't upload configuration #96

[BUG][Security Plugin] Can't upload configuration #96

Comments

ng-bsy commented Sep 15, 2022 • edited Loading

ng-bsy commented Sep 15, 2022 • edited Loading

ng-bsy commented Sep 15, 2022

ng-bsy commented Sep 15, 2022

ng-bsy commented Sep 15, 2022

peterzhuamazon commented Sep 29, 2022

ng-bsy commented Sep 30, 2022

ssi444 commented Sep 30, 2022

ssi444 commented Sep 30, 2022

ssi444 commented Sep 30, 2022 • edited Loading

ssi444 commented Sep 30, 2022

ssi444 commented Sep 30, 2022

ng-bsy commented Oct 7, 2022 • edited Loading

peterzhuamazon commented Oct 7, 2022

ng-bsy commented Sep 15, 2022 •

edited

Loading

ng-bsy commented Sep 15, 2022 •

edited

Loading

ssi444 commented Sep 30, 2022 •

edited

Loading

ng-bsy commented Oct 7, 2022 •

edited

Loading