Enable the system admin to provide complex index settings for Elasticsearch.

[svenoe]

@io-architect provided us with a pull request greatly enhancing the ability to customize Elasticsearch. If sensible we should provide one or two more options to adapt stuff in a simpler form via the SysConfig. (#938)

Please use the branch issue-#961-elasticsearch_index for further development.

[kess-net]

Tested and enhanced a little bit. Branch issue-#961-elasticsearch_index looks is looking good

[kess-net]

merged to rel-10_0

[bschmalhofer]

Reopen for fixing scripts/test/Config/Defaults.t .
@kess-net : Could you take a look at the test? It looks like there are divergent settins in Defaults.pm and Framework.xml.

[kess-net]

@bschmalhofer , this test is giving me an 'All tests successful.'. What is the problem at your side?

[bschmalhofer]

I got a:

Failed test 'Elasticsearch::ArticleIndexCreationSettings must be the same in Defaults.pm and and setting default value (is not equal, see below)

Diff:
+--+------------------------------+--+------------------+
| |Actual data | |Expected data |
+--+------------------------------+--+------------------+
| 1|$VAR1 = { | 1|$VAR1 = { |

• 2| 'FieldsLimit' => '2000', * | |
  | 3| 'NR' => '0', | 2| 'NR' => '0', |
  | 4| 'NS' => '1' | 3| 'NS' => '1' |
  | 5| }; | 4| }; |
  +--+------------------------------+--+------------------+

Actual data:
$VAR1 = {
'FieldsLimit' => '2000',
'NR' => '0',
'NS' => '1'
};

Expected data:
$VAR1 = {
'NR' => '0',
'NS' => '1'
};

'

at scripts/test/Config/Defaults.t line 194.

after I did a bin/otobo.Console.pl Maint::Config::Rebuild.

Maybe it's an effect of using bin/docker/quick_setup.pl instead of using installer.pl.

[bschmalhofer]

I have looked at that in the rel-10_0 branch. IMHO the test is correct in reporting a failure, as the config for Elasticsearch::ArticleIndexCreationSettings are indeed diverging between Defaults.pm and Framework.xml. The deeper issue is that this is a necessary divergence. Installer.pm expects a global config for all indexes while the command Maint::ElasticSearch::Migration expects an index specific config, with the default config for 'all'.

I propose that one of these altternatives is implemented:

1. bring the config for Installer.pm in line with Maint::ElasticSearch::Migration
2. same as 1., buta visa versa
3. keep the configs apart, but then at least give them distinctive names

For the record, here is the error from Defaults.pm:

ERROR: OTOBO-otobo.UnitTest-10 Perl: 5.32.0 OS: linux Time: Mon Jul 12 12:52:36 2021

Message: Setting Elasticsearch::ArticleIndexCreationSettings is invalid!

Traceback (3939):
Module: Kernel::System::SysConfig::SettingGet Line: 200
Module: /opt/otobo/scripts/test/Config/Defaults.t Line: 185

Failed test 'Elasticsearch::ArticleIndexCreationSettings must be the same in Defaults.pm and and setting default value (is not equal, see below)

Diff:
+--+------------------------------+--+-----------------+
| |Actual data | |Expected data |
+--+------------------------------+--+-----------------+

• 1|$VAR1 = { * 1|$VAR1 = \undef; *
• 2| 'FieldsLimit' => '2000', * | |
• 3| 'NR' => '0', * | |
• 4| 'NS' => '1' * | |
• 5| }; * | |
  +--+------------------------------+--+-----------------+

Actual data:
$VAR1 = {
'FieldsLimit' => '2000',
'NR' => '0',
'NS' => '1'
};

Expected data:
$VAR1 = \undef;

'

at /opt/otobo/scripts/test/Config/Defaults.t line 193.

[svenoe]

It has to be option 1., and it should be noted that adaptions to the index can be made in Config.pm before running the installer.

[bschmalhofer]

Another tidbit: mayber rename 'all' to something like 'default' or 'standard'. The config is no longer for all indexes when there is a specific config.

[bschmalhofer]

The commit b0c99a8 by @kess-net did not make it into rel-10_0. This commit was moving the SysConfix XML snippet for Elasticsearch::IndexSettings###ConfigItem into the ITSMConfigurationManagement package. Apparently it was decided to keep it in OTOBO core.

[bschmalhofer]

There is also an old branch by @kess-net , https://github.com/RotherOSS/otobo/tree/issue-%23961-development_by_kess. This looks like useful changes, but nothing that is essential. Let's keep that branch for now.