The error was: KeyError: 'entry' on splunk 9.2.1

[yaroslav-nakonechnikov]

SH-Deployer fails to start:

RUNNING HANDLER [splunk_common : Restart the splunkd service - Via CLI] ********
changed: [localhost]
Tuesday 30 April 2024  12:18:29 +0000 (0:01:00.274)       0:02:09.934 *********
Tuesday 30 April 2024  12:18:29 +0000 (0:00:00.031)       0:02:09.966 *********
Tuesday 30 April 2024  12:18:29 +0000 (0:00:00.035)       0:02:10.002 *********
Tuesday 30 April 2024  12:18:29 +0000 (0:00:00.042)       0:02:10.044 *********
FAILED - RETRYING: Wait for SHC to be ready (60 retries left).
FAILED - RETRYING: Wait for SHC to be ready (59 retries left).
FAILED - RETRYING: Wait for SHC to be ready (58 retries left).
FAILED - RETRYING: Wait for SHC to be ready (57 retries left).
FAILED - RETRYING: Wait for SHC to be ready (56 retries left).
FAILED - RETRYING: Wait for SHC to be ready (55 retries left).
FAILED - RETRYING: Wait for SHC to be ready (54 retries left).
FAILED - RETRYING: Wait for SHC to be ready (53 retries left).
FAILED - RETRYING: Wait for SHC to be ready (52 retries left).
FAILED - RETRYING: Wait for SHC to be ready (51 retries left).
FAILED - RETRYING: Wait for SHC to be ready (50 retries left).
FAILED - RETRYING: Wait for SHC to be ready (49 retries left).
FAILED - RETRYING: Wait for SHC to be ready (48 retries left).
FAILED - RETRYING: Wait for SHC to be ready (47 retries left).
FAILED - RETRYING: Wait for SHC to be ready (46 retries left).
FAILED - RETRYING: Wait for SHC to be ready (45 retries left).
FAILED - RETRYING: Wait for SHC to be ready (44 retries left).
FAILED - RETRYING: Wait for SHC to be ready (43 retries left).
FAILED - RETRYING: Wait for SHC to be ready (42 retries left).
FAILED - RETRYING: Wait for SHC to be ready (41 retries left).
FAILED - RETRYING: Wait for SHC to be ready (40 retries left).
FAILED - RETRYING: Wait for SHC to be ready (39 retries left).
FAILED - RETRYING: Wait for SHC to be ready (38 retries left).
FAILED - RETRYING: Wait for SHC to be ready (37 retries left).
FAILED - RETRYING: Wait for SHC to be ready (36 retries left).
FAILED - RETRYING: Wait for SHC to be ready (35 retries left).
FAILED - RETRYING: Wait for SHC to be ready (34 retries left).
FAILED - RETRYING: Wait for SHC to be ready (33 retries left).
FAILED - RETRYING: Wait for SHC to be ready (32 retries left).
FAILED - RETRYING: Wait for SHC to be ready (31 retries left).
FAILED - RETRYING: Wait for SHC to be ready (30 retries left).
FAILED - RETRYING: Wait for SHC to be ready (29 retries left).
FAILED - RETRYING: Wait for SHC to be ready (28 retries left).
FAILED - RETRYING: Wait for SHC to be ready (27 retries left).
FAILED - RETRYING: Wait for SHC to be ready (26 retries left).
FAILED - RETRYING: Wait for SHC to be ready (25 retries left).
FAILED - RETRYING: Wait for SHC to be ready (24 retries left).
FAILED - RETRYING: Wait for SHC to be ready (23 retries left).
FAILED - RETRYING: Wait for SHC to be ready (22 retries left).
FAILED - RETRYING: Wait for SHC to be ready (21 retries left).
FAILED - RETRYING: Wait for SHC to be ready (20 retries left).
FAILED - RETRYING: Wait for SHC to be ready (19 retries left).
FAILED - RETRYING: Wait for SHC to be ready (18 retries left).
FAILED - RETRYING: Wait for SHC to be ready (17 retries left).
FAILED - RETRYING: Wait for SHC to be ready (16 retries left).
FAILED - RETRYING: Wait for SHC to be ready (15 retries left).
FAILED - RETRYING: Wait for SHC to be ready (14 retries left).
FAILED - RETRYING: Wait for SHC to be ready (13 retries left).
FAILED - RETRYING: Wait for SHC to be ready (12 retries left).
FAILED - RETRYING: Wait for SHC to be ready (11 retries left).
FAILED - RETRYING: Wait for SHC to be ready (10 retries left).
FAILED - RETRYING: Wait for SHC to be ready (9 retries left).
FAILED - RETRYING: Wait for SHC to be ready (8 retries left).
FAILED - RETRYING: Wait for SHC to be ready (7 retries left).
FAILED - RETRYING: Wait for SHC to be ready (6 retries left).
FAILED - RETRYING: Wait for SHC to be ready (5 retries left).
FAILED - RETRYING: Wait for SHC to be ready (4 retries left).
FAILED - RETRYING: Wait for SHC to be ready (3 retries left).
FAILED - RETRYING: Wait for SHC to be ready (2 retries left).
FAILED - RETRYING: Wait for SHC to be ready (1 retries left).

TASK [splunk_deployer : Wait for SHC to be ready] ******************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'entry'
fatal: [localhost]: FAILED! => {
    "attempts": 60,
    "changed": false,
    "rc": 1
}

MSG:

MODULE FAILURE
See stdout/stderr for the exact error


MODULE_STDERR:

Traceback (most recent call last):
  File "/home/splunk/.ansible/tmp/ansible-tmp-1714479900.636353-3244-117424306412885/AnsiballZ_shc_ready.py", line 100, in <module>
    _ansiballz_main()
  File "/home/splunk/.ansible/tmp/ansible-tmp-1714479900.636353-3244-117424306412885/AnsiballZ_shc_ready.py", line 92, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/splunk/.ansible/tmp/ansible-tmp-1714479900.636353-3244-117424306412885/AnsiballZ_shc_ready.py", line 41, in invoke_module
    run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_shc_ready_payload_pkmw08e6/ansible_shc_ready_payload.zip/ansible/modules/shc_ready.py", line 55, in <module>
  File "/tmp/ansible_shc_ready_payload_pkmw08e6/ansible_shc_ready_payload.zip/ansible/modules/shc_ready.py", line 50, in main
  File "/tmp/ansible_shc_ready_payload_pkmw08e6/ansible_shc_ready_payload.zip/ansible/modules/shc_ready.py", line 28, in run
KeyError: 'entry'


PLAY RECAP *********************************************************************
localhost                  : ok=122  changed=12   unreachable=0    failed=1    skipped=68   rescued=0    ignored=1

Tuesday 30 April 2024  12:25:01 +0000 (0:06:31.131)       0:08:41.176 *********
===============================================================================
splunk_deployer : Wait for SHC to be ready ---------------------------- 391.13s
splunk_common : Restart the splunkd service - Via CLI ------------------ 60.27s
splunk_common : Start Splunk via CLI ----------------------------------- 18.32s
splunk_common : Set options in roleMap_SAML ----------------------------- 6.60s
splunk_common : Get Splunk status --------------------------------------- 5.87s
splunk_common : Set options in saml ------------------------------------- 5.81s
splunk_common : Set options in role_admin ------------------------------- 1.39s
Gathering Facts --------------------------------------------------------- 1.13s
splunk_common : Set node as license slave ------------------------------- 1.02s
splunk_common : Setup indexer discovery for index-clustering ------------ 0.75s
splunk_deployer : Set deployer SHC key and label ------------------------ 0.74s
splunk_common : Cleanup Splunk runtime files ---------------------------- 0.72s
splunk_common : Check Splunk instance is running ------------------------ 0.72s
splunk_common : Set options in settings --------------------------------- 0.72s
splunk_common : Set options in authentication --------------------------- 0.70s
splunk_common : Setup default tcpout group for index-clustering --------- 0.70s
splunk_common : Set general pass4SymmKey -------------------------------- 0.52s
splunk_common : Apply admin password ------------------------------------ 0.51s
splunk_common : Find manifests ------------------------------------------ 0.49s
splunk_common : Update /opt/splunk/etc ---------------------------------- 0.47s

looks like failing there:
https://github.com/splunk/splunk-ansible/blob/9.2.1/roles/splunk_deployer/tasks/main.yml#L41
https://github.com/splunk/splunk-ansible/blob/9.2.1/ansible_commands/shc_ready.py

why? how to skip that?

[yaroslav-nakonechnikov]

after removing apps_location section from defaults.yml it skipps this task.