ManageIq server not responding after initializing the server from the appliance_console

[W-A-S-A-Y]

Dear team

I've installed an ova file in my vSphere and configured as given in the documentation.

following the documentation step by step for first instance installation would be the way to go in order to reproduce this error.

3. Describe your environment, including
   Ova file with centos stream 8
   Environment
   For ManageIQ: VMware appliance

Here's a summary of the problem I'm facing:

Hostname: manageIQ-STG
IPv4 Address: 10.50.143.217/255.255.255.0
IPv4 Gateway: 10.50.143.1
IPv6 Address:
IPV6 Gateway:
Primary DNS: 10.50.12.11
Secondary DNS: 10.50.12.12
Search Order: jazz.com.pk
MAC Address: 00:50:56:a6:f7:48
Timezone: Asia/Karachi
Local Database Server: running (primary)
ManageIQ Server: not running
ManageIQ Database: 10.50.143.217
Database/Region: vmdb_production / 1
Messaging: configured
Local Messaging Broker: configured
External Auth: not configured
ManageIQ Version: quinteros-2.2

when I try to start the EVMSERVERD using systemd services or via appliance_console it starts the process and doesn't throw out any logs. I just feel like it gets lost.

here's the output of
[root@manageIQ-STG ~]# systemctl status evmserverd
● evmserverd.service - EVM server daemon
Loaded: loaded (/usr/lib/systemd/system/evmserverd.service; disabled; vendor preset: disabled)
Active: inactive (dead)

journalctl -u evmserverd
-- Logs begin at Tue 2025-02-04 17:41:59 PKT, end at Thu 2025-02-06 17:38:38 PKT. --
-- No entries --

appliance_console.log file's output:

Thu Feb 6 03:19:02 EST 2025 Checking for enough free space to rotate: /var/lib/pgsql/data/log/postgresql.log
Thu Feb 6 03:19:02 EST 2025 File size: 72
Thu Feb 6 03:19:02 EST 2025 Space required to rotate: 86
Thu Feb 6 03:19:02 EST 2025 Available space on volume: 51913012
Thu Feb 6 03:19:02 EST 2025 Rotating file...
I, [2025-02-06T06:59:46.535890 #19674] INFO -- : EVM server start initiated by appliance console.
I, [2025-02-06T17:26:31.693992 #24503] INFO -- : EVM server start initiated by appliance console.

kindly assist...

[Fryguy]

@bdunne Can you help out here?

[W-A-S-A-Y]

Hey guys just for the update. I'm still stuck with this screen. it's not loading the applications and throwing an error

really need ManageIQ to perform in this POC.

please find the attached images for reference.

[agrare]

@W-A-S-A-Y can you try checking the two pre-start services that check that the database and kafka are running properly?

systemctl status manageiq-db-ready.service
systemctl status manageiq-messaging-ready.service

In my case:

[root@manageiq ~]# systemctl status manageiq-db-ready.service
○ manageiq-db-ready.service - ManageIQ DB Ready
     Loaded: loaded (/usr/lib/systemd/system/manageiq-db-ready.service; disabled; preset: disabled)
     Active: inactive (dead) since Wed 2025-02-26 10:22:01 EST; 28s ago
    Process: 1014 ExecStart=/usr/bin/manageiq-db-ready (code=exited, status=0/SUCCESS)
   Main PID: 1014 (code=exited, status=0/SUCCESS)
        CPU: 653ms

Feb 26 10:21:58 manageiq systemd[1]: Starting ManageIQ DB Ready...
Feb 26 10:22:00 manageiq manageiq-db-ready[1014]: localhost  - accepting connections
Feb 26 10:22:01 manageiq manageiq-db-ready[1014]: vmdb_production is up and running
Feb 26 10:22:01 manageiq systemd[1]: manageiq-db-ready.service: Deactivated successfully.
Feb 26 10:22:01 manageiq systemd[1]: Finished ManageIQ DB Ready.
[root@manageiq ~]# systemctl status manageiq-messaging-ready.service
○ manageiq-messaging-ready.service - ManageIQ Messaging Ready
     Loaded: loaded (/usr/lib/systemd/system/manageiq-messaging-ready.service; disabled; preset: disabled)
     Active: inactive (dead) since Wed 2025-02-26 10:22:25 EST; 11s ago
    Process: 1018 ExecStart=/usr/bin/manageiq-messaging-ready (code=exited, status=0/SUCCESS)
   Main PID: 1018 (code=exited, status=0/SUCCESS)
        CPU: 26.963s

Feb 26 10:22:18 manageiq manageiq-messaging-ready[3387]: [2025-02-26 10:22:18,636] ERROR org.apache.kafka.common.errors.TopicExistsException: Topic 'manageiq.ems-events' already exists.
Feb 26 10:22:18 manageiq manageiq-messaging-ready[3387]:  (org.apache.kafka.tools.TopicCommand)
Feb 26 10:22:22 manageiq manageiq-messaging-ready[3863]: [2025-02-26 10:22:22,116] ERROR org.apache.kafka.common.errors.TopicExistsException: Topic 'manageiq.ems-inventory' already exists.
Feb 26 10:22:22 manageiq manageiq-messaging-ready[3863]:  (org.apache.kafka.tools.TopicCommand)
Feb 26 10:22:25 manageiq manageiq-messaging-ready[4338]: [2025-02-26 10:22:25,580] ERROR org.apache.kafka.common.errors.TopicExistsException: Topic 'manageiq.metrics' already exists.
Feb 26 10:22:25 manageiq. manageiq-messaging-ready[4338]:  (org.apache.kafka.tools.TopicCommand)
Feb 26 10:22:25 manageiq. manageiq-messaging-ready[1018]: Topics initialized successfully
Feb 26 10:22:25 manageiq. systemd[1]: manageiq-messaging-ready.service: Deactivated successfully.
Feb 26 10:22:25 manageiq. systemd[1]: Finished ManageIQ Messaging Ready.
Feb 26 10:22:25 manageiq. systemd[1]: manageiq-messaging-ready.service: Consumed 26.963s CPU time.
[root@manageiq ~]# systemctl status evmserverd
● evmserverd.service - EVM server daemon
     Loaded: loaded (/usr/lib/systemd/system/evmserverd.service; enabled; preset: disabled)
     Active: active (running) since Wed 2025-02-26 10:22:25 EST; 16s ago
   Main PID: 4814 (ruby)
      Tasks: 3 (limit: 100422)
     Memory: 222.9M
        CPU: 11.344s
     CGroup: /system.slice/evmserverd.service
             └─4814 /usr/bin/ruby lib/workers/bin/evm_server.rb

Feb 26 10:22:36 manageiq. api[4814]:  INFO -- api:   description             : REST API
Feb 26 10:22:36 manageiq. api[4814]:  INFO -- api:
Feb 26 10:22:36 manageiq. api[4814]:  INFO -- api: Dynamic Configuration
Feb 26 10:22:36 manageiq. api[4814]:  INFO -- api:   token_ttl               : 10.minutes
Feb 26 10:22:36 manageiq. api[4814]:  INFO -- api:   authentication_timeout  : 30.seconds
Feb 26 10:22:36 manageiq. api[4814]:  INFO -- api:   max_results_per_page    : 1000
Feb 26 10:22:36 manageiq. evm[4814]:  INFO -- evm: MIQ(Vmdb::Initializer.init) Initializing Application: Program Name: lib/workers/bin/evm_server.rb, PID: 4814, ENV['EVMSERVER']: true

[W-A-S-A-Y]

please let me know if this is the right configuration.
while I try to check the status of kafka messaging service. it does not work. the DB is running fine though.

However I always get stuck on this message and it does not go on.

Is there a particular way to configure both of application services or if there's an Ova template that comes with these things configured please let me know. I just need to proceed with installations of Infrastructure providers of Openshift, HyperV, and Vmware Vsphere.

so if you guys could share some exact documentation for that. it would be really helpful.

[agrare]

@W-A-S-A-Y what do you get when you run those three systemctl commands? Can you paste the output? Also can you check journalctl -t evm?

[W-A-S-A-Y]

log.txt

it's a pretty large file when I try journalctl -t evm
I've attached the whole log file for your assistance in case you want to see something specific from that output.

[agrare]

Yeah so it looks like it isn't able to connect to your kafka broker. It can't even What does systemctl status kafka.service show you and do you see anything in the kafka logs (/opt/kafka/logs/server.log)?