Loop enabling embedded ansible

[dbarreda]

I have a never ending loop after enabling embedded ansible. Also notification on event tab keeps popping up.

"The role Embedded Ansible has started activation on server manageiq"

[----] I, [2018-07-20T15:14:15.843446 #2027:abf8420] INFO -- : MIQ(EmbeddedAnsibleWorker#log_status) [Embedded Ansible Worker] Worker ID [106], PID [], GUID [b174d491-3d1b-4c4c-b9c3-5d29cf328137], Last Heartbeat [2018-07-20 19:13:29 UTC], Process Info: Memory Usage [], Memory Size [], Proportional Set Size: [], Unique Set Size: [], Memory % [], CPU Time [], CPU % [], Priority [] [----] E, [2018-07-20T15:14:15.843841 #2027:abf8420] ERROR -- : MIQ(EmbeddedAnsibleWorker::Runner) ID [106] PID [2027] GUID [b174d491-3d1b-4c4c-b9c3-5d29cf328137] {"detail":"A server error has occurred."} Worker exiting. [----] I, [2018-07-20T15:14:15.844067 #2027:abf8420] INFO -- : MIQ(EmbeddedAnsibleWorker#start_monitor_thread) SystemExit received, exiting monitoring Thread [----] I, [2018-07-20T15:14:18.115624 #2027:6df13c] INFO -- : MIQ(EmbeddedAnsibleWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [1] [----] I, [2018-07-20T15:14:18.263143 #2027:6df13c] INFO -- : MIQ(MiqQueue.put) Message id: [12563], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [107], PID [], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44]", :type=>"EmbeddedAnsibleWorker"}] [----] I, [2018-07-20T15:14:18.263327 #2027:6df13c] INFO -- : MIQ(EmbeddedAnsibleWorker#start) Worker started: ID [107], PID [], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44] [----] I, [2018-07-20T15:14:18.507480 #2027:b21ddc8] INFO -- : MIQ(EmbeddedAnsibleWorker::Runner#sync_config) ID [107], PID [2027], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44], Zone [default], Active Roles [automate,database_operations,database_owner,embedded_ansible,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Assigned Roles [automate,database_operations,database_owner,embedded_ansible,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Configuration: [----] I, [2018-07-20T15:14:18.999397 #2027:b21ddc8] INFO -- : MIQ(EmbeddedAnsibleWorker::Runner#setup_ansible) Starting embedded ansible service ... [----] I, [2018-07-20T15:14:20.600702 #2303:6df13c] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12563], MiqWorker id: [59], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [107], PID [], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44]", :type=>"EmbeddedAnsibleWorker"}], Dequeued in: [2.346169309] seconds [----] I, [2018-07-20T15:14:20.635308 #2303:6df13c] INFO -- : MIQ(MiqQueue.put) Message id: [12564], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [107], PID [], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44]", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>129, :miq_event_id=>129, "EventStream::event_stream"=>129, :event_stream_id=>129}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] [----] I, [2018-07-20T15:14:25.011276 #2332:6df13c] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12564], MiqWorker id: [62], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [107], PID [], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44]", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>129, :miq_event_id=>129, "EventStream::event_stream"=>129, :event_stream_id=>129}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [4.384605074] seconds [----] I, [2018-07-20T15:14:25.016690 #2332:6df13c] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [107], PID [], GUID [0ab45092-d908-48fd-8a4f-3c2c0f64bd44]", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>129, :miq_event_id=>129, "EventStream::event_stream"=>129, :event_stream_id=>129} for object [MiqServer.1] with state [] to Automate [----] I, [2018-07-20T15:14:35.629791 #2027:6df13c] INFO -- : MIQ(MiqServer#clean_worker_records) SQL Record for Worker [EmbeddedAnsibleWorker] with ID: [106], PID: [], GUID: [b174d491-3d1b-4c4c-b9c3-5d29cf328137], Status: [aborted] is being deleted [----] I, [2018-07-20T15:14:35.643251 #2027:6df13c] INFO -- : MIQ(MiqServer#worker_set_message) Worker [EmbeddedAnsibleWorker] with ID: [107], PID: [], GUID: [0ab45092-d908-48fd-8a4f-3c2c0f64bd44] is being requested to sync_config [----] I, [2018-07-20T15:14:42.535625 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:44.004189 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:45.451117 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:46.895276 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:48.299431 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:49.689219 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:51.253547 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:52.681922 #2027:b21ddc8] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond [----] I, [2018-07-20T15:14:59.618543 #2027:b21ddc8] INFO -- : MIQ(EmbeddedAnsibleWorker::Runner#setup_ansible) Finished starting embedded ansible service.

Let me know if I need to provide any other information that might be useful. This is basically the block of info that keeps showing over and over while doing "tail -f evm.log | grep -i "EmbeddedAnsible"

[skunkr]

Same issue here,
Logs from awx_task docker container are ending with the following error, and docker then keeps restarting all ansible-related containers

Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
“changed”: false,
“db”: “awx”
}
Operations to perform:
Apply all migrations: auth, conf, contenttypes, djcelery, main, oauth2_provide r, sessions, sites, social_django, sso, taggit
Running migrations:
No migrations to apply.
Your models have changes that are not yet reflected in a migration, and so won 't be applied.
Run ‘manage.py makemigrations’ to make new migrations, and then re-run ‘manage .py migrate’ to apply them.
Traceback (most recent call last):
File “/usr/bin/awx-manage”, line 9, in
load_entry_point(‘awx==1.0.7.2’, ‘console_scripts’, ‘awx-manage’)()
File “/usr/lib/python2.7/site-packages/awx/init.py”, line 116, in manage
execute_from_command_line(sys.argv)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management /init.py”, line 364, in execute_from_command_line
utility.execute()
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management /init.py”, line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management /base.py”, line 283, in run_from_argv
self.execute(*args, **cmd_options)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management /base.py”, line 330, in execute
output = self.handle(*args, **options)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management /commands/shell.py”, line 101, in handle
exec(sys.stdin.read())
File “”, line 1, in
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/contrib/auth/mo dels.py”, line 170, in create_superuser
return self._create_user(username, email, password, **extra_fields)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/contrib/auth/mo dels.py”, line 153, in _create_user
user.save(using=self._db)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/contrib/auth/ba se_user.py”, line 80, in save
super(AbstractBaseUser, self).save(*args, **kwargs)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base. py”, line 808, in save
force_update=force_update, update_fields=update_fields)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base. py”, line 838, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, upda te_fields)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base. py”, line 924, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base. py”, line 963, in _do_insert
using=using, raw=raw)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/manag er.py”, line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query .py”, line 1076, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/sql/c ompiler.py”, line 1112, in execute_sql
cursor.execute(sql, params)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/backends/uti ls.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/utils.py”, l ine 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/backends/uti ls.py”, line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint " auth_user_username_key"
DETAIL: Key (username)=(admin) already exists.

[PavelS1]

I have same issue ( clean manageiq-gaprindashvili-4 instal on gcp 4 cores , 15GB ram, just Git Repositories Owner and Embede Ansible role enabled and then went into endless loop)
docker containers starting over and over again

Clean up AWX DB and Ansible role didn`t help

Reproducible 100% - tested 3 times , symptoms seems same, docker logs contain same errors
(manual deploy of docker images for AWX - without enabling role on same machine works fine )

------------------------------------------- last few lines from evm.log ---
[----] I, [2018-08-20T03:22:50.568391 #11771:7795f48] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2018-08-20T03:22:57.623054 #11771:7795f48] INFO -- : MIQ(EmbeddedAnsibleWorker::Runner#setup_ansible) Finished starting embedded ansible service.
[----] E, [2018-08-20T03:22:59.801450 #11771:7795f48] ERROR -- : [AnsibleTowerClient::ClientError]: {"detail":"A server error has occurred."} Method:[block in method_missing]
[----] I, [2018-08-20T03:23:01.704116 #11771:7795f48] INFO -- : MIQ(EmbeddedAnsibleWorker#log_status) [Embedded Ansible Worker] Worker ID [18], PID [], GUID [c2d678be-044c-4cd0-bf2e-bb3525a6e9f6], Last Heartbeat [2018-08-20 07:22:19 UTC], Process Info: Memory Usage [], Memory Size [], Proportional Set Size: [], Unique Set Size: [], Memory % [], CPU Time [], CPU % [], Priority []
[----] E, [2018-08-20T03:23:01.704361 #11771:7795f48] ERROR -- : MIQ(EmbeddedAnsibleWorker::Runner) ID [18] PID [11771] GUID [c2d678be-044c-4cd0-bf2e-bb3525a6e9f6] {"detail":"A server error has occurred."} Worker exiting.

[skunkr]

Hi,
Any updates on this issue ? The last error in awx_task container seems to be :

django.db.utils.IntegrityError: duplicate key value violates unique constraint "auth_user_username_key"
DETAIL: Key (username)=(admin) already exists.

[skunkr]

Anybody interested about this situation ? I find very usefull the Embeded Ansible role, but just can't make it work.
Same error in latest version 5 :(

[tutunak]

yes in latest version I has some problem.

[skunkr]

Same problem i had on gaprindashvili-3. Actually i have installed 4, hoping the issue has be sorted out...

…

On Thu, Sep 20, 2018 at 2:52 PM, Dmitry ***@***.***> wrote: yes in latest version I has some problem. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#17738 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aea8GED6qz8FVDWHYmAo9Bz5HtIOGyVYks5uc4GPgaJpZM4VY_PZ> .

-- Best regards, Leo David

[dbarreda]

gaprindashvili-4 had the same issue for me.

Let me know if you have a different result.

[tjyang]

Just another me too.
Fresh Gapri-5 (standalone,non-regional) appliance has same issue for me also.
Linking to external AWX instance(using awx rpm installation approach) is ok.

[skunkr]

Same thing with G6. Is there any interest in solving this issue ?

[carbonin]

I think this was solved in the Hammer RC-1 build.

I believe one of the main fixes that allows this to work in Hammer is #17407 which has been requested for Gaprindashvili backport.

[Fryguy]

Closing this issue...if it's still an issue after the backport, please reopen.

[skunkr]

Thank you, So which version should I use at the moment that has aws properly working ? Hammer beta, or G6 ?

…

On Wed, Dec 19, 2018, 01:19 Jason Frey ***@***.*** wrote: Closed #17738 <#17738>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#17738 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aea8GI9ep9-4jaDjUkXsVlGXuPxa_q1tks5u6XgVgaJpZM4VY_PZ> .

[carbonin]

@Fryguy unfortunately it doesn't look like the fix was backported for Gap-6. So Hammer is still the only version with this fix. @simaishi is there any plan to backport #17407 to Gaprindashvili?

[simaishi]

@carbonin No plan to backport to Gaprindashvili currently... If this is really needed in Gaprindashvili branch, please get an approval from @dmetzger57

[dmetzger57]

@simaishi @carbonin given we have the fix in Hammer and where we are in the Gaprindashvili life cycle, I'm going to say no to back porting at this time (giving code base stability priority).

[skunkr]

Hi, I've just tested Manageiq 7 - the one available on oVirt image repo, and it seems the embedded ansible still not working. Anyone had a success with this release ? Maybe i am doing something wrong.. Thank you ! Leo

…

On Wed, Dec 19, 2018, 18:37 Dennis Metzger ***@***.*** wrote: @simaishi <https://github.com/simaishi> @carbonin <https://github.com/carbonin> given we have the fix in Hammer and where we are in the Gaprindashvili life cycle, I'm going to say no to back porting at this time (giving code base stability priority). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#17738 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aea8GGfbDzUw6oZOmsLYClMpF_FrPnlTks5u6mtggaJpZM4VY_PZ> .

[skunkr]

Hi Everyone, Still scratching my head trying to sort this out ..... On a fresh installed ManageIQ 7 , I get the following awx_task container error when enabling embeded ansible feature: django.db.utils.IntegrityError: duplicate key value violates unique constraint "auth_user_username_key" DETAIL: Key (username)=(admin) already exists. An organization is already in the system, exiting. The all four docker containers get restarted in an continuous never ending loop. Any thought what could be the problem ? Thank you very much ! Leo

…

On Tue, Jan 15, 2019 at 5:42 PM Leo David ***@***.***> wrote: Hi, I've just tested Manageiq 7 - the one available on oVirt image repo, and it seems the embedded ansible still not working. Anyone had a success with this release ? Maybe i am doing something wrong.. Thank you ! Leo On Wed, Dec 19, 2018, 18:37 Dennis Metzger ***@***.*** wrote: > @simaishi <https://github.com/simaishi> @carbonin > <https://github.com/carbonin> given we have the fix in Hammer and where > we are in the Gaprindashvili life cycle, I'm going to say no to back > porting at this time (giving code base stability priority). > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#17738 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/Aea8GGfbDzUw6oZOmsLYClMpF_FrPnlTks5u6mtggaJpZM4VY_PZ> > . >

-- Best regards, Leo David

[JPrause]

@skunkr can you open this as a new issue. Since this particular issue is closed, it's not going to get the visibility a new issue would receive. (in the new issue, link back to this one if it will help give background)

[simaishi]

@skunkr The fix for this issue is available in Hammer release, but not in Gaprindashvili releases. Please try with Hammer-1 build.

[JPrause]

@skunkr ignore my comment and follow @simaishi directions. 👍

[skunkr]

Thank you very much Sirs ! Installed Hammer-1 and discovered with real pleasure that embeded ansible tower is working. Thank you !

…

On Mon, Jan 21, 2019, 15:49 John Prause ***@***.*** wrote: @skunkr <https://github.com/skunkr> ignore my comment and follow @simaishi <https://github.com/simaishi> directions. 👍 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17738 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aea8GGjAdgbERV9vQo_gz8p8fkCgF86Vks5vFcWHgaJpZM4VY_PZ> .

[tjyang]

• verification the fix on hammer-1

[root@miqregion01 vmdb]# vmdb;cat VERSION;echo
hammer-1
[root@miqregion01 vmdb]#
[root@miqregion01 vmdb]# docker ps
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                                NAMES
f6d03b7458f6        ansible/awx_task:latest   "/tini -- /bin/sh ..."   15 minutes ago      Up 15 minutes       8052/tcp                             awx_task
f4cd04971bae        ansible/awx_web:latest    "/tini -- /bin/sh ..."   15 minutes ago      Up 15 minutes       0.0.0.0:54321->8052/tcp              awx_web
431a6e525e2d        memcached:alpine          "docker-entrypoint..."   16 minutes ago      Up 16 minutes       11211/tcp                            memcached
91399a94102a        rabbitmq:3                "docker-entrypoint..."   16 minutes ago      Up 16 minutes       4369/tcp, 5671-5672/tcp, 25672/tcp   rabbitmq
[root@miqregion01 vmdb]# date
Wed Jan 23 10:43:49 EST 2019
[root@miqregion01 vmdb]#

[JPrause]

@tjyang thanks for the verification!

[dbarreda]

Thanks all!

[SabrineHechmi]

@carbonin
I have the same issue with Hammer 3, a loop starting EmbeddedAnsibleWorker.
Any info, workaround on this issue please?

Thanks in advance for the help
[root@manageiq-standalone log]# tail -f log/evm.log | grep -i "EmbeddedAnsible"

[----] I, [2019-04-11T09:14:04.068399 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:05.370593 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] E, [2019-04-11T09:14:06.657841 #7030:d94f54] ERROR -- : MIQ(MiqServer#validate_worker) Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] has not responded in 1207.357519546 seconds, restarting worker
[----] I, [2019-04-11T09:14:06.667068 #7030:d94f54] INFO -- : MIQ(MiqQueue.put) Message id: [3499], id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_not_responding", {:event_details=>"Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] has not responded in 1207.357519546 seconds, restarting worker", :type=>"EmbeddedAnsibleWorker"}]
[----] I, [2019-04-11T09:14:06.667205 #7030:d94f54] INFO -- : MIQ(MiqServer#stop_worker) Stopping Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db], status [starting]...
----] I, [2019-04-11T09:14:06.674314 #7030:d94f54] INFO -- : MIQ(MiqQueue.put) Message id: [3500], id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_stop", {:event_details=>"Stopping Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db], status [starting]...", :type=>"EmbeddedAnsibleWorker"}]
[----] I, [2019-04-11T09:14:06.679777 #7030:d94f54] INFO -- : MIQ(MiqServer#worker_set_message) Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] is being requested to exit
[----] I, [2019-04-11T09:14:06.720765 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:07.271533 #7260:d94f54] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [3499], MiqWorker id: [13], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_not_responding", {:event_details=>"Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] has not responded in 1207.357519546 seconds, restarting worker", :type=>"EmbeddedAnsibleWorker"}], Dequeued in: [0.611309791] seconds
[----] I, [2019-04-11T09:14:07.306986 #7260:d94f54] INFO -- : MIQ(MiqQueue.put) Message id: [3501], id: [], Zone: [default], Role: [automate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_not_responding", :event_details=>"Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] has not responded in 1207.357519546 seconds, restarting worker", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>145, :miq_event_id=>145, "EventStream::event_stream"=>145, :event_stream_id=>145}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}]
[----] I, [2019-04-11T09:14:07.319582 #7260:d94f54] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [3500], MiqWorker id: [13], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_stop", {:event_details=>"Stopping Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db], status [starting]...", :type=>"EmbeddedAnsibleWorker"}], Dequeued in: [0.65009278] seconds
[----] I, [2019-04-11T09:14:07.346223 #7260:d94f54] INFO -- : MIQ(MiqQueue.put) Message id: [3502], id: [], Zone: [default], Role: [automate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_stop", :event_details=>"Stopping Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db], status [starting]...", :type=> EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>146, :miq_event_id=>146, "EventStream::event_stream"=>146, :event_stream_id=>146}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}]
[----] I, [2019-04-11T09:14:08.044294 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:09.388248 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:10.714920 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:12.028924 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:12.367364 #7268:d94f54] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [3501], MiqWorker id: [14], Zone: [default], Role: [automate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_not_responding", :event_details=>"Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] has not responded in 1207.357519546 seconds, restarting worker", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>145, :miq_event_id=>145, "EventStream::event_stream"=>145, :event_stream_id=>145}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [5.069795604] seconds
[----] I, [2019-04-11T09:14:12.374490 #7268:d94f54] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_not_responding", :event_details=>"Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db] has not responded in 1207.357519546 seconds, restarting worker", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>145, :miq_event_id=>145, "EventStream::event_stream"=>145, :event_stream_id=>145} for object [MiqServer.1] with state [] to Automate
[----] I, [2019-04-11T09:14:12.541708 #7286:d94f54] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [3502], MiqWorker id: [16], Zone: [default], Role: [automate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_stop", :event_details=>"Stopping Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db], status [starting]...", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>146, :miq_event_id=>146, "EventStream::event_stream"=>146, :event_stream_id=>146}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [5.203515758] seconds
[----] I, [2019-04-11T09:14:12.548379 #7286:d94f54] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_stop", :event_details=>"Stopping Worker [EmbeddedAnsibleWorker] with ID: [58], PID: [], GUID: [7b2e6ed5-6ce1-481d-9e7d-54de8f6834db], status [starting]...", :type=>"EmbeddedAnsibleWorker", "MiqEvent::miq_event"=>146, :miq_event_id=>146, "EventStream::event_stream"=>146, :event_stream_id=>146} for object [MiqServer.1] with state [] to Automate
[----] I, [2019-04-11T09:14:13.345811 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:14.678900 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:15.988600 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:17.337070 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:18.677098 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:20.014366 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:21.320780 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:22.663992 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:23.957751 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:25.278123 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:26.613098 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:27.914288 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:29.238253 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:30.581913 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:31.928422 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:33.228421 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:34.533896 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:35.847515 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond
[----] I, [2019-04-11T09:14:37.163820 #7030:47eb388] INFO -- : MIQ(DockerEmbeddedAnsible#start) Waiting for Ansible container to respond

[root@manageiq-standalone vmdb]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a077e4f53c3c ansible/awx_task:latest "/tini -- /bin/sh ..." 22 minutes ago Up 22 minutes 8052/tcp awx_task
013e7b6db16b ansible/awx_web:latest "/tini -- /bin/sh ..." 22 minutes ago Up 22 minutes 0.0.0.0:54321->8052/tcp awx_web
365bff724006 memcached:alpine "docker-entrypoint..." 22 minutes ago Up 22 minutes 11211/tcp memcached
4bda0eb42296 rabbitmq:3 "docker-entrypoint..." 22 minutes ago Up 22 minutes 4369/tcp, 5671-5672/tcp, 25672/tcp rabbitmq

[nikobo]

@SabrineHechmi
Look here : #18605