Integrate with external Tower Workflow

[jameswnl]

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1468339

Using this issue to track all related work

Backend

To inventory Tower Workflow objects to allow user launching them from within MIQ.

A Workflow in Tower

The following figure shows a Tower workflow_job_template

Native Tower models

• The boxes (Demo Job Template, billy etc.) are the workflow_job_template_nodes of the containing workflow_job_template
• Each workflow_job_template_nodes points to a job_template (Demo Job Template etc)
• The color of the links indicate the condition of launching that job_template and they are either success|failure|always

When this workflow_job_template is started, it will launch job_template named as Demo Job Template and then billy and then finally depending on outcome of billy, launch hello_template_with_survey on success or hello_template on failure.

Tower API output of a `workflow_job_template`

{
"id": 298,
"type": "workflow_job_template",
"url": "/api/v1/workflow_job_templates/298/",
"related": {
"created_by": "/api/v1/users/1/",
"modified_by": "/api/v1/users/1/",
"last_job": "/api/v1/workflow_jobs/420/",
"notification_templates_error": "/api/v1/workflow_job_templates/298/notification_templates_error/",
"notification_templates_success": "/api/v1/workflow_job_templates/298/notification_templates_success/",
"workflow_jobs": "/api/v1/workflow_job_templates/298/workflow_jobs/",
"notification_templates_any": "/api/v1/workflow_job_templates/298/notification_templates_any/",
"object_roles": "/api/v1/workflow_job_templates/298/object_roles/",
"access_list": "/api/v1/workflow_job_templates/298/access_list/",
"launch": "/api/v1/workflow_job_templates/298/launch/",
"labels": "/api/v1/workflow_job_templates/298/labels/",
"survey_spec": "/api/v1/workflow_job_templates/298/survey_spec/",
"schedules": "/api/v1/workflow_job_templates/298/schedules/",
"copy": "/api/v1/workflow_job_templates/298/copy/",
"activity_stream": "/api/v1/workflow_job_templates/298/activity_stream/",
"workflow_nodes": "/api/v1/workflow_job_templates/298/workflow_nodes/",
"organization": "/api/v1/organizations/1/"
},
"summary_fields": {
"last_job": {
"id": 420,
"name": "james-wf",
"description": "",
"finished": "2018-05-14T04:26:07.286Z",
"status": "successful",
"failed": false
},
"last_update": {
"id": 420,
"name": "james-wf",
"description": "",
"status": "successful",
"failed": false
},
"organization": {
"id": 1,
"name": "Default",
"description": ""
},
"created_by": {
"id": 1,
"username": "admin",
"first_name": "",
"last_name": ""
},
"modified_by": {
"id": 1,
"username": "admin",
"first_name": "",
"last_name": ""
},
"object_roles": {
"admin_role": {
"id": 4336,
"description": "Can manage all aspects of the workflow job template",
"name": "Admin"
},
"execute_role": {
"id": 4335,
"description": "May run the workflow job template",
"name": "Execute"
},
"read_role": {
"id": 4337,
"description": "May view settings for the workflow job template",
"name": "Read"
}
},
"user_capabilities": {
"edit": true,
"start": true,
"copy": true,
"schedule": true,
"delete": true
},
"labels": {
"count": 2,
"results": [
{
"id": 1,
"name": "dev"
},
{
"id": 2,
"name": "test"
}
]
},
"survey": {
"description": "",
"title": ""
},
"recent_jobs": [
{
"status": "successful",
"finished": "2018-05-14T04:26:07.286Z",
"id": 420
}
]
},
"created": "2018-05-14T04:24:19.594Z",
"modified": "2018-05-14T04:32:54.725Z",
"name": "james-wf",
"description": "",
"last_job_run": "2018-05-14T04:26:07.286600Z",
"last_job_failed": false,
"next_job_run": null,
"status": "successful",
"extra_vars": "",
"organization": 1,
"survey_enabled": true,
"allow_simultaneous": false
}

Tower API output of its `workflow_job_template_nodes`

{
"count": 4,
"next": null,
"previous": null,
"results": [
{
"id": 1,
"type": "workflow_job_template_node",
"url": "/api/v1/workflow_job_template_nodes/1/",
"related": {
"unified_job_template": "/api/v1/job_templates/216/",
"success_nodes": "/api/v1/workflow_job_template_nodes/1/success_nodes/",
"failure_nodes": "/api/v1/workflow_job_template_nodes/1/failure_nodes/",
"always_nodes": "/api/v1/workflow_job_template_nodes/1/always_nodes/",
"workflow_job_template": "/api/v1/workflow_job_templates/298/"
},
"summary_fields": {
"workflow_job_template": {
"id": 298,
"name": "james-wf",
"description": ""
},
"unified_job_template": {
"id": 216,
"name": "Demo Job Template",
"description": "",
"unified_job_type": "job"
}
},
"created": "2018-05-14T04:29:17.104Z",
"modified": "2018-05-14T04:29:17.104Z",
"unified_job_template": 216,
"success_nodes": [],
"failure_nodes": [],
"always_nodes": [
2
],
"inventory": null,
"credential": null,
"job_type": null,
"job_tags": null,
"skip_tags": null,
"limit": null,
"workflow_job_template": 298
},
{
"id": 2,
"type": "workflow_job_template_node",
"url": "/api/v1/workflow_job_template_nodes/2/",
"related": {
"unified_job_template": "/api/v1/job_templates/262/",
"success_nodes": "/api/v1/workflow_job_template_nodes/2/success_nodes/",
"failure_nodes": "/api/v1/workflow_job_template_nodes/2/failure_nodes/",
"always_nodes": "/api/v1/workflow_job_template_nodes/2/always_nodes/",
"workflow_job_template": "/api/v1/workflow_job_templates/298/"
},
"summary_fields": {
"workflow_job_template": {
"id": 298,
"name": "james-wf",
"description": ""
},
"unified_job_template": {
"id": 262,
"name": "billy",
"description": "billy",
"unified_job_type": "job"
}
},
"created": "2018-05-14T04:29:17.318Z",
"modified": "2018-05-29T19:30:05.398Z",
"unified_job_template": 262,
"success_nodes": [
7
],
"failure_nodes": [
3
],
"always_nodes": [],
"inventory": null,
"credential": null,
"job_type": null,
"job_tags": null,
"skip_tags": null,
"limit": null,
"workflow_job_template": 298
},
{
"id": 3,
"type": "workflow_job_template_node",
"url": "/api/v1/workflow_job_template_nodes/3/",
"related": {
"unified_job_template": "/api/v1/job_templates/341/",
"success_nodes": "/api/v1/workflow_job_template_nodes/3/success_nodes/",
"failure_nodes": "/api/v1/workflow_job_template_nodes/3/failure_nodes/",
"always_nodes": "/api/v1/workflow_job_template_nodes/3/always_nodes/",
"workflow_job_template": "/api/v1/workflow_job_templates/298/"
},
"summary_fields": {
"workflow_job_template": {
"id": 298,
"name": "james-wf",
"description": ""
},
"unified_job_template": {
"id": 341,
"name": "hello_template",
"description": "test job",
"unified_job_type": "job"
}
},
"created": "2018-05-14T04:29:17.426Z",
"modified": "2018-06-01T20:59:26.451Z",
"unified_job_template": 341,
"success_nodes": [],
"failure_nodes": [],
"always_nodes": [],
"inventory": null,
"credential": null,
"job_type": null,
"job_tags": null,
"skip_tags": null,
"limit": null,
"workflow_job_template": 298
},
{
"id": 7,
"type": "workflow_job_template_node",
"url": "/api/v1/workflow_job_template_nodes/7/",
"related": {
"unified_job_template": "/api/v1/job_templates/342/",
"success_nodes": "/api/v1/workflow_job_template_nodes/7/success_nodes/",
"failure_nodes": "/api/v1/workflow_job_template_nodes/7/failure_nodes/",
"always_nodes": "/api/v1/workflow_job_template_nodes/7/always_nodes/",
"workflow_job_template": "/api/v1/workflow_job_templates/298/"
},
"summary_fields": {
"workflow_job_template": {
"id": 298,
"name": "james-wf",
"description": ""
},
"unified_job_template": {
"id": 342,
"name": "hello_template_with_survey",
"description": "test job with survey spec",
"unified_job_type": "job"
}
},
"created": "2018-05-29T19:30:05.275Z",
"modified": "2018-05-29T19:30:05.275Z",
"unified_job_template": 342,
"success_nodes": [],
"failure_nodes": [],
"always_nodes": [],
"inventory": null,
"credential": null,
"job_type": null,
"job_tags": null,
"skip_tags": null,
"limit": null,
"workflow_job_template": 298
}
]
}

MIQ models and their Tower native counterparts

• (MIQ) workflow => (Tower native) workflow_job_template

PRs:
#17440
ManageIQ/manageiq-providers-ansible_tower#86
ansible/ansible_tower_client_ruby#104

The above PRs achieve the following:

1. A workflow would have the following attributes collected:
   • name
   • survey_spec
   • variables
   • workflow nodes
2. A workflow can be launched by a_workflow.run(params_json).

Frontend

@h-kataria

Automate (?)

@gmcculloug ?

[Fryguy]

I would leave the "job_template" bit out of the schema naming. We shouldn't use provider specific terminology in an agnostic modeling. I also don't really understand there being two "trees" of data (workflows and workflow nodes). Can you give a more complex example, and how it would fit into your model?

[Fryguy]

Additionally, why do we even need all of this information. The request is to just launch a workflow. I would think we'd only need the workflows themselves and not their component parts.

Also need to expose the job_templates inside it so that their survey_specs will be available for MIQ to create a dialog to obtain user inputs.

Can you elaborate a bit? Can the surveys just be part of the workflow defintion inventory?

[jameswnl]

@Fryguy I've updated the issue description. (The job_template isn't in MIQ models). See if the added description of the eg. workflow helps.

Inventorying the individual workflow_job_template_node because their corresponding job_templates can have their own survey_spec. Since these are defined by user through Tower UI and we have no way to force them into the workflow_job_template.

[jameswnl]

Talked to @Fryguy and we don't need the level of detail into workflow_job_template_nodes as of now.

[JPrause]

@jameswnl is this still an open issue? If not, can you close it.

[jameswnl]

Yep, we can close this.