GuideDeploymentManagement
One of the most useful services provided by the Overlord: DTGov system is the Deployment Service. This is a service that makes it possible to deploy a binary artifact stored in the S-RAMP repository into a target runtime environment such as JBoss EAP. This Deployment Service can easily be invoked from a Governance Workflow and is often included as part of a Deployment Lifecycle business process.
Invoking the Deployment Service from a Governance Workflow should be a simple matter of using the HttpClientDeploy task defined in the HttpClientWorkDefinitions.wid file as described in the Governance Workflows chapter of this guide. Within a BPMN2 process, the XML markup might look something like this:
<bpmn2:task id="Task_1" drools:taskName="HttpClientDeploy" drools:displayName="Deploy" drools:icon="run-build-install.png" name="Deploy to DEV">
<bpmn2:incoming>bpmn20:SequenceFlow_9</bpmn2:incoming>
<bpmn2:outgoing>bpmn20:SequenceFlow_10</bpmn2:outgoing>
<bpmn2:ioSpecification id="_InputOutputSpecification_10">
<bpmn2:dataInput id="_DataInput_47" itemSubjectRef="__NameInputItem" name="Url"/>
<bpmn2:dataInput id="_DataInput_48" itemSubjectRef="__NameInputItem" name="Method"/>
<bpmn2:dataInput id="_DataInput_49" itemSubjectRef="__NameInputItem" name="Uuid"/>
<bpmn2:dataInput id="_DataInput_50" itemSubjectRef="__NameInputItem" name="Target"/>
<bpmn2:inputSet id="_InputSet_10" name="Input Set 10">
<bpmn2:dataInputRefs>_DataInput_47</bpmn2:dataInputRefs>
<bpmn2:dataInputRefs>_DataInput_48</bpmn2:dataInputRefs>
<bpmn2:dataInputRefs>_DataInput_49</bpmn2:dataInputRefs>
<bpmn2:dataInputRefs>_DataInput_50</bpmn2:dataInputRefs>
</bpmn2:inputSet>
</bpmn2:ioSpecification>
<bpmn2:dataInputAssociation id="_DataInputAssociation_47">
<bpmn2:sourceRef>DeploymentUrl</bpmn2:sourceRef>
<bpmn2:targetRef>_DataInput_47</bpmn2:targetRef>
</bpmn2:dataInputAssociation>
<bpmn2:dataInputAssociation id="_DataInputAssociation_48">
<bpmn2:targetRef>_DataInput_48</bpmn2:targetRef>
<bpmn2:assignment id="Assignment_1">
<bpmn2:from xsi:type="bpmn2:tFormalExpression" id="FormalExpression_16">POST</bpmn2:from>
<bpmn2:to xsi:type="bpmn2:tFormalExpression" id="FormalExpression_17">_DataInput_48</bpmn2:to>
</bpmn2:assignment>
</bpmn2:dataInputAssociation>
<bpmn2:dataInputAssociation id="_DataInputAssociation_49">
<bpmn2:sourceRef>ArtifactUuid</bpmn2:sourceRef>
<bpmn2:targetRef>_DataInput_49</bpmn2:targetRef>
</bpmn2:dataInputAssociation>
<bpmn2:dataInputAssociation id="_DataInputAssociation_50">
<bpmn2:targetRef>_DataInput_50</bpmn2:targetRef>
<bpmn2:assignment id="Assignment_14">
<bpmn2:from xsi:type="bpmn2:tFormalExpression" id="FormalExpression_17">dev</bpmn2:from>
<bpmn2:to xsi:type="bpmn2:tFormalExpression" id="FormalExpression_18">_DataInput_50</bpmn2:to>
</bpmn2:assignment>
</bpmn2:dataInputAssociation>
</bpmn2:task>
The above task uses the DeploymentUrl and ArtifactUuid parameters that were passed in to the business process when it was invoked. It populates the inputs required by HttpClientDeploy including an input parameter named Target. The Target parameter maps to a configured Deployment Target. The target is a logical name and corresponds to a physical runtime environment configured in the DTGov configuration file (dtgov.properties). See the next section for details.
In order to make logical Deployment Targets available they must be configured in the DTGov configuration file. Typically an organization would configure three or four Deployment Targets, including:
-
dev - the development environment
-
qa - the test environment
-
stage - the staging environment
-
prod - the final production environment
Of course, any number of targets can be configured. Here is an example of how to configure the above four targets in the DTGov configuration file:
governance.targets= dev|http://www.jboss.org/overlord/deployment-status.owl#InDev|copy|/tmp/dev/jbossas7/standalone/deployments governance.targets= qa|http://www.jboss.org/overlord/deployment-status.owl#InQa|copy|/tmp/qa/jbossas7/standalone/deployments governance.targets=stage|http://www.jboss.org/overlord/deployment-status.owl#InStage|copy|/tmp/stage/jbossas7/standalone/deployments governance.targets= prod|http://www.jboss.org/overlord/deployment-status.owl#InProd|copy|/tmp/prod/jbossas7/standalone/deployments
The format of each target is as follows:
LogicalName|Classifier|DeploymentType|TypeSpecificParams
-
'LogicalName' : used in the BPMN process as described in the previous section as the value of Target
-
'Classifier' : a classifier that should get added to the binary deployment artifact when deployed to the target environment (and removed when undeployed)
-
'DeploymentType' : how to deploy to the environment. Valid values as of this writing include copy, rhq, as_cli, maven
Depending on the type of the deployment, additional parameters may be required. In the example above, the 'copy' deployment type requires a folder on the server, which is where it will copy the deployment artifact.
Here are some examples of how to use the other deployment types:
# Deploy using RHQ/JON (username, password, rhq url) governance.targets=rhq|rhquser::rhqpassword::rhqbaseUrl # Deploy using the JBoss AS CLI (username, password, host, port) governance.targets=as_cli|asuser::aspassword::ashost::asport # Deploy to a Maven Repository (maven-url, is-release-enabled, is-snapshot-enabled governance.targets=maven|scp://m2.example.com/m2/snapshot-repository::false::true
Whenever the Deployment Service is used to deploy an artifact from the repository, it also annotates that artifact with relevant undeployment information. This annotation takes the form of another artifact in the repository of type ext/UndeploymentInformation. The annotation artifact will have a relationship named describesDeployment pointing from it back to the deployment artifact it annotates.
This undeployment information is used whenever an artifact needs to be undeployed. Undeploy of an artifact happens when a new version of that artifact is being deployed to a particular environment (deployment target). When this happens, the old version (whichever version is currently deployed in that environment) is undeployed in preparation of the new deployment.
Once the artifact is undeployed, its undeployment information artifact is deleted from the repository and any relevant classifier associated with the target environment is removed from the deployment artifact.
Note: please see the Configuring DTGov chapter for information about how to coordinate the configuration of the Deployment Service with the configuration of the DTGov User Interface (the Deployment Management UI).
For a short video see: http://jboss-overlord.blogspot.com/2013/11/managed-deployments-using-dtgov.html
This demo shows how DTGov can start a Project Workflow via a simple Maven build. This demo contains two projects:
-
project: a very simple maven module with artifactId of "project".
-
service: a maven module containing the Hello World Service API, expressed in WSDL.
The goal is to show that an upload of a pom.xml with artifactId of "project" kicks of a workflow of type overlord.demo.SimpleReleaseProcess.png.
To receive email notifications you need a running email server. The connection settings can be provided in the dtgov.properties file. It uses the mail-smtp settings defined in the standalone/configuration/standalone.xml
<outbound-socket-binding name="mail-smtp"> <remote-destination host="localhost" port="25"/> </outbound-socket-binding>
The easiest way to fulfill this requirements is to run the FakeSMTP server (http://nilhcem.github.io/FakeSMTP/), since this will list emails send to all email addresses. Just download it and run
sudo java -jar fakeSMTP-1.0.jar
This demo uses the roles dev, qa, stage and prod. By default the admin user has all of these roles, see the standalone/configuration/application_roles.properties
admin=overlorduser,admin.sramp,dev,qa,stage,prod
which means that the admin will see tasks for all roles in this example. Follow the instructions in the application-users.properties to add users. Note that a new new user needs at least the overlorduser role.
Be aware that you must supply the maven build with credentials for your S-RAMP repository. This can be done by adding a section to your settings.xml file (or by providing a custom settings.xml on the mvn command line using the '-s' argument).
For more information see: http://maven.apache.org/settings.html
Your settings.xml file should contain two additional server entries in the servers section:
<server>
<id>local-sramp-repo</id>
<username>admin</username>
<password>YOUR-PASSWORD</password>
</server>
<server>
<id>local-sramp-repo-snapshots</id>
<username>admin</username>
<password>YOUR-PASSWORD</password>
</server>
To get this demo working you must be running the DTGov server and the S-RAMP repository (see documentation for the DTGov and S-RAMP projects to learn how to run the S-RAMP repository).
The first step is to build and deploy all artifacts to S-RAMP using Maven. The demo is configured to run against the default security settings found in the distribution.
|
Important
|
Authentication settings in your .m2/settings.xml are required. |
$ mvn -Pdemo clean deploy
The -Pdemo flag will enable the 'demo' profile, which tells the Maven distributionManagement to point to your local S-RAMP repository (http://localhost:8080/s-ramp-server/). Therefore you need to be running S-RAMP on port 8080 and deployed as the 's-ramp-atom' context.
The build should complete successfully and on the server you should see logging along the lines of
09:04:10,929 INFO [org.overlord.sramp.governance.QueryExecutor] (EJB default - 5) Starting workflow overlord.demo.SimpleReleaseProcess for artifact 44021610-f85e-48bf-9a1c-9adcdbe485b6
At this point there should be a number of artifacts stored in the S-RAMP repository. You can verify that by deploying the "s-ramp-ui" project and then navigating to http://localhost:8080/s-ramp-ui (or the appropriate URL for you).
You should see the following primary artifacts in the S-RAMP repository:
-
dtgov-demos-switchyard-VERSION.jar
-
dtgov-demos-switchyard-VERSION.pom
-
OrderService.wsdl
-
beans.xml
-
orders
-
org.overlord.sramp.demos.switchyard.service.InventoryService
-
org.overlord.sramp.demos.switchyard.service.InventoryServiceBean
-
org.overlord.sramp.demos.switchyard.service.Order
-
org.overlord.sramp.demos.switchyard.service.OrderAck
-
org.overlord.sramp.demos.switchyard.service.OrderService
-
org.overlord.sramp.demos.switchyard.service.OrderServiceBean
-
org.overlord.sramp.demos.switchyard.service.Transformers
Where the OrderService.wsdl, beans.xml, orders as well as some class With the query defined in the dtgov.properties:
governance.queries=/s-ramp/ext/SwitchYardApplication|overlord.demo.SimpleReleaseProcess|DeploymentUrl={governance.url}/rest/deploy/{target}/{uuid}::NotificationUrl={governance.url}/rest/notify/email/{group}/deployed/{target}/{uuid}::UpdateMetaDataUrl={governance.url}/rest/update/{name}/{value}/{uuid}....
it will start a SimpleReleaseProcess workflow when a SwitchYardApplication lands in the repository. You should see a custom property getting created which should look similar to
workflowProcessId=overlord.demo.SimpleReleaseProcess_0:workflowParameters=DeploymentUrl=http://localhost:8080/dtgov/res...
Note that the definition of the 'overlord.demo.SimpleReleaseProcess' lives in the SRAMPPackage in the dtgov-workflows-VERSION.jar. If you want to update the workflow you need to use eclipse tooling to modify the bpmn, build and upload a new dtgov-worflow.jar to S-RAMP. The GAV settings are defined in the dtgov.properties file.
The workflow (and the deployment ontology) specifies four different environments:
-
dev - development: machine hosting deployed released artifact before they go to QA. Developers can do a quick test to make sure things work on more then just their desk.
-
qa - quality assurance: machine hosting deployed released artifacts so that they can go through the testing process.
-
stage - staging: an environment identical to production where qa’ed artifacts can be tested on the real hardware and with interactions with other systems.
-
prod - production: the final place where the artifacts are deployed and run
When the _SimpleReleaseProcess_ is instantiated it deploy the artifact to the _dev_ environment. The workflow makes a POST call to DeploymentUrl={governance.url}/rest/deploy/{target}/{uuid} where * {governance.url} is location where the DTGov REST API is hosted; this defaults to 'http://localhost:8080/dtgov' and can be overridden in the dtgov.properties. * {target} is the name of the deployment target which defined in the dtgov.properties and is referenced in the 'Deploy to Dev' task. * {uuid} is the UUID of the artifact which is set as a process parameter in the _SimpleReleaseProcess_ instance at creation time.
In this case, we assume the dev target is defined as
governance.targets= dev|http://www.jboss.org/overlord/deployment-status.owl#InDev|copy|/tmp/dev/jbossas7/standalone/deployments
where * dev: name of the target * http://www.jboss.org/overlord/deployment-status.owl#InDev: classification when deployed to Dev * copy: use file copy * /tmp/dev/jbossas7/standalone/deployments: deploy directory
We assume there is jbossas7 server running in /tmp/dev/jbossas7, and thus it uses a simple file copy to place the artifact in /tmp/dev/jbossas7/standalone/deployments. The appserver will take of deploying the artifact and on the server we should see logging along the lines of
09:04:11,168 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (EJB default - 5) Calling POST TO: http://localhost:8080/dtgov/rest/deploy/dev/44021610-f85e-48bf-9a1c-9adcdbe485b6
09:04:11,274 INFO [org.jboss.resteasy.cdi.CdiInjectorFactory] (http-/127.0.0.1:8080-13) Found BeanManager at java:comp/BeanManager
09:04:11,300 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (http-/127.0.0.1:8080-13) Deploying javax.ws.rs.core.Application: class org.overlord.sramp.governance.services.GovernanceApplication$Proxy$_$$_WeldClientProxy
09:04:12,170 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (EJB default - 5) reply={status=success, target=COPY:/tmp/dev/jbossas7/standalone/deployments/dtgov-demos-switchyard-2.0.0-SNAPSHOT.jar}
The next task Classify #DevTest calls a REST service in DTGov using endpoint using a PUT to:
UpdateMetaDataUrl={governance.url}/rest/update/{name}/{value}/{uuid}
where * {governance.url} is location where the DTGov REST API is hosted; this defaults to 'http://localhost:8080/dtgov' and can be overridden in the dtgov.properties. * {name} is the type which is 'classification' in this case. This is set in the task. * {value} is the value of the classification which is http://www.jboss.org/overlord/deployment-status.owl#DevTest * {uuid} is the UUID of the artifact which is set as a process parameter in the SimpleReleaseProcess instance at creation time.
which adds the #DevTest classification onto the artifact. You can verify by navigating to this details of this artifact in the s-ramp-ui or by using the s-ramp.sh cli. The logging on the server should read
09:04:12,202 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (EJB default - 5) Calling PUT TO: http://localhost:8080/dtgov/rest/update/classification/http%3A*2F*2Fwww.jboss.org*2Foverlord*2Fdeployment-status.owl%23DevTest/44021610-f85e-48bf-9a1c-9adcdbe485b6
09:04:12,414 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (EJB default - 5) reply={artifactName=dtgov-demos-switchyard-2.0.0-20131107.140403-1.jar, artifactCreatedBy=admin, status=success}
The next task in the SimpleReleaseProcess is an email notification. The "Notification Task" calls a REST service in DTGov using a POST to:
NotificationUrl={governance.url}/rest/notify/email/{group}/deployed/{target}/{uuid}
where * {governance.url} is location where the DTGov REST API is hosted; this defaults to 'http://localhost:8080/dtgov' and can be overridden in the dtgov.properties. * {group} is name of the group to which the notification will be send. This is set in the task and is is set to dev the first go-around. * deployed is the name of the notification template. * {target} is the name of the deployment target which defined in the dtgov.properties and is referenced in the 'Deploy to Dev' task. This info is construct the notification message. * {uuid} is the UUID of the artifact which is set as a process parameter in the SimpleReleaseProcess instance at creation time.
On the server we should see the following logging
09:04:12,419 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (EJB default - 5) Calling POST TO: http://localhost:8080/dtgov/rest/notify/email/dev/deployed/dev/44021610-f85e-48bf-9a1c-9adcdbe485b6
09:04:12,862 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (EJB default - 5) reply={status=success}
By default an email is sent the server localhost at port 25. By default the TO address used is {group}@example.com, which in this case is dev@example.com. The default FROM address used is overlord@example.com. If you don’t want to use example.com then this can be overridden in the dtgov.properties using key governance.email.domain and governance.email.from for the FROM address. It is recommended to use an email alias or group to tie the 'group' email to actual email addresses.
The email contains the following info:
Subject: [Overlord-dev] dtgov-demos-switchyard-2.0.0-20131106.145057-1.jar is deployed Artifact 6eccc2f4-b687-4882-9a05-fc446bbb8a44 with name 'dtgov-demos-switchyard-2.0.0-20131106.145057-1.jar' has been deployed to target dev. Please claim this task, test this deployment and set a pass/fail status at the taskform at http://localhost:8080/dtgov-ui/#taskInbox --Overlord
and Figure Notification Email in FakeSMTP. shows displays the email in the FakeSMTP UI.
Email templates are deployed in the dtgov.war/WEB-INF/classes/governance-email-templates directory. The template subject and body picked are {template}.subject.tmpl and {template}.subject.tmpl}, which in this case are deployed.subject.tmpl and deployed.subject.tmpl.
Any user in the dev group can now navigate to the taks list, and the user should see at least one entry
The user can click on this task to arrive at the detail screen:
The dev user should claim and start the task and then mark as Pass/Fail and Complete. The artifact will get classified as #DevPass and you should see the following logging on the server:
09:17:21,161 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (http-/127.0.0.1:8080-39) Calling PUT TO: http://localhost:8080/dtgov/rest/update/classification/http%3A*2F*2Fwww.jboss.org*2Foverlord*2Fdeployment-status.owl%23DevPass/44021610-f85e-48bf-9a1c-9adcdbe485b6
09:17:21,359 INFO [org.overlord.dtgov.jbpm.util.HttpClientWorkItemHandler] (http-/127.0.0.1:8080-39) reply={artifactName=dtgov-demos-switchyard-2.0.0-20131107.140403-1.jar, artifactCreatedBy=admin, status=success}
Based on the user input during the Test Dev task, a Gateway will now determine where to go next: * PASS - mark as passed in Dev, and send the artifact to QA * FAIL - mark as failed in Dev and Stop
A PASS will basically rerun the same tasks we just discussed but now in for qa, followed by stage and prod. So proceed Classify as DevTest, but you need to pretend it reads Classify as QaTest and so on.
When you navigate to the detail screen of the dtgov-demos-switchyard-VERSION.jar you should see the #DevPass, #InQA and #QaTest classifiers set on this artifact.
The DTGov console has a screens specifically designed to govern deployment like these. Navigate to http://localhost:8080/dtgov-ui/#deployments and select your deployment from the list.
From here you can look at the deployment history, the interesting content of the artifact.
This completes this demo.