The MDW Automated Testing facility uses Groovy DSL as its scripting language. Groovy runs on the Java platform and supports Java language semantics, so it's readily adopted by developers. But in addition, MDW test case scripting offers a friendly DSL syntax that makes it easy for anyone with a technical background to quickly build and run automated tests.
Groovy and Java provide their own exhaustive documentation. For a quick understanding of basic Groovy language constructs, such as comments, the Groovy Statements and Operators reference pages are helpful.
STDOUT is directed to the test results execute.log file, and is also visible in MDW Studio's Test Exec view output pane. Standard Groovy variable substitutions are applied to double-quoted strings, so the following lines both display the master request id for a run:
println "masterRequestId: " + masterRequestId
println "masterRequestId: ${masterRequestId}"
The context for variable substitutions is an instance of GroovyTestCaseScript. By default the master request id for the test is auto-generated, but you can assign it yourself as well. You can also use the standard Groovy assert keyword to check values and fail the test if some condition is not met:
// ensure a numeric master request id masterRequestId = System.currentTimeMillis(); assert masterRequestId > 0
One of the key functions of an MDW test is to run a workflow process and compare the outcome versus the expected result. This concept is explained in detail in the automated testing documentation. The simplest way to start a process is like this:
start "MyProcess"
This kicks off the latest version of the process named MyProcess in any workflow package.. To specify a particular package and process version you can use this form:
start "com.centurylink.mdw.mytests/MyProcess v1.2"
To designate a version range using Smart Versioning (this example specifies a minimum version of 1.2 and a maximum version up to, but not including, 2.0): todo: smart versioning not fully implemented
start "com.centurylink.mdw.mytests/MyProcess v[1.2,2)"
Frequently when launching a process you"ll want to specify input variable values, which can be done this way:
start process("com.centurylink.mdw.mytests/MyProcess") {
// set the input variables firstName and LastName
variables = [firstName: "Donald", lastName: "Oakes"]
}
For multiline strings you can use three quote characters to start and end the value:
start process("com.centurylink.mdw.mytests/MyProcess") {
variables = [employee: """<Employee>
<workstationId>dxoakes</workstationId>
<firstName>Donald</firstName>
<lastName>Oakes</lastName>
</Employee>"""]
}
Or it's frequently convenient to externalize a long string as a separate asset:
start process("com.centurylink.mdw.mytests/MyProcess") {
variables = [employee: asset("Employee.xml").text]
}
By default the Employee.xml asset is loaded from the same workflow package where the test case resides, but this path can also be qualified with the package name ("com.centurylink.mdw.mytests/Employee.xml").
For processes that include manual task activities, you can trigger a task action like so:
action task("My Manual Task") {
outcome = "Claim"
}
This assigns the manual task to the user executing the tests. The outcome keyword indicates the action to be performed on the task, which can be any standard or custom Task Action.
Completing a manual task frequently requires that input data be entered along with the action. This can be accomplished using the variables keyword just like initializing input variables for a process:
action task("My Manual Task") {
outcome = "Complete"
variables = [customer: "Donald Oakes", ackDate: new java.util.Date()]
}
If you need to launch multiple processes with the same task name then you can specify the task instance you want to
target by appending [task number -1] at the end of task name. Example:
action task("My Manual Task[1]") {
outcome = "Claim"
}
In above example you would be Claiming a Task named "My Manual Task" in second process instance of same master request id.
For test cases that need to notify waiting in-flight processes, the following syntax can be used:
notify event("CorrectionEvent-${masterRequestId}") {
message = file("CorrectionEventMessageContent.xml").text
}
The event ID parameter (to correlate with the waiting process instance) can embed the following
server-side placeholders, which will be substituted with the appropriate runtime values:
${masterRequestId}, ${processInstanceId}, ${activityInstanceId},
${<variable_name>}.
Note: Although the syntax looks the same as the general variable substitution syntax, this special usage involves substitution on the server (the raw placeholders are passed in the test runner's message to the server, and substitution happens on that side). In this usage, only the specific expressions above are supported. Unlike the general mechanism, your custom Groovy script bindings cannot be substituted.
In order for ${processInstanceId} to be correctly substituted, you'll need to specify processName
for your event, and for ${activityInstanceId} you'll also need to specify activityLogicalId:
notify event("CorrectionEvent-${processInstanceId}-${activityInstanceId}") {
message = file("CorrectionEventMessageContent.xml").text
processName = "com.centurylink.mdw.mytests/OrderProcess"
activityLogicalId = "A7"
}
In order for ${<variable_name>} to be correctly substituted, you'll need to load the process that has this variable
and read the variable value
load process("MyProcess")
notify event("CorrectionEvent-${variable("varName")}") {
message = file("CorrectionEventMessageContent.xml").text
}
Before the process outcome can be compared against the expected results, the test execution needs to wait for the process to be completed. The simplest form of the wait command is like this:
wait process
This waits for completion of the process most recently launched through the start command, using the default timeout of 60 seconds. If the process was launched through an event message, you'll need to specify it by name:
wait process("com.centurylink.mdw.mytests/MyProcess") {
timeout = 120
}
The example above designates a timeout value to override the default of 60 seconds. In either case, if the process finishes before the allotted timeout, the test case will proceed right away rather than continuing to wait. By default this waits for process "Completed" status. You can also wait for other statuses like this:
wait process("com.centurylink.mdw.mytests/MyProcess") {
status = "Cancelled" // valid values are Completed, Cancelled, Failed, Waiting and Hold
timeout = 90
}
For even greater granularity you can wait for a specific activity within a process by designating its logical ID:
wait process("com.centurylink.mdw.mytests/MyProcess") {
activityLogicalId = "A16"
status = "Cancelled" // valid values are Completed, Cancelled, Failed, Waiting and Hold
timeout = 90
}
The sleep command tells process execution to pause for a designated period. The syntax for sleep is very simple:
sleep 10 // wait for ten seconds
This compares the expected results file versus the actual results for a process that was started through either start process or send message. These are stored as files, in the format described in MDW Test Results Format (this document also describes the Groovy expression mechanism for substituting dynamic values in expected results). If the comparison show differences, the test case fails with a message identifying the line number of the first delta.
If a single process was launched using the start command, the following simple form can be used:
verify process
Otherwise you'll need to designate which process to verify:
verify process("com.centurylink.mdw.mytests/MyProcess")
If your process launches any subprocesses, use verify processes:
verify processes("MyProcess", "MySubprocess")
There's no need to include embedded subprocesses like Exception Handlers in your verify command. These are automatically included in the actual results YAML output.
If your process executes activities in parallel and they run in an unpredictable order, you may choose to sort activities in the results YAML by ID rather than the default sort of start time:
verify process {
resultsById = true
}
Beyond using regular expressions to substitute known dynamic values as described above, you can also exclude selected variables from comparison altogether. Here's the syntax for excluding some values from output results YAML:
verify process {
excludeVariables = ['ignoredString','ignoredJson']
}
To exclude variables from multiple processes:
verify(processes('SmartProcessParent', 'SmartProcessChild'), {
excludeVariables = ['testCase', 'inputVar']
})
Note: Values in the excludeVariables list are omitted from the output of all processes/subprocesses.
Some cases need to compare only specific values from a complex document variable instead of the entire results. The result document may have many subelements that are dynamic and yet are not relevant to the test's success. Here's an example of how to handle that scenario:
start process("com.centurylink.mdw.tests/MyJaxbTest")
wait process
load process
assert masterProcessInstance.status == 'Completed'
// test individual element values in a JAXB variable
def jaxbVarXml = masterProcessInstance.variable['myJaxbVar']
def jaxbVar = new XmlSlurper().parseText(jaxbVarXml)
assert jaxbVar.RequiredElement.text() == 'requiredElementValue'
In this example "load process" is used instead of "verify process" (which would fail due to unrelated dynamic elements). Then the master process instance status is checked, along with selected values from a specific jaxb variable.
If your test process involves an adapter activity, you may want to inject a stubbed response to the adapter call so that the test doesn't depend on availability of an external system. To inject a stubbed response based on an XPath test condition, use a snippet like this:
stub adapter(xpath("GetEmployee")) {
delay = 2
response = file("GetEmployeeResponse.xml").text
}
This registers a stub handler to respond to any adapter whose request content is XML with a "GetEmployee" root node (based on MDW XPath evaluation), and set the response to the contents of the file GetEmployeeResponse.xml. You can do the same thing using GPath:
stub adapter(gpath("request.name() == 'GetEmployee'")) {
delay = 2
response = file("GetEmployeeResponse.xml").text
}
Note: For adapter stubbing to be applied, the "Use Stubbing" checkbox must be selected on the test case launch dialog.
Multiple stub declarations can be included in a test case (before triggering any processes with the adapters you wish to stub). Stubs are consulted in the order they were declared. If no matching stub is encountered for an adapter, then it will be invoked according to its design attributes.
You're not limited to simple XPath or GPath expressions for matching requests. For example, the following script matches JSON requests if the top-level object name is "employee" and the employee type is "manager":
// json stub for "employee" object
def jsonMatcher = { Object request ->
def result = new groovy.json.JsonSlurper().parseText(request);
return result.employee != null && result.employee.type == "manager"
}
stub adapter(jsonMatcher) {
delay = 2
response = file("employeeResponse.json").text
}
In mdw6 there is a more flexible option for stubbing adapters. Whereas the stub adapter option allows you
to control stubbing according to the request payload, stub endpoint give you the ability to consider other factors
such as the endpoint URL and HTTP method. Instead of a string, the input into the stub endpoint matching closure
is an instance of AdapterStubRequest.
So, for example, to stub a specific REST endpoint URL, you could do something like this:
stub endpoint({ request ->
request.method == 'POST' && request.url.endsWith("e911ServiceProfile/phoneNumber/${tn}")
}) {
response = asset("sd.v1.api.samples/e911_PostResponse.json").text
statusCode = 202
statusMessage = 'Accepted'
}
Expressions in a stubbed response are automatically processed using Groovy variable substitutions. So, for example, the following stub response in EmployeeLookupResponse.xml will contain the actual master request ID at runtime:
<EmployeeLookupResponse>
<masterRequestId>${masterRequestId}</masterRequestId>
<name>Donald Oakes</name>
</EmployeeLookupResponse>
A common scenario requires including some values from the request in the response. In that case you can make use of the implicit request variable, and assign response values using GPath expressions:
<EmployeeLookupResponse>
<masterRequestId>${masterRequestId}</masterRequestId>
<workstationId>${request.userId}</workstationId>
<name>Donald Oakes</name>
</EmployeeLookupResponse>
In case this built-in GPath substitution mechanism does not exactly fulfill your requirements, you can define a custom Groovy closure to populate the stubbed response based on the raw request string. Here we simply echo the request:
def responder = { Object request ->
return request;
}
stub adapter(gpath("request.name() == 'EmployeeLookup'"), responder) {
delay = 2
}
In fact you can stub any activity in your workflow. This is particularly useful for Timer Waits, Event Waits, Manual Tasks, an other long-running steps that should be short-circuited during automated testing. The following example starts a process and stubs the activity named My Timer and after two seconds initiates the default outbound transition. It also stubs another activity with logical id = A4, sets some process variables and then transitions to myOutcome. Finally, it also stubs the A3 activity in MySubProcess with the default outcome.
start process("com.centurylink.mdw.tests/MyProcess") {
activityStubs = [
activity("My Timer") {
sleep = 2 // wait two seconds instead of configured time
return null // default transition
},
activity("A4") { // logical id for some other activity
variables = [updatedString: "updated", updatedDoc: "<updated/>"]
return 'myOutcome' // non-null result code controls outbound flow
},
activity("MySubProcess:A3") { // subproc logical id
// no need to return null for default transition
// NOTE: this only works when variables/sleep not specified
}]
}
Note: Activity stubbing requires MDW workflow package com.centurylink.mdw.testing to be imported into the environment where the test cases are run. For adapter activities, it's still usually better to use Adapter Stubbing since this actually populates the request and response documents.
Sometimes its useful to stub all timer waits in a test case. For that, you can implement a custom closure to be executed which determines whether each activity is stubbed. Here's an example that automatically stubs every TimeWaitActivity instance.
def timerStubMatcher = { Object runtimeContext -> // instance of ActivityRuntimeContext
return "com.centurylink.mdw.workflow.activity.timer.TimerWaitActivity"
.equals(runtimeContext.getActivity().getImplementorClassName());
}
start process("com.centurylink.mdw.tests/TestActivityStub") {
variables = [flowPath: "all timers"]
activityStubs = [
activity(timerStubMatcher) { // logical id for timer activity
return null // default transition
}]
}
Service API tests start with request template assets that are most easily created in Postman. Export your Postman test collection to an asset with extension '.postman'. You can then execute the Postman tests directory, or you can use them in a Groovy test.
In the following example admin-apis.postman is the asset, GET is the HTTP method, and workgroups/{group-name} is the Postman test:
def res = submit request('admin-apis.postman/GET:workgroups/{group-name}') {
values = ['group-name': 'GroupA']
}
assert res.time < 1000 // less than a second
assert res.status.code == 200
assert res.headers['content-type'] == 'application/json'
def workgroup = new JsonSlurper().parseText(res.body)
assert workgroup.name == 'GroupA'
The response for each submit in a test case is appended to the yaml results file,
so multiple tests can be chained together, and their output compared. Process verification
outcomes can also be included, so that the overall results for service orchestration workflows
can be verified
NOTE: The Service API Testing section above describes a better to perform this testing that supports declarative results expectations. Sometimes it can be useful to submit a raw HTTP request to an MDW service or other endpoint. An example usage is to invoke a REST resource request to verify some test case outcome.
def resp = get http("Services/Tasks/" + task.getId() + "/indexes?format=json")
This submits an HTTP GET request and captures the response in a TestCaseResponse object named 'resp'. The response contents can them be accessed to compare with what's expected:
def indexes = new groovy.json.JsonSlurper().parseText(resp.getContent()) assert indexes.dateIndex == today.toString()
The parameter to the get() method can be a full HTTP URL, or a path relative to the MDW services endpoint URL.
Similarly, an HTTP POST request can be submitted:
def resp = post http("Services/MyUpdateService") {
payload = asset('com.centurylink.mytests/my-request.yaml').text
}
The HTTP PUT and DELETE methods are also supported using the same syntax. You can verify the response vs. an asset (with expression substitutions) this way:
assert response.code == 200
verify response {
expected = asset("my-json-asset.json").text
}
Another way to trigger your workflow is to send an event message:
send asset("GetEmployeeRequest.xml").text
This sends a message to MDW with the contents of the specified file over the default REST protocol. To send a message using another supported protocol use a form like this:
send message("SOAP") {
payload = '''\
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<EmployeeLookupRequest>
<workstationId>dxoakes<workstationId>
</EmployeeLookupRequest>
</soapenv:Body>
</soapenv:Envelope>'''
}
println "Message Sent..."
The payload keyword indicates the contents to send to MDW. (Naturally, the SOAP protocol requires that the payload include a SOAP envelope element). In either form, the raw message contents can contain dynamic values in the form ${myRequestPlaceholder}. Standard Groovy expression evaluation is performed for these values. The evaluation context is the same as described in MDW Test Results Format.
When a test case involves a send message step to a service process, frequently success needs to be evaluated according to the contents of the response:
verify response {
expected = '''\
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<EmployeeLookupResponse>
<workstationId>dxoakes<workstationId>
<sapId>DHO115360<sapId>
<firstName>dxoakes<firstName>
<lastName>dxoakes<lastName>
</EmployeeLookupResponse>
</soapenv:Body>
</soapenv:Envelope>'''
}
Usually it's handy to keep the expected response in a separate asset:
verify response {
expected = asset("EmployeeLookupResponse.xml").text
}
An easy way to initially populate the expected response is to run the test case once and copy/paste the actual response from the Text Exec view console window. In your expected response content you can embed dynamic expressions as described in MDW Test Results Format.
TODO: describe unit tests with examples