MDW Test Results Format

Process Results

The format of MDW automated test process result files is the same for both actual and expected results. These are compared after test execution to determine success or failure.

The general format for MDW test results is YAML. Each yaml asset represents at least one master process instance along with its subprocesses, sorted by process name. In an actual results yaml file, the instance ID for each process is included in a (#-delimited) comment on the same line. Here's an example results file from mdw-demo:

process: # 13066
  name: HandleOrder
  instance: 1
  activity: # 1 "2016-02-16 12:04:33"
    name: Start
    status: Completed
  activity: # 3 "2016-02-16 12:04:33"
    name: |
      Prepare Lookup
      Request
    status: Completed
    result: false
  activity: # 5 "2016-02-16 12:04:34"
    name: |
      Customer
      Lookup
    status: Completed
  activity: # 6 "2016-02-16 12:04:39"
    name: Manual Ack
    status: Completed
  activity: # 2 "2016-02-16 12:05:03"
    name: Stop
    status: Completed
  variable: # 18779
    name: ackDate
    value: ${today}
  variable: # 18776
    name: acknowledged
    value: true
  variable: # 18774
    name: customerId
    value: DHO115360
  variable: # 18777
    name: customerLookupRequest
    value: |
      
        DHO115360
      
  variable: # 18778
    name: customerLookupResponse
    value: |
      
          DHO115360
          dxoakes
          Donald
          Oakes
      
  variable: # 18780
    name: customerName
    value: Donald Oakes
  variable: # 18775
    name: orderNumber
    value: ${masterRequestId}
  variable: # 18773
    name: request
    value: |
      
          ${masterRequestId}
          DHO115360

Within a process structure are elements for each activity executed for this instance. Each activity structure shows its name and end-state status. The path that a workflow process instance took can be reconstructed from this information, so its a reliable indication of whether the expected flow actually occurred. For activities the values after the comment flag indicate the logical ID and instance ID (not relevant for comparison but useful for investigating failures).

The variables set by a process are also included as elements of that process. The variable structure includes name and value elements. The values can contain placeholders for dynamic content, as illustrated by ${today} and ${masterRequestId} in the example above. See Dynamic Values for an explanation of this syntax.

Variable instance lines are sorted by variable name to produce a predictable result.

Send Message Responses

As discussed in Verify Message Response, it's a good idea to externalize the expected response content to a result asset. This asset can contain XML, JSON, or any other text-based content. Frequently you'll need to embed dynamic values in your expected response file so that comparison with the actual response will succeed. This is accomplished using dynamic placeholders as described below.

Dynamic Values

Dynamic values can be referenced in the expected result file using the form ${myDynamicValue}. These are evaluated using the standard Groovy expression syntax. The this object in an evaluation context is an instance of GroovyTestCaseScript, with bindings supplied from any variables declared in your test case script. Certain built-in values are also available:

${masterRequestId} (String) - always populated with either the default generated value or the value assigned in the test case script
${masterProcessInstance} (ProcessInstance) - populated when "verify process" is executed
${responseMessage} (String) - raw response value for most recent "send" command

These implicit variables are accessible using standard Java Bean conventions. For example, to access the master process instance ID you'd use an expression like:

${masterProcessInstance.id}

Groovy shorthand notation is also supported, so to access a process variable value you can use:

${masterProcessInstance.variable['myVar']}

In the case of process results, there may be rare occasions when a variable value is unknowable ahead of time. In that case, you can insert the comment character immediately to the right of the variable name in the expected results file to ignore its value altogether when comparing:

  variable: # 18749
    name: customerName
    value: Donald Oakes
    VAR variable-name #

There may be occasions where the only thing knowable about a result is its general format (numeric, XML, etc). Java Regular expressions can be used to support more complex dynamicism. Anything between ${~ and } is evaluated as a regular expression (regex). Note: Because of the way results comparison operates, a maximum of one regular expression is allowed per line, and the expression will not match across multiple lines. Here are some examples:

<workstationId>${~d\D}oakes</workstationId>
<firstName>${~Don.*}</firstName>
<lastName>${~O.*s}</lastName>

This will match with following:

<workstationId>dxoakes</workstationId>
<firstName>Donald</firstName>
<lastName>Oakes</lastName>

In MDW Studio, if verbose output is selected in the test launch dialog you'll see the substituted expected results in the Test Exec view output pane when you click on the test case. This is logged along with the actual results to help you troubleshoot a failing test case. You can also log your own output to this window using STDOUT:

println 'myVar variable value: ' + masterProcessInstance.variable['inputVar']
assert masterProcessInstance.variable['inputVar'] == 'my input variable runtime value'