You will need to write your own custom validation when the built-in validation provided by SOASTA CloudTest is not sufficient for your needs, or when you wish to implement validation that involves multiple Messages or Browser Actions.
You will need to write a Script that plays after your Message or Browser Action and does whatever custom validation logic you desire.
Your Script may access any previously completed Message or Browser Action. For example, you can compare the results of multiple Messages or Browser Actions, or validate values computed from multiple items (such as cumulative response time).
Your Script can place an error message into the Result, and/or it can cause the Test Composition to abort.
Insert a Script following the Message(s) and/or Browser Action(s) that you wish to validate.
Ensure that the Script doesn't play until the Message(s) and/or Browser Action(s) have completed.
If you are using a Sequenced Test Clip, this will always be the case and there is nothing special to do.
If you are using a Timed Test Clip, then you must either put the Message(s) and/or Browser Action(s) and the Script together into a Chain or else put a Checkpoint before the Script. You could also insert a Delay before the Script that you know will always be long enough, but this may not be consistently reliable if the play times of your Message(s) and/or Browser Action(s) vary.
Write the Script.
Key Scripting topics
Navigating the Test Composition Object Hierarchy.
Your Script can traverse the object hierarchy of the Test Composition to find the items in the Test Composition, such as Messages or Browser Actions that have completed prior to Script play. The Test Composition is at the top level. It contains Bands, which contain Tracks, which contain Test Clips, which contain all other items.
For example, the following statement retrieves the item in the Test Clip that precedes the current Script:
Properties of Context (
Properties of items in the object hierarchy useful for navigation include::
Accessing Messages and Browser Actions
For Messages that have completed previously, you can obtain the response (or a portion of the response), response time, count of bytes received, and other information. Refer to the Scripting documentation on the Message object.
For Browser Actions that have completed previously, you can obtain the outputs from the response. Refer to the Scripting documentation on the BrowserAction object.
Custom, System, and Global Properties
You can also access Custom Properties, System Properties, and Global Properties. Refer to the Scripting documentation on the $prop, $sysprop, and $globalprop variables.
Accessing external data
If your validation needs data from outside of SOASTA CloudTest, you will need to make that data available through Web Service calls. Make a Target for the Web Service, and then insert one or more Messages into your Test Composition to retrieve the necessary data. Ensure that the data retrieval Messages are played before the data is needed. Your validation can then access the responses to the data retrieval Messages.
If your validation needs data from outside of SOASTA CloudTest that does not change often, you could manually enter that data into Global Properties, and then access the Global Property values from your Script.
If you want to parameterize your validation so that it behaves differently each time the Test Composition is played, without having to edit the Test Composition each time, then write your Script to change its behavior according to one or more Global Property values. By editing the values of the Global Properties before the Test Composition is played, you can change the behavior. The Global Properties accessed by the Script could contain "flags" or "commands" that the Script interprets, or simply values that it uses during validation.
Another way to parameterize your validation so that it behaves differently each time the Test Composition is played, would be to have your Script access data outside of SOASTA CloudTest and change its behavior according to that data. (See the paragraph above about accessing external data.)
Posting messages to the Result
When your custom validation Script plays, it can log both informational and error messages to the Result. Refer to the $context.result object, and the postMessage method of that object in the Scripting documentation.
Aborting or stopping the Test Composition
Your custom validation Script can cause the Test Composition to abort or be stopped. Refer to the Test Composition object's abort and stop methods in the Scripting documentation.