As of SOASTA 56.15, the Service-Level Agreement (SLA) feature allows users to make performance measurement validations a part of continuous testing. By applying performance goals to tests, any regression in performance is immediately flagged, which significantly narrows down the cause of the regression. Simply put, this feature makes it easier to identify and resolve performance issues. SLA rules are automatically evaluated at the end of every test and marked as Passed, Warning, or Failed.
Below, you will find details on:
- Creating an SLA
- Viewing SLA Results
- Creating Your Own Dashboard
Creating an SLA
- Click the Central tab > SLAs and then the new button on the top.
- Name the SLA and enter a Location.
- Collection: Pick a rule and enter values to specify the ranges. The start of each range is inclusive.
- Minimum Completions Per Second: Number of completions of the Transaction/Group/Page divided by the number of seconds in the test.
- Maximum Nth Percentile Duration: The Nth Percentile for all collections where N is specified by the user (eg. 90th). Only successfully completed collections are included. (If the Maximum Nth Percentile Duration is greater than the threshold, then the rule is in violation.)
- Maximum Error Percentage: Number of Transaction/Group/Page errors divided by the number that started.
- Minimum Duration:The collection with the shortest effective duration. Only successfully completed collections are included. (If the Minimum Duration is less than the threshold, then the rule is in violation.)
- Maximum Duration: The 90th (default) percentile duration of the Transaction/Group/Page.
- Average Duration: The total effective duration for all collections (transactions, pages, groups) divided by the total number of collections which started. Only successfully completed collections are included. (If the Average Duration is greater than the threshold, then the rule is in violation.)
- Average HTML Doc Duration:The average duration of all the base HTML pages for all Page type collections. Only successfully completed pages are included. (If the Average HTML Doc Duration is greater than the threshold, then the rule is in violation.)
- Composition: Pick a rule and enter values to specify the ranges. The start of each range is inclusive.
- Average Response Time: The average (mean) response time for all messages in the test. Only messages which received a response are included.
- Throughput (bits Per Second): The total number of bits, sent and received, divided by the number of seconds in the test.
- Throughput Sent (bits Per Second):The total number of bits sent divided by the number of seconds of the test.
- Throughput Received (bits Per Second): The total number of bits received divided by the number of seconds of the test.
- Minimum Message Completions Per Second:The total number of messages/actions that received a response, divided by the number of seconds of the test
- Maximum Message Error Percentage:The total number of message/action errors divided by the number of message/action send attempts.
- Open the Composition Editor (by clicking on Compositions > New Test Composition) for the Composition for which SLAs will be applied. You should already have a clip ready in the Composition Editor.
- In the lower panel, click Properties > Composition > SLA
- Click to add a new rule and browse for an SLA. For SLA rules that apply to collections, click the Choose Collections button to browse for a transaction path.
- Alternatively, wild cards can be applied to multiple transactions. Below, the path CommonBrowsingClip/ProductViewingGroup/Page* also applies to the following.
Viewing SLA Results
Access the SLA Dashboard one of the following ways:
- Open a parent Composition in the Composition Editor, Results tab.
- Open the result of a parent Composition from Central (in which case, the SLA Dashboard is shown by default).
- Open the Build in Jenkins and viewing the test result of a parent Composition.
Hover over the Last Result column to see actual values. If you want to open a detailed lower-panel where you can choose which columns to display, click Compositions Overview.
Create a Report Using Composition Overview
As of SOASTA 57, reports render the Composition Overview widget using native tables. You can generate a report to view results in HTML or MS Word.
You can create a report by logging into your CloudTest instance and clicking Compositions on the leftmost panel. In your list of Compositions, double-click your Composition and select Create a New Report.
When the Create a New Report dialog appears, choose between a MS Word or HTML report template. In the Widgets section, expand All Other Widgets > Functional, and then select Compositions Overview.
Creating Your Own Dashboard
If you are creating your own dashboard, open the Widget Type panel on the left side and go to Functional > Compositions Overview. Place a checkmark on the Force SLA Mode checkbox.