Service-Level Agreement (SLA)

Document created by DPM Admin Employee on Jul 20, 2017Last modified by DPM Admin Employee on Jul 20, 2017
Version 2Show Document
  • View in full screen mode

As of SOASTA 56.15, the Service-Level Agreement (SLA) feature allows users to make performance measurement validations a part of continuous testing.  By applying performance goals to tests, any regression in performance is immediately flagged, which significantly narrows down the cause of the regression. Simply put, this feature makes it easier to identify and resolve performance issues. SLA rules are automatically evaluated at the end of every test and marked as Passed, Warning, or Failed.

 

Below, you will find details on:

  • Creating an SLA
  • Viewing SLA Results
  • Creating Your Own Dashboard

 

Creating an SLA

 

  1. Click the Central tab > SLAs and then the new button on the top.

 

  1. Name the SLA and enter a Location.

 

  1. Appiles To: Choose whether the SLA rules you create will apply to a Collection or Composition.

  1. Collection: Pick a rule and enter values to specify the ranges. The start of each range is inclusive.
  • Minimum Completions Per Second: Number of completions of the Transaction/Group/Page divided by the number of seconds in the test.
  • Maximum Nth Percentile Duration:  The Nth Percentile for all collections where N is specified by the user (eg. 90th).  Only successfully completed collections are included. (If the Maximum Nth Percentile Duration is greater than the threshold, then the rule is in violation.)
  • Maximum Error Percentage: Number of Transaction/Group/Page errors divided by the number that started.
  • Minimum Duration:The collection with the shortest effective duration. Only successfully completed collections are included. (If the Minimum Duration is less than the threshold, then the rule is in violation.)
  • Maximum Duration: The 90th (default) percentile duration of the Transaction/Group/Page.
  • Average Duration: The total effective duration for all collections (transactions, pages, groups) divided by the total number of collections which started. Only successfully completed collections are included. (If the Average Duration is greater than the threshold, then the rule is in violation.)
  • Average HTML Doc Duration:The average duration of all the base HTML pages for all Page type collections.  Only successfully completed pages are included. (If the Average HTML Doc Duration is greater than the threshold, then the rule is in violation.)
  1. Composition: Pick a rule and enter values to specify the ranges. The start of each range is inclusive.
  • Average Response Time: The average (mean) response time for all messages in the test.  Only messages which received a response are included.
  • Throughput (bits Per Second): The total number of bits, sent and received, divided by the number of seconds in the test.
  • Throughput Sent (bits Per Second):The total number of bits sent divided by the number of seconds of the test.
  • Throughput Received (bits Per Second): The total number of bits received divided by the number of seconds of the test.
  • Minimum Message Completions Per Second:The total number of messages/actions that received a response, divided by the number of seconds of the test
  • Maximum Message Error Percentage:The total number of message/action errors divided by the number of message/action send attempts.

 

Note: SLA rules are evaluated at the end of the test and apply to the entire test. The rule is applied to the aggregated metrics for a specific Transaction/Group/Page across all virtual users.

 

  1. Open the Composition Editor (by clicking on Compositions > New Test Composition) for the Composition for which SLAs will be applied. You should already have a clip ready in the Composition Editor.

Caution: Save your Composition before continuing.

 

 

  1. In the lower panel, click Properties > Composition > SLA

  1. Click to add a new rule and browse for an SLA. For SLA rules that apply to collections, click the Choose Collections button to browse for a transaction path.

 

  1. Alternatively, wild cards can be applied to multiple transactions. Below, the path CommonBrowsingClip/ProductViewingGroup/Page* also applies to the following.

 

  • CommonBrowsingClip/ProductViewingGroup/PageSportsEquipment
  • CommonBrowsingClip/ProductViewingGroup/PageClothing

 

Note: Wildcards must be entered manually and must be completed with an asterik.

 

 

Viewing SLA Results

 

Access the SLA Dashboard one of the following ways:

  • Open a parent Composition in the Composition Editor, Results tab.
  • Open the result of a parent Composition from Central (in which case, the SLA Dashboard is shown by default).
  • Open the Build in Jenkins and viewing the test result of a parent Composition.

 

Hover over the Last Result column to see actual values. If you want to open a detailed lower-panel where you can choose which columns to display, click Compositions Overview.

 

Create a Report Using Composition Overview

 

As of SOASTA 57, reports render the Composition Overview widget using native tables. You can generate a report to view results in HTML or MS Word. 

You can create a report by logging into your CloudTest instance and clicking Compositions on the leftmost panel. In your list of Compositions, double-click your Composition and select Create a New Report.

 

When the Create a New Report dialog appears, choose between a MS Word or HTML report template. In the Widgets section, expand All Other Widgets > Functional, and then select Compositions Overview.

 

Creating Your Own Dashboard

 

If you are creating your own dashboard, open the Widget Type panel on the left side and go to Functional > Compositions Overview. Place a checkmark on the Force SLA Mode checkbox.

Attachments

    Outcomes