How do I incorporate SOASTA CloudTest into an automated build?

Document created by DPM Admin on Jul 21, 2017
Version 1Show Document
  • View in full screen mode

SOASTA test compositions and playlists can be incorporated into an automated build process using either Apache Ant (for Java-based builds), and subsequently, the Jenkins/Hudson plugin; or the SOASTA SCommand utility (for other builds). Additionally, the SCommand utility comes with an Ant task that is used to facilitate build integration and can play a role in any command-line execution of test compositions such as that used in a Jenkins job.

This article provides some introductory material and samples of use to build engineers. For a complete guide to Jenkins Continuous Integration, refer to the TouchTest Jenkins CI Tutorial.

Section 1: Apache Ant

The Apache Ant class provided in SCommand:

com.soasta.tools.scommand.anttasks.PlayTask

has the attributes shown in the table below:

AttributeDescriptionRequired
typeThe type of test object (either "composition" or "playlist")Yes
nameThe test object name
urlThe SOASTA CloudTest URL
usernameThe SOASTA CloudTest user name
passwordThe SOASTA CloudTest password
waitforcompletionSet to "true" to block the build until the composition (or playlist) completesNo, defaults to "false"
haltonfailureSet to "true" to stop the build process if the composition (or playlist) fails
failureproperty

The name of a property to set if the composition (or playlist) fails.

No
dir

The directory in which to write test output.

No, if missing then output is written to standard out.

formatThe output format to use.  Possible values are "text", "xml", and "junitxml".  The "junitxml" format is similar to the output produced by JUnit XML formatter, and is compatible with the Ant's JUnitReport task, as well as Hudson and Jenkins (see below).

No, defaults to "text"

 

Note: The xml output will contain information about the evaluation of each SLA rule, if applicable.


This complete example uses these attributes in a typical scenario.

Reporting Results using Apache Ant

 

  • Set the "format" attribute to "junitxml" in order to create reports using any tool capable of reading JUnit output. For example, the JUnitReport task that comes with Ant.

The example on the right will produce a file called "cloudtestresults/My tests/Composition 1.xml".

 

<property name="cloudtest.out" location="cloudtestresults" />
<play type="composition" name="/My tests/Composition 1" url="${cloudtest.url}" username="${cloudtest.username}" password="${cloudtest.password}" waitforcompletion="true" failureproperty="cloudtest.failure" format="junitxml" dir="${cloudtest.out}" />

  • A subsequent HTML report can be created using this example (shown right).

<junitreport todir="cloudtestreports">
   <fileset dir="${cloudtest.out}">

     <include name="**/*.xml"/>
   </fileset>
   <report format="frames" todir="cloudtestreports/html"/>
</junitreport>

Section 2: Using the Jenkins/Hudson Plugin

SOASTA CloudTest includes first-class support for running tests via Apache Ant, and including the output in build reports for Jenkins and Hudson via the Jenkins/Hudson plugin. This section provides the necessary steps for build integration using this plugin.

 

Downloading the Plugin

In the Resources page of SOASTA CloudTest, right-click the "CloudTest Jenkins/Hudson Plugin" link, and save the linked file (cloudtest.hpi) to your computer.

Alternatively, if you are running Jenkins on a "headless" server, you can curl or wget.

 

Installing the Plugin

  1. Once you've downloaded the file, you can either upload it via the Jenkins web interface, or using the command line. To upload it via the web interface, browse to the Jenkins Plugin Manager page and click the Advanced tab.

Alternatively, you can manually copy the cloudtest.hpi file to the plugins sub-directory of your Jenkins installation.

 

 

 

  1. Restart Jenkins once this is finished. After the re-start is finished, you should see "SOASTA CloudTest Plugin" appear in the list of installed plug-ins.

 

 

Configuring the Jenkins Job(s)

A functional Ant file and its concomitant Jenkins job are a prerequisite for the remaining instructions.

Note:  This can be done by following the Apache Ant instructions in Section 1 above. CloudTest also supports build integration in non-Java platforms via the sCommand utility (which comes with an Ant task as well). This latter method is described in Section 3 below.

  1. Edit the Jenkins job configuration and scroll down to the Post-build Actions section.
  2. Check the "Publish JUnit test result report" box, and set "Test report XMLs" to the wildcard pattern for the Ant output.
  3. Next, check the "Include links to SOASTA CloudTest dashboards", and save the job configuration.

 

 

 

Displaying CloudTest Dashboards in Jenkins Build Reports

  1. Once you've updated the Jenkins configuration and run your first build, you can browse through the Jenkins build report to see the test results, and easily open the CloudTest dashboard for full test details.

 

 

 

 

 

Section 3: Other automated build tools

Any build tool that can run a program and check the exit code can be used to incorporate SOASTA CloudTest.

Here is an example using make:

 

 

 

 

SCOMMAND_HOME=/Downloads/scommand
SCOMMAND_BIN=$(SCOMMAND_HOME)/bin
CLOUDTEST_URL=localhost:8080/concerto
CLOUDTEST_USERNAME=exampleuser
CLOUDTEST_PASSWORD=***

all: cloudtests

cloudtests:
$(SCOMMAND_BIN)/scommand cmd=play type=composition name="/My tests/Composition 1" 
url=$(CLOUDTEST_URL) username=$(CLOUDTEST_USERNAME) password=$(CLOUDTEST_PASSWORD) wait

Using the play command with the wait parameter causes SCommand to wait until the composition (or playlist) finishes, and then exit with code 0 if the test passes, or code 1 if it fails.

Attachments

    Outcomes