CloudTest Glossary

Document created by Chris Sommerstad Employee on Jul 21, 2017
Version 1Show Document
  • View in full screen mode
CloudTest Components
  • CloudTest Components - a dynamic web application that combines a Web Browser UI with distributed web services to support test creation, test grid deployment, test execution and real-time performance analytics. CloudTest Components include CloudTest Builder, CloudTest Main, CloudTest Load Server, CloudTest Results Server, CloudTest Database Server, and CloudTest Conductor. CloudTest Components also include the TouchTest Device Agent and the TouchTest Server for use on one Device.
    • CloudTest Analytics Dashboard - a display, which works in conjunction with the CloudTest Results Server to display interactive, real-time streaming analytic dashboards. The CloudTest Analytics Dashboard correlates multiple data streams from a distributed load test into a single test result synchronized and displayed on a synchronized timeline.
    • CloudTest Builder - a web application that can be accessed from the following Web browsers: Internet Explorer 7.0 and later, Firefox 3.0 and later, Safari 4.0 and later and Chrome. CloudTest Builder includes the visual editors used to build performance, load, functional and Web UI tests for a modern web application.
    • CloudTest Conductor - an installable multi-purpose software agent. It can be used to records http(s) message and/or browser actions for test creation, launch browser instances and plays UI/Ajax/Flex tests on client machines and monitor and collect performance metrics from SOASTA components and from the customer's infrastructure for inclusion in the analytics.
    • CloudTest Database Server - a persistent data store for Repository’s metadata and the Results Server’s data. Additional Database Servers may be deployed by Customer in the Cloud in order to optimize performance and bandwidth use.
    • CloudTest Load Server- a test engine that executes all or parts of a complex test composition. Multiple Load Servers can be deployed using the CloudTest Grid Wizard to each execute parts of a load test that can scale out virtual users (actual scale is dependent on available servers). Load Servers can be geographically distributed across multiple cloud services, if available, and single test compositions can run geographically.
    • CloudTest Main - a physical or virtual software appliance that contains the CloudTest Web Application, the CloudTest Repository, CloudTest Results Server and CloudTest Database Server. The CloudTest Repository stores and retrieves SOASTA CloudTest Objects such as Test Compositions, Test Clips, Users, Targets, Scenario Recordings, and Cloud Account Credentials. The CloudTest Web Application includes CloudTest Builder, CloudTest Grid Wizard and CloudTest Analytics Dashboard.
    • CloudTest Grid Wizard - a fully automated capability to define, provision, detect and correct cloud vendor errors, health check and create cross-cloud vendor test server grids. Test Servers, which are not provided by SOASTA, may come from public and private clouds and from manually provisioned test servers.
    • CloudTest Results Server - an in-memory, real-time analytic service (with persistence to disk) purpose-built to handle very large result sets and analytics from complex distributed load tests. A Results Server is included with CloudTest Main. Additional Results Servers are deployed by CloudTest Grid Wizard to work in concert with the Load Servers to analyze the real time streams of data as the test runs
CloudTest Terminology

Message – Messages are the HTTP requests or XML sent to a web application or web service. Messages are represented visually in SOASTA CloudTest as boxes in the icon view and URL values in the list view.

 

Script - SOASTA CloudTest Scripts are written in JavaScript and allow you to add custom logic to a Test Clip. For example, a script that modifies the input data for a message based upon a set of status codes from the previous message response extracts data or validates responses.

 

Scope - the ability to set scope on objects in the repository that will define whether an item is private, public, or local. Scope determines whether the information about an item can be accessed from outside of the item. What information that may be depends upon the item.

 

Chain – A chain is an arbitrary grouping of messages, scripts, or checkpoints within the current Test Clip. This grouping may be used to control the timing of the chain members. Chains are often used in analytics.

 

Page – A page is a representation of a given HTML page, or URL as represented by its main "HTML Document" and all the resources to which it refers. In CloudTest, a page can retrieve its resources statically (as recorded) or dynamically (as it exists at runtime of the test). Pages have additional load and cache settings as well as override settings at the page, clip, track, and composition level.

 

Test Clip - The building block of testing in SOASTA CloudTest is the Test Clip. A Test Clip is a visual script that is composed of a series of timed or sequenced events (an event can be, for example, an HTTP request, a SOAP message or a UI browser action). Once built and debugged, these Test Clips can be reused for any type of testing . . . functional testing, load testing, performance testing and Web UI testing where appropriate.

 

Composition - A Test Composition is a multi-track interface where test clips can be placed at different times and tracks to create complex testing scenarios. This interface is similar to music and movie digital creation tools. Test Compositions are created by dragging and dropping Test Clips directly onto a timeline or in a sequence based on when they should execute.

 

Target - A Target points to the application, service, or web site that you want to test, including its location (URL) and any authentication information necessary to successfully run a test. A Target also contains a specification of the information needed to properly format and send messages to the given service or site.

 

Monitor Server  - A Monitor Server defines which hardware resources will be monitored. Typical Monitor Servers are Linux/Windows/Mac, database, and application servers. SNMP devices are also Monitor Servers. A Monitor Server does not define the metrics to be captured; just ‘how’ they will be captured (IP Addresses, ports, authentication, etc.).

 

Monitor - A Monitor defines the resources for which you will capture data to be used in analytics. This data typically includes statistics on resource usage, processes and activity. Monitored data appears in the Dashboard on the same timeline as test metrics such as response times and error rates. Reference: SOASTA Monitoring FAQ

 

Dashboard - A Dashboard is where you view and analyze your test results. SOASTA CloudTest provides a wide variety of analytic widgets to fill your Dashboards. You can use one or more widgets to track your test in real time and capture results for post-test analysis.

 

Conductor - SOASTA Conductor is an agent that runs on a client machine. You will need to install a SOASTA Conductor on a machine for any one of the following purposes:

 

  • To record HTTP/S network traffic.
  • To run Web UI tests in a browser.
  • To monitor machine statistics (CPU usage, for example)

Virtual Users are software programs or services that, when executed iteratively, generate traffic from their location to target servers or services, as would real users of these same servers or services.

 

Load Testing (also known as Performance Testing) is the use of a coordinated community of virtual users to place “load” on servers or services simulating some future workload for evaluation. Many different categories or terms for performance testing with virtual users are in current use such as benchmarks, endurance, stress tests, blow-up tests, etc.


In the context of this tutorial “Load Testing” and “Performance Testing” will be used interchangeably.

Attachments

    Outcomes