Configuring mPulse to measure A/B (Bucket) Tests

Document created by DPM Admin Employee on Jul 14, 2017
Version 1Show Document
  • View in full screen mode

A/B testing is a great way to compare two (or more) versions of your site to see if one is more popular with users than another. Perhaps you'd like to test out new CDN providers, a different page layout, or new performance tricks. mPulse's A/B test integration allows you to compare your tests for performance. We allow you to tag the mPulse code with the name of the currently running test so we can report on each test separately.

There are two steps to using A/B Tests:

    1. Define a Test Name in your page.
    2. Define a JavaScript variable on your page that contains the name of a page group. This variable name may be namespaced.

For example, use SOASTA.ab_test or ab_test.

SOASTA = {}; // don't use var to ensure it's global
SOASTA.ab_test = "test name"

  1. Add the test variable name to the mPulse Domain by entering it in the A/B (bucket) tests variable name field, which is located on the General tab in the Configuration screen of your app.
  2. Click OK after making any change.



Test Name Restrictions

There are two restrictions on the page group name:

  • Test names may only contain alphanumeric characters, dashes, underscores and spaces.
  • Test names must be at most 25 characters long.

mPulse uses the regular expression below to validate an A/B Test Name, and you may use the same to verify your code:

/^[a-zA-Z0-9_ -]{1,25}$/

Note: Measurements that fall into a test bucket will not affect your main site measurement. This allows you to use your main site (ie, with no ab_test variable defined) as the control, or baseline measurement, and define separate tests under the main site to test out new features. This also means that the overall beacon count on your main site will be lower than first expected. There is only 1 A/B variable available per App, but you can assign it many values; however, we recommend keeping the different values assigned as small a number as possible for easier analysis.