1. Customer contacts Akamai PS or Akamai Account Management, indicating that they are interesting in Load Testing.
2. Akamai advises customer on the options for Load Testing through Akamai (PS-managed load test packages or Self-managed load testing), and notes that depending on the load test objectives, bypassing Akamai during the load tests may be best. If customer is interested in Self-managed load testing though Akamai, Akamai advises customer on the process and best practices, highlighting those that are required.
3. Customer engages SOASTA (how this happens will depend largely on whether it's an on-demand load testing engagement managed by SOASTA or the testing is completely done by the customer).
4. Customer engages Akamai PS, for initial setup. Provides contact info for SOASTA's engagement lead.
5. If Akamai-PS led, Akamai PS completes Initial Setup. Plan on this step taking 2-3 weeks. It might go faster, but could take this long to get the initial setup done on the Akamai side. The Akamai customer needs to have a special product on their account "Advanced Cache Optimization". It could take some time to get contracts and other technical aspects (CP Code, for example, which allows Akamai to segment and log SOASTA traffic) completed on the Akamai customer account.
6. Customer and load test vendor follow Load Test Execution process and best practices for each load test
Load Test Execution (completed by Customer and SOASTA)
- Validate that testing via Akamai is consistent with, and necessary for, achieving the objectives of the load test. Else, test should bypass Akamai.
- Validate that the customer has completed the initial Akamai setup for a self-managed load tests. If the Akamai PS lead for the customer is known, contact them directly to confirm.
Load Test Configuration
- Test load generation should be distributed across all available load server regions, as close to even as possible (depending on the nature of the test). This is required so as to be representative of real traffic distribution and to measure performance from a variety of locations, making those measurements more statistically significant. Using the calibration test to evaluate load from each scenario will allow SOASTA to determine how scenarios should be distributed across load servers to provide optimal distribution of load during full test.
- Load ramp-up should not occur faster than 0-full in 15 minutes, assuming a linear load growth.
- Use as many different load generating regions as possible to distribute load across a statistically significant sample of geographic locations/ISP.
- All request from SOASTA hitting Akamai need to include Accept Encoding gzip header.
- Include If-Modified-Since / If-None-Match headers when appropriate. More header help.
- Use Connection:keep-alive
- DNS lookup must be over UDP
- User Agent must include a string that identifies the load test vendor: "soasta". This is so Akamai can track the load traffic more efficiently and enable additional logging, which comes in handy when troubleshooting if issues are discovered during the load test.
- If website sets cookies, they must be persisted by each virtual user session, independently of other virtual users.
- DNS TTL of Akamai hostnames should be honored
- Load testing should occur between 11pm and 5am ET
- Load test traffic volume should not exceed 300Mb/s from a given geographic area (metro area) + ISP combination. So, for example, if Amazon EC2 has a DC in San Jose with L3 as the ISP and another in San Jose that uses Quest, then each can put out up to 300 Mb/s; If both DCs are in San Jose and both are on Quest, then no more than 300 should come from the two combined.
To ensure the 300Mb/s threshold will not be breached during a test, perform a calibration test for all new tests. The calibration test should consume no more than 10Mbps peak, but there are no limitations on the time of day it can be executed.
Customer should create CCare ticket at least 24h prior to the test start time (for each test), providing:
- Digital properties that will be load tested
- Start and end date/time of test
- Load test objective (example: 10000 page views per second OR 1000000 concurrent sessions), or full load test script documentation (if available)
- Expected test load from each load agent location
- Results of the calibration test
- SOASTA contact for the duration of the test (if appropriate)
- Customer contact for the duration of the test
- A proactive ticket should be opened before every self-managed test execution
Enable LDS for Load Test Traffic CP Code(s) during each load test, if appropriate
Be aware of and evaluate the impact of load test on billing (the potential for bursting/superburtsing). Contact Akamai ASM if assistance is needed with evaluation (recommended)
Contact Akamai at email@example.com if you have any questions
If there are errors in the test, the following things are what Akamai would like tracked:
- HTTP Errors: (1) error type, (2) exact time and (3) Akamai server IP, (4) IP address of load generators, (5) URL
- DNS Errors: (1) error type, (2) exact time and (3) DNS Server IP, (4) IP address of load generators
Info should be delivered within 24 hours, otherwise logs are deleted. Customer should open a ticket and include all info.