Boris Chalinski

Akamai’s Requirements and Best Practices for Loadtesting

Blog Post created by Boris Chalinski Employee on Aug 23, 2016

(Updated 1-Sep-2017)

OVERVIEW

 

Customers interested in load testing their Akamai delivered digital properties are required to engage their Akamai account team for each load test to avoid violating Akamai’s Acceptable Use Policy  and to ensure the test is valid and distributed. Akamai’s Professional Services (PS) Team will help valid a customer’s testing framework, ensure even and proper distribution, update the metadata with troubleshooting best practices and ensure proper Services Performance team approvals. This will ensure:

  • Tests do not adversely impact our network and delivery of our customers' production traffic.
  • Tests are configured such that the artificial traffic is representative of the customer's real user traffic to ensure that measurements during the test give unbiased insight into the performance of delivery over the Akamai network.

 

By following the simple guidelines below customers can properly test after the one-time setup from Akamai is completed. In the event that a customer cannot meet the required guidelines below, a Managed PS agreement can help get proper tests set up.

 

SELF MANAGED SCOPE

 

Self Managed Load Tests have the below limitations. All other testing variables will require a Managed PS agreement. • Presently, only North America load-generating servers (i.e., load agents) and customer origin(s).

  • Presently, only Soasta and Keynote Load Testing vendors are approved for self-managed load testing.
  • Only content delivered by HTTP or HTTPS Production Networks
  • Only customers subscribed to Advanced Cache Optimization

 

 

ONE-TIME SETUP REQUIREMENTS

 

Summary of the Akamai PS components of the one-time setup:

  • Review Load Testing framework with Customer and Vendor to validate it follows self-managed requirements.
  • Evaluate with the Customer the impact of load testing on traffic bill.
  • Confirm the Advance Cache Optimization is on the contract.
  • Obtain Load Testing vendor’s contact information.
  • Open Proactive Technical Support ticket  for load testing timeframes.
  • Verify embedded content for all digital properties being tested.
  • Create a new CP Code for load testing traffic.
  • Update appropriate Akamai configurations with new CP code.
  • Configure High Edge Traffic Alert for Load Test CP Code.
  • Notify Service Performance of Self Managed test with requirements met details.
  • RECOMMENDED: Assist with enabling Log Delivery Services.
  • RECOMMENDED: Train customer and load testing vendor on LDS and Portal reports for troubleshooting.

 

 

LOAD TESTING FRAMEWORK

 

Required for each load test executed through Akamai:

  • Load Testing Framework reviewed with Akamai PRIOR to load test execution (3 weeks minimum lead time).
  • Load Testing PS One-Time Setup completed PRIOR to load test execution.
  • Load Test generation distributed across all available load server regions (minimum 9 locations)
  • Load ramp up should not occur faster then 0-full in 15 minutes, assuming a linear load growth.
  • All requests from Soasta must include Accept Encoding gzip header.
  • DNS lookup must be over UDP.
  • User Agent must include a string that identifies load test vendor.
  • DNST TTL of Akamai hostnames should be honored.
  • Load testing must occur between 11 pm and 5 am EST.
  • Load testing cannot be done during major Internet events: Black Friday/Cyber Monday any other major events like the Olympics or Back to School period.
  • Load testing traffic volume should not exceed 500Mb/s from a give geo area + ISP combination.
  • Perform a calibration test for all new test scripts (10 Mbps peak).
  • Load testing must NOT be done on properties protected by Akamai Web Application Firewall if the test is over 100Mb/s 
  • Customer opens a pro-active Technical Support ticket 24 hrs prior to each test providing:
    • Digital properties testing
    • Start and end date/time of test
    • Load test objective (i.e. 10000 page view per second, etc)
    • Expected load from each region o Results of calibration test
    • Load test vendor and customer contact for the test Recommended for each load test executed through Akamai:
  • Include If-Modified-Since/If-None-Match headers
  • User Connection:keep-alive
  • If site sets cookies, persist by each virtual user session independent of other virtual users
  • Enable LDS for Load test CP codes
  • Be aware and review impact of load test traffic to billing

 

 

Recommended for each load test executed through Akamai:

 

  • Include If-Modified-Since/If-None-Match headers
  • User Connection: keep-alive
  • If site sets cookies, persist by each virtual user session independent of other virtual users
  • Enable Akamai Log Delivery Service (LDS) for all Load tested CP codes
  • Be aware and review impact of load test traffic to billing

 

ADDITIONAL INFORMATION

 

Contact your Akamai Account Team or load-testing@akamai.com for further information.

 

Some common mistakes in load testing to avoid

 

Bandwidth bottleneck ignored. A load test against the origin server using a local agent runs fine as they are interconnected by a 100T LAN. The agent and/or the origin server are connected to the Internet by a relatively small uplink, e.g., T1. Testing against an Akamaized site cannot exceed T1 throughput.

 

Testing from one (or very few) location(s). When using one or more testing agents behind a single DNS resolver, all the traffic to an "Akamaized" site is sent to one Akamai region. Live traffic is distributed across many Akamai regions. Even if the user base is in a small geographical area, live traffic would be balanced across multiple regions so a single region load test is almost never valid. In a real world scenario, traffic originates from multiple (client) diverse IP sources in various ISPs using diverse DNS resolvers and hence distributing the load across multiple Akamai servers in various POPs. Load testing from a single source IP (Load Testing agent) against a few single Akamai Edge servers may lead to unexpected results and be against our Akamai’s Acceptable Use Policy

 

Testing cacheable content. Cacheable images cannot be readily load tested in that a small number of agents will repeatedly ask for the same images to only a handful of ghosts. The cache miss rate will be near zero during a load test even if the TTL is low (even 1 min) and the miss rate will be much higher in production.

 

Simultaneous performance monitoring does not target the same regions hit by load test. A Keynote or similar test from agent A1 to region R1 may not see any of the effects of the load test if the load test is mapped to regions other than R1.

Outcomes