AnsweredAssumed Answered

Difference between CloudTest average response time and New Relic

Question asked by Rajeswari Jayaraman on Feb 7, 2018
Latest reply on Feb 21, 2018 by Salvador Camacho Reyes

Hi

 

We are using CloudTest for our server load testing (RESTful services) and New relic for monitoring.  We have been noticing huge difference in average response time between New Relic and CloudTest.  We reached out to New Relic and SOASTA CloudTest. We were told CloudTest calculates the whole time including non-application time (including dns, tcp and other net work time etc)

 

However New relic is measuring only the time it takes for the request and response.  Time it takes for the application server to receive the request until the time the application server sends the response.

 

We still want to find out why there is a huge difference.  Could someone share some pointers?  

Appreciate your input.

 

Here is our sample load test data and average response time

 

Number of virtual users = 8004

New Relic Average response time = 14 ms

SOASTA cloudtest

Ramp up time = 20 mins
Load test servers = 12
Errors = 60
SOASTA average response time = 587 ms
Effective throughput = 3142 msgs/sec

 

Thanks

Outcomes