We are using CloudTest for our server load testing (RESTful services) and New relic for monitoring. We have been noticing huge difference in average response time between New Relic and CloudTest. We reached out to New Relic and SOASTA CloudTest. We were told CloudTest calculates the whole time including non-application time (including dns, tcp and other net work time etc)
However New relic is measuring only the time it takes for the request and response. Time it takes for the application server to receive the request until the time the application server sends the response.
We still want to find out why there is a huge difference. Could someone share some pointers?
Appreciate your input.
Here is our sample load test data and average response time
Number of virtual users = 8004
New Relic Average response time = 14 ms
Ramp up time = 20 mins
Load test servers = 12
Errors = 60
SOASTA average response time = 587 ms
Effective throughput = 3142 msgs/sec