What Is Monitoring?
Monitoring is the ability to capture resource metrics about target servers and network devices involved in CloudTest-generated tests. Monitoring provides information about how servers and other network devices (load balancers, routers, etc.) are handling the load being generated against the web application. Real-time monitored metrics are included in the SOASTA CloudTest dashboards to help identify possible bottlenecks in the web application being tested. Monitored information is available at several different levels including system resources (i.e. CPU, network, disk, and memory utilization), processes (i.e. apache thread counts), and server specific information (i.e. application server JVM information, database commits/rollbacks, etc.).
SOASTA CloudTest's monitoring feature may be implemented in one of two ways:
- Agent-less approach
The agent-less approach makes use of open network protocols such as SSH (secure shell), SNMP, and RMI, to connect to any remote server t hat you wish to monitor. Any SOASTA CloudTest with access to the destination machine can capture resource information. This is the preferred method for capturing information on servers inside the same LAN as it does not require an installed application and has a minimal impact on performance.
- Agent-based approach
The agent-based approach is primarily for target servers that are behind the firewall, and does not require direct access from the SOASTA CloudTest to the target server. Instead, the SOASTA CloudTest Conductor, installed on the target server, creates an outbound HTTP(S) connection to SOASTA CloudTest and provides the resource information. This approach is also required when interfacing with Windows performance counters.
- External Data Sources
Why monitor with SOASTA?
If SOASTA monitoring is not enabled, deeper insight into target servers is not available in the SOASTA CloudTest dashboards. Instead, only external information is available; such as response times and errors. SOASTA monitoring data gives insight into how target servers are operating during a test and this data is automatically correlated on a timeline in the CloudTest real-time dashboards. This integrated data from the test gives users the ability to drill into the data and filter for just the information they are interested in. While drilling into the data, all the charts on the dashboard show the data based on the same timestamps. If SOASTA monitoring is not enabled, deeper insight into target servers is not available in the SOASTA CloudTest dashboards. Instead, only external information is available; such as response times and errors.
Does an agent need to be installed on the server to be monitored?
It depends. The agent method requires installing an agent (SOASTA Conductor) on the server(s) that you want to monitor., while the agent-less approach makes use of open network protocols such as SSH (secure shell), SNMP, and RMI. The agent-less approach is the easiest and fastest option for monitoring but is unavailable on Windows and requires some ports to be opened in the firewall. The agent runs on major operating system platforms: Linux, Windows, and Mac OS X and requires only outbound ports.
How do I choose whether to use an agent for monitoring or not?
The non-agent method is the preferred method for monitoring because it is the quickest and easiest to implement—and does not require the installation of software on customer hardware.
There are two requirements to utilize the non-agent monitoring option:
- System resources can only be monitored without an agent if the server is Unix-based (i.e. Linux, Solaris, FreeBSD, or Mac OS X). You can still monitor system resources on Windows, but the agent is required.
- The necessary port(s) must be open in the firewall. The port number varies depending on what is being monitored. For example, port 22 (SSH) is needed for monitoring system resource information, port 3306 for MySQL, etc. Consult the monitoring documentation for details about your specific server.
The image on the right shows an example of how monitoring system resources with SSH works architecturally. Any other services being monitored (i.e. databases, application servers, network devices, etc.) will have to have corresponding ports in the firewall opened.
There are specific cases when using an agent is required for monitoring. The two primary reasons for needing to monitor via agent are: (1) the operating system to be monitored is Windows-based or (2) the servers being monitored are behind a firewall and cannot be accessed externally via direct port access. No inbound ports need to be opened to support this monitoring method, but ports 80 or 443 (HTTP/HTTPS) must be opened outbound from the firewall.
Two approaches for using the agent are possible. The agent can be installed on each server being monitored (see Figure 2). This is best in cases where there are just a few servers that require monitoring—or if the servers to be monitored are all Windows-based.
An alternative to installing an agent on each server is to install a single agent behind the firewall and have that agent monitor the other servers (see Figure 3). That agent monitors the other servers behind the firewall via the ports required for those services. Figure 3 shows an example of using SSH to monitor system resources on two servers. This approach is used in cases where a large number of servers need to be monitored during a test. It saves time since an agent does not have to be installed on each server.
How do I install the agent?
Download the monitoring agent (SOASTA Conductor) from the SOASTA CloudTest > Resources page > Downloads. Refer to Installing SOASTA Conductor.
Are different metrics available using an agent for monitoring?
No. All the metrics are the same whether monitoring is done via agent or not.
What performance impact does monitoring have on the servers?
Very little. The interval for gathering the monitored metrics is configurable. The typical interval for gathering metrics is 10-20 seconds. If this interval is used, the amount of resources used by monitoring is virtually immeasurable.
What ports need to be open in the firewall to monitor the target servers?
If using the agent-less approach to monitor servers, the firewall needs to allow external access to the servers on the ports for the servers/services being monitored. Here are some of the typical ports needed for monitoring various services:
- System resources: port 22 (SSH)
- MySQL: port 3306
- PostgreSQL: port 5432
- SNMP: port 161
- JBoss: ports 1099, 1098, 4444
- Tomcat/Resin: port 9004
If using the agent to monitor servers, no inbound ports need to be opened, but ports 80/443 (HTTP/HTTPS) need to be opened outbound from the customer network to SOASTA CloudTest in the Internet.
Can the monitored information be encrypted when sent to the CloudTest servers?
Yes. By nature of SSH, monitored information is automatically encrypted when using the agent--less method of monitoring. Using the agent method, data can be sent to SOASTA CloudTest using HTTPS over port 443.
What data is sent back to the CloudTest servers? What is the format?
The data sent back to the CloudTest servers does not contain any test data – it only contains metrics around server resources used during the test. The data contains a timestamp of when the resource metric was measured, a description of what the metric is, which device the metric came from, and the value associated with the metric. For example: server1.test.com, 1/1/2009 3:45pm, CPU, 45%.
What operating systems and servers can be monitored? What metrics are available?
SOASTA has integrated some of the key monitored metrics necessary to help identify primary bottlenecks in typical web applications. Here is a summary of the core metrics monitored by SOASTA CloudTest:
Operating system and process information (Linux, Windows, Solaris, FreeBSD, and Mac OS X)
- CPU Percentage Utilization
- Disk IO Read/Write
- Memory Usage
- Network Receive/Send
- Per process information (CPU, Disk, Memory, Network)
- Process counts
- Any Windows performance counter
Application servers (JBoss, Oracle OC4J, Tomcat, Resin)
- Database connection pool size
- Database connection pool usage
- Database connections created
- HTTP/HTTPS thread pool size
- HTTP/HTTPS threads busy
- J2EE servlet requests
- Any numeric JMX attribute
- JSP requests
- JVM heap size
- JVM heap usage
- JVM thread count
- SNMP metrics
Databases (MySQL, PostgreSQL)
- Per catalog connections
- Total connections
- Active Requests
- Average database access time
- Average request queue time
- Average total request time
- Database requests
- Page timeouts
- Pages served
- Queued requests
Can custom monitors be added?
SOASTA CloudTest supports JMX, SNMP, and Windows Performance Monitor metrics. Most of these metrics can be added to the monitor and captured during the execution of tests.
Additional metrics can be retrieved with Custom Commands.
What commands are run on Linux to collect the monitored metrics?
These Linux commands are used:
SOASTA CloudtTest also uses the cat and grep commands to read the following files:
What Linux distributions can be monitored using the agent-less approach via SSH?
SOASTA CloudTest currently supports the Linux distributions below for monitoring system resource information. Due to the similarity of many Linux distributions, it is likely possible that other Linux distributions can also be monitored via SSH – without any product changes.
Supported Linux distributions for agent-less monitoring:
- Red Hat Enterprise Linux
- SUSE Linux Enterprise Server
- Oracle Enterprise Linux
What Linux user privileges are required to monitor via SSH?
A standard (i.e. non-privileged) user account is all that is needed; an administrator account is not necessary. Simply create an account using the ‘useradd’ command and assign a password to the account using ‘passwd’.
What happens to the agent once the testing is over?
When testing is complete, the CloudTest agents can be uninstalled from the servers or left installed for the next round of testing. If upgrades are necessary before the next round of testing, the agent will upgrade automatically the next time it is started.
How does SNMP monitoring work? Is an agent required?
SNMP traffic is on port 161. If this port is open directly to the devices being monitored, an agent is not required. Just like with other monitoring, however, if that port is not available externally, an agent must be installed on a server inside the firewall to gather these SNMP metrics.
How is a database monitored?
The database client for the monitored database must be installed on both the SOASTA CloudTest and the monitored servers. Connectivity to monitor the target server is done on the port used by the database client tools. The SOASTA documentation describes the individual database permissions that are needed for MySQL and PostgreSQL monitoring. In addition, for PostgreSQL there are some configuration changes that may need to be made.
Here are some links to more detail on how to setup monitoring of databases:
Can I monitor a Java application that is not J2EE-based?
Yes, if you are using a Sun JVM, and the JMX Remote Agent is enabled. See http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html for more info. Note that when you monitor a non-J2EE JVM, the only available metrics are Heap Size, Heap Usage, JVM Thread Count, and custom JMX attributes.
Can I use an RSA keypair instead of a user name and password to monitor via SSH?
Yes. Every installation of SOASTA CloudTest comes bundled with a 2048-bit RSA public key that you can add to the "authorized keys" list of any Unix-based server you want to monitor. Alternatively, you can use your own key pair by entering it into the Monitoring Server wizard. The bundled public key is available in the "System Resource Type and Authentication" step of the Monitoring Server Wizard.