PC Week Labs was impressed with the capabilities of the service, which allowed us to create test agents by recording our actions when browsing our sites. Doing this, we created regular tests of the core features of the sites and even tested the entire purchase cycle on an e-commerce site.
Evity, which launched SiteAngel 2000 last month, monitors sites from its office in Austin, Texas. The company provides remote monitoring and alerts for stoppages and performance drops.
However, the service's limitation in monitoring from Austin is one of the its biggest weaknesses. For example, any Internet problem related to Evity's area will cause inaccuracies in the data provided to sites using the service because the agents will report problems accessing the sites. According to Evity officials, there are plans to expand the service so that agents hit sites from several geographic locations, allowing sites to differentiate problems in the Internet from problems on their sites.
A service from Keynote Systems Inc. also provides geographic testing, although Keynote's reporting is mainly based on uptime rather than on detailed application performance. The company's service can monitor by geographical location or by backbone.
SiteAngel 2000 could be a good complement to a traditional performance testing tool, but it's a real question whether a company would want to add the considerable cost of the service once it had already invested in a full-featured application. In addition, it's only a matter of time until testing application vendors begin offering similar services that work in conjunction with their products.
SiteAngel 2000's lack of a realistic price scheme is a problem that it shares with many other ASP (application service provider) offerings. Depending on the frequency of scans and number of agents, businesses can expect to spend US$12,000 to US$850,000 per year on the service. This pricing is hard to swallow, especially when one considers that site testing applications range from US$20 to US$30,000, with many in four-digit territory.
In addition, although SiteAngel 2000 does regular testing over the Internet, tools such as RadView Software Inc.'s WebLoad and RSW Software Inc.'s e-Test can also perform scheduled tests and have the ability to launch thousands of simulated clients as well. With SiteAngel 2000, the agents are single clients testing a path through the site.
PC Week Labs tested SiteAngel 2000 by creating agents to test several key paths through the PC Week Web site and parts of ZDNet. Although most of the data during the weeks we tested the service was very consistent, we were able to get some differentiation due to the notorious DDoS (distributed denial-of-service) attacks that affected ZDNet, among other major Web sites, during the second week of February.
Creating an agent is a simple and straightforward process. Once we clicked on the new agent link, the service launched a separate browser window. We simply browsed the path or Web application that we wanted to test, and SiteAngel recorded the information. We then viewed and edited the agent's path.
After creating the agent, we assigned a monitoring frequency, which can be every 30 minutes, every 1 to 4 hours or daily. We set performance goals, which were the maximum time to complete the path, the maximum time to complete a step and the maximum time to resolve a problem. It is possible to send alerts via e-mail or a pager if a specific performance problem occurs. We received alerts in tests during the DDoS attack on our sites.
SiteAngel 2000 provides a variety of high-level executive and detailed technical reports, which are useful to business managers and IT staffers. Scheduled reports can be sent via e-mail on a daily, weekly or monthly basis.
A secure connection is used during log-in to SiteAngel 2000 but not throughout the rest of the service. We would like to have the option for a secure connection for reports, which can be considered competitive information.
SiteAngel can scan a site protected by either Secure Sockets Layer or by a password. It cannot scan plug-in applications (that is, ShockWave and so forth) and Java.
Senior Analyst Jim Rapoza can be reached at jim_rapoza@zd.com.
Executive Summary: SiteAngel 2000
USABILITY A
CAPABILITY B
PERFORMANCE C
INTEROPERABILITY B
MANAGEABILITY B
Evity's Web site monitoring and performance testing service is one of the few useful ASP offerings we've seen, providing constant, accurate and flexible testing of Web applications. The service could be a complement to performance testing apps, but its high price could make investment in both unpalatable.
Short-term Business Impact: The SiteAngel 2000 service can be up and running in less than an hour, and businesses should see meaningful reports in a week or two. The service's regular monitoring will let site administrators know when problems occur and also can help identify recurring performance problems.
Long-term Business Impact: The service will need to grow in order to provide sites with reports that differentiate Internet problems from site-based problems.
Provides detailed, agent-based tests of Web site paths and applications; good usability.
Only scans from Austin area; limited secure connections; high price.
