Buying the latest and the greatest sounds like a good idea, but who can afford it? We look at ways you can get better performance and a better bottom line with your existing infrastructure.

It seems a fact of early 21st Century corporate life that boards are reluctant to approve any expenditure that won’t be recouped quickly. In this environment, forklift upgrades can be hard to justify, so IT managers and CIOs need tools and services that will squeeze the maximum performance from existing systems.

Another way of keeping the bean counters happy is to be open to alternatives, or better still, to seek them out. Instead of throwing more of the same resources at a problem—more or faster servers, more disks, more bandwidth—look for other ways of spending your money that will give the same or better results at lower costs. Rather than buying more bandwidth, consider buying something that will let you maximise the value of your existing links.

We’ll explore both these avenues in this guide to making the most of what you’ve got.

Finding the real problem
It is important to realise that response time can be one of the most important metrics from a user perspective. The shift to server farms and similar arrangements for running hardware in parallel means a high level of availability can be achieved relatively easily, but users do not consider an application is “available“ unless the response time is acceptably short.

Tools such as NetIQ’s AppManager allow IT managers to actively monitor response times and quickly identify the source of any problem that arises. Mark Lazarus, senior consultant at NetIQ, stresses that it’s important to check application flows, not merely the state of the network. For example, mail servers at two sites might stop communicating even though the inter-site link is functioning normally.

Response time might be the key criterion, but it still important to monitor the performance of individual servers in a farm, he says, along with all of the subsystems involved in particular transactions (Web server, database, legacy system, etc) in order to locate the seat of any problem.

Information collected in this way can be used for capacity planning purposes. By determining the trends, managers can better predict when a major change will be necessary or appropriate, such as a migration from direct attached storage to a SAN. The finely grained data may also show that a relatively small change to a particular system—the installation of extra RAM, perhaps—will extend its life. It also provides a basis for identifying spare capacity, load balancing and server consolidation.

“When you see an organisation in pain with lots of [IT] people running around, it’s usually because the systems were implemented without management tools to control them,“ he says.

Detailed understanding and performance measurement lets IT “get on the front foot“ when dealing with users, Lazarus says, especially as it allows accurate figures to take the place of perceptions.

While some products and services home in on particular aspects such as network or storage performance, “it’s always the business application not the technology“ that should be the centre of attention, according to Graham Sowden, managing director of Mercury Interactive.

Mercury’s process begins with the preparation of test scripts that will simulate users, with different scripts for different tasks. These scripts are used to present a test load to the system during an off-peak period, and the effects are measured. A load-related problem is usually uncovered quickly, says Sowden, and the underlying issue corrected by the client’s staff or its other providers. For example, if a firewall were the bottleneck, the security team would be called in. The test is repeated to ensure the problem really was fixed, and then the load is stepped up to reveal the next problem. This process continues over as many iterations as necessary. “Finding the problem is 80 percent of the work,“ he says.

Problems might only occur under particular types of load, explains Sowden, which makes them hard to spot without systematic testing.

“Don’t measure the components, measure the end [result],“ he says, pointing to the example of a major Australian bank that spent six months optimising a database so that it ran at 50 percent of capacity rather than 80 percent, but with no effect on the end-user experience.

A major US parcel service wanted to deliver a superior online tracking service and estimated the system would need to handle 100 transactions per second (TPS). It took delivery of two large servers (with another two on order), but found they could only reach 25TPS, so it was apparent that the other two servers wouldn’t get them to 100TPS. Mercury tuned the system over 12 weeks, and achieved 85TPS from two servers. This was deemed sufficient, so the company didn’t need to spend US$1.5 million on the other two servers.

Sowden says the ability to repurpose existing infrastructure is important. Some parts of the infrastructure last as long as 15 years, but business drivers change so rapidly that scrap-and-replace isn’t an option. A CIO will typically add more servers and extra bandwidth, but that “doesn’t relate to what the business is trying to do,“ he says. By focusing on business-related performance, Mercury’s methodology can deliver 400-500 percent improvements without adding hardware or software, Sowden claims. If you can prove that the user experience stays the same when you remove an application server, you can save money on software licences and redeploy the hardware in another role (eg, as a Web server) that will improve the user experience.

Most of Mercury’s customers are large organisations with distributed IT environments, such as banks, telcos, and governments. “We [organisations] have installed millions in IT, we’ve got to get more out of it,“ says Sowden.

Where Mercury’s process could be characterised as incremental refinement, Compass Management Consulting uses a benchmarking model to compare clients with the best performing organisations in 30 countries. The company has developed well-defined, detailed models for all aspects of IT, according to managing director Rawdon Simon. For example, the comparison may reveal excessive staff in managerial or administrative positions.

Compass measures the performance of all its clients in unit cost terms, and finds its new clients incur costs 20 percent higher than its long-term clients do. “If you’ve got a budget of $20 million a year, 20 percent is a lot of money,“ says Simon, adding that the cost of the service is small by comparison.

Once the gaps between current and best practice have been identified, Compass uncovers the causes and makes recommendations. These comparisons might reveal that the client spends much less time in the analysis and design phases of development projects than the best performing companies do, or that it has greater excess capacity in its mainframe systems.

“Quite often, people have a perception about where the problem lies,“ says Simon, but if the comparison shows that perception was wrong, that “saves millions of dollars spent going down the wrong route.“

“We are advisory consultants, not implementers,“ he says, explaining that this arrangement allows the company to maintain its objectivity. Its reports include specific details of what needs to be done, but “we’re reluctant to recommend any particular organisation to do the work.“

Compass’s market is similar to Mercury’s: banks, insurance companies, governments, telcos, and major industrial companies—organisations with larger IT budgets have the most to gain, explains Simon.

Tune-up and oil change

Bryn Pears, managing consultant at Oakton agrees with Simon about the importance of getting to the bottom of the problem. “Application architectures have become vastly more complex,” he says, and unless you understand the system and identify the real bottlenecks “you’re fixing things that aren’t broken, or beefing up links that aren’t weak.”

A company operating an outsourced dispatching service consulted Oakton because its home-grown operations system was failing under peak loads. The problem was that the application was opening a new connection to an Oracle database for every request made by users. After tuning the Oracle configuration to make optimal use of the memory installed in the Windows NT server, 30 percent more connections could be opened simultaneously. Other changes were made to improve performance, and although a hardware upgrade was still required, the result was sufficient to keep the company running until the new equipment was commissioned.

Another Oakton client, a major energy company using Oracle Financials, provides an extreme case of the results that can be achieved with skilled tuning. Based on a subset of the data, the company had estimated that its fixed assets end-of-month reporting would take 32 days to process. The problem was that older versions of the Oracle E-business Suite “didn’t scale well” for very large numbers of assets, Pears says. By making adjustments such as ensuring that smaller indexes were searched before larger ones, Oakton was able to reduce the time needed to produce the reports to two hours. “Obviously the results aren’t always that dramatic,” says Pears, “five to 10 percent is more typical.”

“Often the first response . . . is beefing up the hardware, but hardware isn’t really cheap,” he says, suggesting that $15,000 spent on expert advice can be more effective than $150,000 worth of hardware.

But some organisations find it easier to justify the purchase of new hardware rather than expenditure on software or services, says Sal Ciardulli, systems consultant manager at Quest Software. For that reason, one company spent $500,000 on extra hardware to overcome what was really a tuning problem that could have been fixed for a fraction of that cost.

Don’t overlook the software vendor as a source for tuning advice. “Organisations [consulting firms] that implement the application for a certain amount of money ensure they make a profit,” says Ray Whitfield, director of Oracle Consulting. Oracle makes its profits from licensing the software, so it has to make sure it works well. “We’re a products company . . . we just happen to have a services arm.”

One telco had another organisation perform an Oracle installation, but wasn’t satisfied by the system’s performance. Oracle Consulting did a review, identified problems relating to the installation itself and a lack of staff training, and the company now acts as an Oracle reference site and is installing other modules. “Oracle stands by the client,” says Whitfield, pointing to five recent cases where the company has fixed inadequate installations done by others.

“You obviously can’t tune everything for maximum performance,” he says. The best strategy is to identify and address the most critical functions: trimming an hour from the time needed to run the payroll might not make any difference to the business, but instant response might be essential when dealing with a customer.

Oracle uses various tools to locate bottlenecks, but the experience of its staff in identifying trouble spots is also important. “Sometimes some code has to be written . . . sometimes in older applications some of the customisations are a problem,” says Whitfield, who explains that more recent Oracle products use a different mechanism for creating customer-specific functions so they do not impact on future upgrades.

Optimisation services

Tuning and optimisation services are sometimes available as add-ons to systems management contracts. For example, Dimension Data’s Insite for Applications Response is basically a monitoring service that compares the current performance of applications and infrastructure with agreed baselines. The focus is on specific applications such as Oracle database, Microsoft Exchange, and Microsoft Internet Information Server.

Karen James, operational services director at Dimension Data Australia, explains that software agents can be installed on all workstations to measure system responsiveness. In practice, only a representative sample is kept live and the remainder activated as necessary to establish the scope of any problems that arise.

The action taken when a measurement reaches a threshold level depends on the client’s requirements. If a user experiences a delay of say five seconds for a particular type of transaction, the client might choose to be alerted to the situation. If it reaches 10 seconds, Dimension Data might be authorised to identify the cause and either take action to correct the problem or manage a third party to fix it. “We have about 36 vendors we directly support” including Microsoft, Novell, Citrix, and Dimension Data’s own software, says James. Dimension Data also has partnerships to support mainframe systems and enterprise software such as Siebel and SAP.

Apart from ensuring rapid rectification, customers are using the information collected to extend the life of infrastructure by delaying technology refreshes until they are really needed and redeploying equipment for greater efficiency. “It’s easy to measure capex,” says James, “but the operating environment and operating expenses . . . are rarely measured.” Indicative ROI for managed application services is 103 percent for three year agreements and 108 percent for five years, based on a $500,000 investment, whereas spending the same amount on hardware only returns 14.1 percent or 40.2 percent over the same periods, she claims.

DIY optimisation
You don’t need to pay someone else to do the work for you. Some organisations prefer to licence the tools and do the work themselves. St George Bank uses MAINVIEW to automatically collect data about the operation of its mainframe hardware and software, and to model those systems in order to predict future requirements, says BMC’s Crawford-Smith. This process is faster and more accurate than manual data collection, he says, and it lets the bank “invest only when they need to”.

By monitoring services such as online banking from end to end, IT administrators can quickly identify and resolve problems that may occur, thanks to the detailed understanding that comes from the modelling process.

This understanding also enables active rather than reactive management, identifying opportunities for tuning particular applications or to relocate systems on the network to reduce traffic and thereby improve performance.

“Capacity planning is one of the key roles in IT,” says Crawford-Smith, and while it is more challenging in a mixed environment than with traditional mainframe systems, automated data collection provides a basis for more accurate choices.

Storage

Disk I/O can be a major bottleneck for CRM, ERP, and other database-oriented enterprise applications. In order to get adequate performance, it may be necessary to spread the data over a large number of drives, but that can prove very expensive if you end up buying much more storage than you need just to get sufficiently fast responses.

An alternative is to use solid-state storage devices such as Platypus Technology’s QikDATA. According to co-founder Geoff O’Reilly, solid-state storage is appropriate when you need fast, genuinely random access to small blocks of data. CRM is “a classic application”, he says, because call duration is normally a primary metric for call centres. Customers ringing in provide randomness, and only a small amount of data involved per transaction.

Either the entire database can reside on solid-state storage, or just the hot files (most frequently accessed data) with the remainder on conventional storage.

“It’s not hard to see a tripling of performance,” says O’Reilly, and some customers have reported a twenty-fold improvement, depending on the nature of the bottleneck, processor power, and other factors.

Solid-state storage looks expensive on paper—a 32GB QikDATA with battery backup and standby hard disks to ensure data is preserved in the event of a power outage costs around $100,000—you would need “100 to 200 spindles” of hard disk storage to get equivalent performance, says O’Reilly.

Energis, a major UK telecommunications and e-business solutions provider, was using 30 e-mail servers to handle 10 million e-mails per day. By replacing conventional storage with a pair of QikDATA units (two were used to provide redundancy, not capacity or performance), four dual-CPU servers were able to do the job of the previous 30, which could then be redeployed elsewhere.

External disk storage units may also offer the option of large amounts of solid-state cache memory. Some systems allow administrators to lock particular files into the cache so the data remains available for very fast access. Any changes are written back to disk in case of an outage. For example, Hitachi Data Systems’ (HDS) Lightning 9980V can be fitted with up to 64GB of cache memory.

Juggling data
Consolidated storage gets around the problem of having too much space attached to one server and not enough on another, but not every organisation is ready to move to a SAN. As an alternative to providing sufficient disk capacity to meet any spikes in demand, BMC Software’s PATROL Suite can be used to automatically move non-critical files from one location to another or begin an archiving operation when free space becomes short. According to channel sales director Terry Crawford-Smith, a mining company is using this capability with its SAP-based system to reduce the risk of system failure, as an outage can stop the movement of ships and trucks at the docks at a cost of $250,000 per hour. In these circumstances, even a 10 percent reduction in outages yields a worthwhile saving, he says.

Quest Software’s LiveReorg is another product that can help make the most of disk space by restructuring Oracle databases with virtually no downtime. One user, a Singapore utility, was able to free up over half a terabyte of previously unusable space, says Martyn Pregnell, managing director for Asia Pacific.

If you do have a SAN, HDS claims its HiCommand software will help you manage it better. Designed to integrate storage systems from different vendors into a single environment, it uses open standards such as SOAP and works with third-party software.

“Under-utilised storage systems can be easily recognised and reallocated to needy areas with the movement of a mouse,” according to a spokesperson for Hitachi. The Device Manager module provides a 10- to 20-fold productivity improvement over existing software, the company claims, while the Tuning Manager monitors and analyses SAN resources from a single screen and forecasts storage requirements.

Installing a SAN can be a big, expensive project, but HDS’s director of marketing for Australia and New Zealand Graeme Gleave says SMEs are easing into SANs. The first step is to separate the storage element of the next server purchase, and buy an external storage unit to be directly attached to the server. This only involves a small price premium, he says. Subsequent servers can also be directly connected to the unit (additional drives will probably be needed). Eventually, SMEs can buy a switch to interconnect everything, allowing for non-disruptive changes in future. At that stage, they should also buy appropriate management software such as HiCommand to make full use of the SAN.

“It’s a planning exercise—you have to sit down with the vendor or the vendor’s partner” to establish a roadmap, says Gleave. In any case, such projects “should be planned well in advance. These are not knee-jerk purchases,” he adds.

A progressive rollout is the strategy adopted by medical device manufacturer ResMed, which started by directly attaching an HDS Thunder 9200 storage unit to an AlphaServer cluster. The second phase saw the purchase of a pair of Brocade 3800 Fibre Channel switches interconnecting the Thunder, the cluster and three Sun servers. The next step will be to add ResMed’s Windows 2000 servers to the SAN, either by Fibre Channel or iSCSI.

Creative financing or leasing arrangements are an alternative to phased implementations as a way of getting a project past short-term financial hurdles, suggests Gleave. A SAN installation may offer a “fabulous” ROI over three years, he says, but if the board requires a measurable benefit from the first quarter the answer may be a smart leasing model that reduces the upfront costs.

Efficient backup

Tape libraries are commonly used in large organisations for backup and archiving, but the growing volume of data and the lengthening of the business day make it hard to complete overnight backups in the time available. The “obvious” solution is to add another library so that backup can be done in parallel. While tape has a very low incremental cost per gigabyte, a new library is a substantial investment and also has a relatively low utilisation.

Organisations are now backing up to disk to achieve significantly greater performance, but integrating that approach with an existing backup strategy can be difficult. Quantum has addressed these problems with the DX30, a disk-based backup unit that emulates a tape library and therefore works with current backup software and procedures, yet has a throughput of 80MB per second.

Once the primary backup is complete, the data can be spooled off to tape at leisure. And when a backup file must be recovered, there’s a good chance that it will still be on the DX 30 as 96 percent of all restores are performed within 90 days, according to Quantum’s product marketing manager Mike Sparkes.

Although data is stored in tape format rather than individual files, the DX30 is amenable to synthetic full backups, he says. This involves performing full backups at infrequent intervals interspersed with frequent incremental backups, and relying on software that’s smart enough to restore the most recent versions. That strategy is complicated when dealing with tapes, involving lots of tape swaps in order to restore a complete set of files, but works well in terms of storage efficiency and ease of restoration with a disk-based backup device.

With a price tag in the vicinity of $110,000, you wouldn’t regard the DX30 as a cheap alternative—but if you’ve maxed out the throughput of your existing library, a DX30 could save you spending $380,000 on an additional library with equivalent performance, so the term “relatively inexpensive” may be appropriate. If the 3TB capacity isn’t sufficient for your needs, the first quarter of 2003 will see the release of the DX100 with 10-15TB capacity and throughput in excess of 100MB per second.

“As applications grow, they drive more and more [batch] jobs that need to be done,” says Terry Crawford-Smith, channel sales director at BMC Software. He points to the experience of a Victorian power utility where overnight backups took around nine hours to process. Various reports and other batch jobs had to be run first, so if there were any delays the backup might not finish before staff arrived the next morning.

The obvious solution was to “throw more hardware at the problem,” but instead the organisation chose BMC’s SQL-BackTrack, which provides object-level backup and recovery. The combination of selectively backing up new or changed data and compressing it slashed backup times to around one third, Crawford-Smith says. Apart from the operational advantages, the organisation reduced its tape bill by $50,000 per year. The software was used in conjunction with CONTROL-M, another BMC product, which provides smarter logic, he adds, running jobs in parallel where possible and automatically restarting failed jobs.

This idea of a storage hierarchy can also be seen in other companies’ offerings. Rather than trying to keep everything on expensive high-performance storage, older (or less important) data can be migrated to slower, cheaper devices such as ATA disks, nearline storage such as tape libraries or DX30-style units, or even offline storage requiring operator intervention.

“Anyone that’s got large amounts of data that isn’t used very much after the first week or so” can benefit from hierarchical storage management (HSM) says Joan Tunstall, marketing manager at StorageTek Australia/New Zealand. While early targets for the technology included audio-visual material and scientific data, “it’s becoming more mainstream, driven by the fact that everyone’s experiencing a data explosion,” she says. One clearly mainstream application for hierarchical storage is e-mail. Mail volumes continue to grow, but requiring users to manage their own mail is a real time-waster, and also risks the inadvertent deletion of legally critical messages. “You get a clear payback” by applying HSM to e-mail, Tunstall says, because the productivity costs can be readily quantified. While she cites the annual cost of each mailbox at $5000 without HSM and $1200 with it, she admits “business cases built on individual productivity [depend] very much on what the organisation is prepared to concede.” Another example of the application of HSM to e-mail is Veritas Storage Migrator for Exchange, which migrates e-mail attachments from disk to tape.

If an organisation is facing data growth that in excess of storage budget growth (after allowing for price/performance improvements), HSM should be considered, says Tunstall.

Server consolidation

Faster storage isn’t the only thing driving server consolidation. As a general rule, the more servers you have, the more time and effort it takes to manage them. In the past, “one application, one server” was adopted as a rule of thumb in order to achieve the reliability needed for business-critical functions, but more modern operating systems and applications means this strategy is less appropriate.

Migrating from one application to another can dramatically reduce the number of servers required. Oracle is a big proponent of this approach, having saved $13 million in the first year and $11 million annually thereafter by replacing 97 e-mail servers and 120 message stores with a consolidated system with a two- (later three-) node cluster running Oracle9iAS Email fronted by 10 SMTP servers and three IMAP servers.

Apart from any advantages inherent in the technology, such large savings occurred partly because Oracle has around 50,000 e-mail users distributed across many time zones thereby providing economies of scale and flattening out the usage peaks.

Server consolidation will rarely fall into the category of “do more with what you’ve got” because of the need to maintain services while the new systems are commissioned, but it may make it possible to “do more within the budget you’ve got,” especially if you can show the savings on administration, maintenance, and licensing will quickly recoup the new capital outlay and transition costs.

But that problem can be minimised with the right tools. Quest’s SharePlex helps administrators consolidate databases “with minimal effect on the business,” reducing downtime from days to minutes even when different platforms are involved, Ciardulli says.

Network priorities
One way to get the most out of your network bandwidth is to ensure that the traffic is appropriately prioritised. Much of the attention given to devices such as Packeteer’s PacketShaper has concerned their use—particularly at schools and tertiary institutions—to suppress unauthorised traffic such as peer-to-peer file sharing software. For example, Chisholm Institute of TAFE in Victoria trialled a PacketShaper and according to network manager Sue Jones discovered “we had non-business-related things like radio and music channels, Kazaa, and Gnutella chewing up all of our two megabits.”

It is not only educational establishments that suffer from this problem. Simon Richardson, technical and communication services manager at Mercy Aged and Health Care had “applications grinding to a halt for no apparent reason on links that should have been sufficient”. The organisation has 18 sites connected to a central hub, and users were calling for faster links. A protocol analyser didn’t help pin down the problem, and network probes were too expensive at around $25,000 per subnet. Richardson installed a PacketShaper: “we bought it, plugged it in, and it worked . . . [we] began fixing problems in the first hour,” he says. The PacketShaper revealed MP3 downloads, filesharing, games, video streaming, chat, and unauthorised webcams were consuming bandwidth. “No matter how much bandwidth I had thrown at some sites, we never would have got the application performance we needed,” he says. Upgrading just one link to 2Mbps would cost $20,000 per year. Instead, Richardson spent a similar amount on PacketShapers for the hub and major sites, and only increased bandwidth where required to support additional services to particular sites.

Even in situations where all the traffic is bona fide, packet shaping can be used to ensure that the most important and time-critical applications get the bandwidth they need when they need it.

An example of this can be seen at AusBulk, a South Australia-based bulk handler, where a WAN connects the head office with 22 bulk handling facilities in South Australia, New South Wales, and Victoria. By using PacketShapers to reserve bandwidth for its operational management and financial systems, AusBulk’s annual savings are around $40,000 in bandwidth costs and $25,000 in overtime payments to staff.

“It simply wasn’t viable to throw bandwidth at the problem. Bandwidth is at an absolute premium in these remote areas. It becomes very expensive when you are only operating at maximum capacity during three to four months of the year,” says technical services manager Paul Bassett. “If staff cannot access the [operational management] system in a timely fashion, our people in the field have to work in shifts virtually 24 hours a day during peak times such as the annual grain harvest, just to enter operational data.”

“PacketShaper is a must-have technology for anyone attempting to deploy time critical applications across a congested network to remote sites,” he adds.

“The AusBulk deployment is significant for Packeteer because it demonstrates one of the great strengths of the PacketShaper product line: quality of service,” says Bob Jones, Australian territory manager for Packeteer. “Deploying PacketShaper means AusBulk can rely on its core operational systems in remote areas where continuing to add more bandwidth would be prohibitively expensive. It’s about controlling network traffic instead of having your network traffic control you.”

Software works too

But installing an appliance isn’t the only way to control network traffic. Adelaide-based Foursticks offers a software-based alternative called Foursticks NP, allowing users to choose the hardware to suit their needs for throughput and availability. In the longer term, Foursticks visualises end-to-end deployment, right down to handheld devices.

According to Alisdair Faulkner, vice president of marketing, Foursticks NP delivers higher resolution monitoring and reporting than competing products. “Our customers find the at-the-second monitoring essential,” he says.

The software dynamically controls bandwidth usage in reaction to network conditions, and its patent-pending algorithms provide more accurate and more efficient management of interactive and real-time applications, says Faulkner. The result is “more efficient use of network resources, better application stability, [and] more precise configuration,” he says. This ability extends to managing variable speed networks such as frame relay, as Foursticks NP can detect the availability of bandwidth above the committed rate.

One problem with many types of network controls is that setting incorrect policies can severely impact performance, sometimes bringing the network to a halt. Foursticks’ patent-pending methods for checking policy correctness overcame this problem, making the system easier and safer to use. The cost of implementing Foursticks NP is typically returned within six to 12 months, according to Faulkner. One customer—Bridgestone—reported an immediate saving of $100,000 in bandwidth costs, he says.

“The forklift approach to the [network performance] problem—bringing in truckloads of bandwidth—isn’t even a bandaid solution,” says Martin Sheahan, Foursticks state manager for Victoria. “What the Foursticks NP software does is create express lanes through networks, and when you talk to enterprise customers, they get the idea immediately.”

The product has only been shipping for a few months, but customers are reporting good results.

“We run a truck every 10 minutes in the early stage of vintage. We need things to happen quickly and consistently,” says Doug Wheatley, network administrator at winemaker BRL Hardy.

“We can ill afford to have trucks backed up . . . we have customers phoning in orders every minute, and we need that to happen instantly.

“On one occasion we had a user send a 20MB PowerPoint presentation to a small site operating on narrow bandwidth and critical business activity was severely hampered.

“We found that putting in an NP resolved these problems, with sub-two-second response times for all data over to remote sites such as Berri.”

Optimising code
While the build-or-buy pendulum has been pointing towards “buy” for the last several years, there are some suggestions that it is starting to swing back and in any case there have always been some jobs that required custom development. How can you be sure that such software was implemented efficiently?

It’s not as if you can rationalise ignoring suspected inefficiencies by arguing that it’s cheaper to add an extra server than tune the code. “A lot of Java development is done [for] the server side . . . and application server licences aren’t cheap—you can be talking about $50,000 to add a processor,” says Mark Foley, senior Java architect at Borland.

One project performed by Borland’s professional services group involved software that was required to support Windows, Mac OS, and other operating systems. That’s a situation where Java is supposed to excel, but a problem arose because many of the Mac users had old, slow hardware and weren’t getting satisfactory performance. By using Borland’s Optimizeit, Foley identified over 20 areas of the code that could be optimised, and in two days reduced 10-second delays to an acceptable one second.

While you don’t always get such a big speed boost from using Optimizeit, it’s rare that you don’t see a worthwhile improvement, he claims.

“It’s easy to make a mistake” when writing code, he says, but the problem is that such mistakes can be buried under millions of lines of code. That said, performance problems do not always relate to particular lines of code but in memory management. “A poorly tuned application [may spend] 25 percent of the time doing garbage collection,” he suggests. This makes Java tuning quite different to C or C++ tuning, he says: “It’s quite hard to see what’s happening without a tool.” In this situation, adding another processor to the server might not help, but adding memory would. “The way Java manages memory is absolutely key in a Java environment,” Foley says. You need to be able to look inside the JVM to tell what is happening, and the next release of Optimizeit will have more features to address memory-related problems.

According to Foley, the 80:20 rule applies to Java software: 20 percent of the code usually accounts for 80 percent of the execution time. The situation is more extreme with application servers, which spend little time executing application code. “It can be very difficult to optimise code” in these circumstances, says Foley, so what you need to do is examine how your program is interacting with the application server. “A lot of the issues are database related—identifying them is a key problem,” he explains. Optimizeit is good for finding out which database queries take a long time to answer.

Application server configuration is another issue. A client was about to purchase an additional server in an attempt to overcome a performance problem, but an hour’s work with Optimizeit showed that the configuration was preventing the use of all the server’s memory. “It’s just a tool. It’s a good tool, but the result relies on the skill of the person using it,” says Foley.

So why don’t all developers use code optimisation tools? Foley says it is an increasingly common practice, but such tools cost money and optimisation sometimes gets a lower priority than other aspects of the development process. However, it’s much better to do any optimisation work before testing and deployment—it should be “the fence at the top of the cliff, not the ambulance at the bottom,” says Foley. Another reason for doing tuning during the development phase is that once an application is in use, feature enhancements tend to take priority over tuning, says Quest’s Pregnell.

Following its recent acquisition of Sitraka, Quest now has tools to help diagnose and tune J2EE environments. JProbe and PerformaÃ,­Sure allowing customers to identify potential performance problems down to a specific line of Java code. “By combining Quest’s existing database and application performance management solutions with Sitraka’s in-depth Java expertise, we will be better positioned to monitor current Java applications and the next generation of packaged applications,” says Pregnell.

Just as packet shaping allows IT managers to control the bandwidth available to particular applications, other tools are available to control programs’ CPU usage. An example is ONEAPP CPU SeNTinel, which runs on Windows NT or 2000 Server, Terminal Server, and NT, 2000, or XP Workstation.

According to distributor Infotronics Software, “a single user opening an application can take processor usage to 100 percent; the result can be significant across the enterprise, machines can lock up or run very slowly.” Capping the CPU usage of individual programs prevents them becoming processor hogs and ensures more consistent across the board operation. “Applications that have sucked up so much CPU resource that they have required their own server may be run in conjunction with other applications, thereby decreasing hardware and operating systems costs,” according to a company spokesperson.

Thin client

Some organisations have succeeded in extending the life of desktop systems by switching to the thin-client computing model, even when bandwidth is limited. One example is Owens Container Services, which operates from over 15 sites on the east coast of Australia, plus New Zealand and Fiji.

IT Manager Ross Pavey says, “If we’d gone ahead with a straightforward upgrade to the separate networks and client machines, we would have had to spend around $190,000.” Instead, the company built a Citrix MetaFrame server farm from nine existing Compaq systems so applications could be run centrally. Communications with remote sites was achieved by piggybacking on a frame relay network being installed by another division of Owens.

Apart from extending the life of the desktop systems, Pavey and his small team are able to install new applications with greater ease. “If we had to install a new application to all 100 desktops, we would need to have someone install the program on each client machine in three countries. Now it’s simply a case of install it once on a MetaFrame server.”

Owens is following a strategy of growth through mergers and acquisitions, and the MetaFrame strategy makes it much easier to integrate new parts of the organisation.

“With MetaFrame and NFuse Classic, we’re able to bring new business units online within a day. Even in cases where the new business unit doesn’t have Frame Relay access, they can get online to the network over the Internet via Citrix NFuse,” says Pavey.

Web veneer
Another life-extending strategy, this time for legacy systems, is to put a Web front end on the application. This enables the use of old mini or mainframe software without the need for a terminal emulation program on each PC. It can also reduce the need for training, and (depending on the licence terms) may increase the number of people that can use the application at little or no extra cost.

WRQ’s Reflection for the Web provides Web-based terminal emulation, avoiding the need for emulation software on individual PCs, but the traditional green screen look remains. If that’s becoming alien to most of your users, Reflection Webfront transforms such screens into Web-style forms, consolidating data from multiple screens if necessary.

If you need to go a step further and consolidate functions and data from multiple systems, the same company’s Verastream converts host functions into objects that can be used to build composite applications. For example, Singapore-based Tenet Insurance took a 12-year-old insurance application running on OpenVMS and built a self-service Web-based system that allows agents and actuaries to obtain the information they need, reducing operating expenses by 30 percent. Although the logic and data is provided predominantly by one host system, Verastream made it possible for Tenet to provide users with a more intuitive workflow without changing the underlying application.

These examples are by no means a complete list of the strategies to help make your IT dollar go further. We’d love to hear from you about your own experiences in making the most of the infrastructure you’ve already got—e-mail us at edit@zdnet.com.au

Case study: getting in tune

Anand Ramachandram, director and senior vice president, information services, Asia Pacific at T D Waterhouse, believes tuning is an ongoing process that needs to start from a known point.

-It's best to gather the results and start with an outsourced service," he says, -you want a professional opinion on where you are today." Then you can gather the tools needed to track your performance against that baseline. -A lot of people jump the gun" and don't establish the baseline or understand their problems before trying to make improvements.

T D Waterhouse consulted several companies to establish its baselines: Dimension Data (network), Mercury Interactive (Web site), and Enstor (storage). Tremendous improvements have been made in the year or so following that exercise, Ramachandram says. For example, the company had budgeted $600,000 for network improvements, but the analysis showed that was overkill and the actual expenditure needed was around 45 percent less.

The savings arose because the process revealed where problems were andââ,¬"just as importantlyââ,¬"where they weren't. For example, a network problem was cured with $50 worth of new cables instead of a new router as originally planned. -Small tweaks here and there" fixed a variety of issues, he says.

While T D Waterhouse was expanding rapidly, Ramachandram was considering EMC storage systems to help cope with the growth in data. When that growth slowed, he instead reviewed the utilisation of the existing disks. -We had to do a lot of cleaning up," he says, but by rearranging drives and rebuilding various servers the company avoided a major investment in new storage.

He recommends -a disciplined approach, looking at the bottom up for everything." It's important to approach optimisation in a stepwise fashion, because if you make ten changes at once, you can't tell where the benefits came from.

Ramachandram's experience is that it is important to take time evaluating tools for measuring software performance. It is important to understand how they interact with proprietary programs so that the results are analysed correctly. -It's best to be more conservative with time, and more aggressive with the budget."

Prioritisation is important, but for Ramachandram that doesn't mean starting with the jobs that promise the biggest bang for the buck. He advocates doing the simple housekeeping tasks first and moving on from there, partly because projects that promise big returns don't always work out that way. He admits this strategy isn't easy: -it's a very hard thing to do when you're a senior IT executive" as you must motivate staff to find time to do the boring tasks while keeping other projects on track. Ramachandram recommends involving staff in the plans and after three to six months they will see the results and draw satisfaction from the difference they've made. -Developers are the hardest to convince," he says, -but it can be done."

While his strategy is to do the work in-house to avoid becoming dependent on a third party, Ramachandram will get an outside opinion of his team's progress every couple of years to check he is still on track.

One difference between Australian and US businesses is that US companies are more likely to compare their performance with organisations in the same industry, whereas Australian companies generally compete on their own track record says Ramachandram, who previously worked for T D Waterhouse in New York.

Subscribe now to Australian Technology & Business magazine.

Copyright © 2009 CBS Interactive, a CBS Company. All Rights Reserved.
ZDNET is a registered service mark of CBS Interactive. ZDNET Logo is a service mark of CBS Interactive.