The release of Service Pack 2 for Windows XP will mark a milestone in the life of this operating system. Microsoft is pulling out all the stops to improve security. So much so, in fact, that it will cause many problems -- because SP2 will de-emphasise backward compatibility with legacy systems and code for the sake of security. Administrators need to know in advance just what SP2 will mean for the Windows XP systems on their networks.

Here are some of the most important security changes that are part of XP SP2:

• The Internet Connection Firewall is now enabled by default, which should improve security for SOHO users. However, in a corporate environment it could cause problems for users trying to connect to network resources. The firewall will also now activate much earlier in the boot cycle, even before the network stack is enabled. On shutdown, it will now remain active until after the stack is disabled.
• The Messenger service is now disabled by default.
• A pop-up ad blocker has been turned on by default.
• A unified security application called the Windows Security Center has been added (for more information on this feature, see this News.com article). It is supposed to bring all of the most basic security configuration information into one easy-to-manage place that will show whether your firewall is enabled, if your antivirus software is working, and if you have the latest software updates installed.
• NX support is added to Windows XP. NX (no execute) will allow NX-enabled CPUs to mark certain areas of memory as non-executable; that is, any code pushed into those areas (perhaps by malware such as Blaster or other viruses) will just sit there, unable to run and therefore will be rendered harmless. This will harden the OS against the notorious buffer overrun threats. NX is currently only supported for AMD's K8 and Intel's Itanium processors, but 32- and 64-bit support for this important security feature is expected in most future processor releases.
• DCOM (the Distributed Component Object Model) gets a new set of restrictions in the form of an access control list for nearly every action of any COM server. There will also be a more detailed set of COM permissions, which will allow administrators to fine-tune COM permission policies.
• There is improved port management. It will no longer be up to the application to close ports after it is finished. Before, if a developer left out the closing routine or the application crashed, a port could remain open and leave XP open to attack. SP2 encourages port management with an application white list that only a user with administrator privileges can alter. Placing an application (such as a peer-to-peer program) on the white list causes ports to be managed automatically. Such applications can also now be run as a regular user rather than needing local administrator privileges to open ports in ICF.
• New RPC restrictions help tighten communications. The XP SP2 changes in this area let administrators fine-tune RPC services. This granular control over RPC will allow you to specify that a port be used for RPC even if the application is not on the white list. There are a lot of changes for RPC, including a new RestrictRemoteClients registry key that by default blocks most, but not all, remote anonymous access to RPC interfaces on the system. The RPC interface restriction will require an RPC caller to perform authentication, which makes it much more difficult to attack an interface, and helps mitigate against Trojan attacks.

RPC changes are the most likely to wreak havoc with existing applications. In the pre-SP2 Windows XP implementation, there are literally scores of RPC-based services running, all of which provide a window for attack. That changes dramatically with SP2.

Page II: Learn about the plethora of security enhancements that Microsoft has included in Windows XP Service Pack 2, as well as how these security features could impair the functionality of some applications.

Because of the change in port management, if an application needs to open ports but doesn't use stateful filtering, administrators installing it need to place the program on the white list. With the built-in firewall enabled by default, IPv4-application inbound connections for audio and video, such as for MSN or Windows Messenger, need to have their port opening and closing managed automatically. Inbound services connections (IPv4) will require some changes to configuration and/or code. Services that listen on fixed ports should ask users if the service should be permitted to open the port in ICF and, if so, the service should use the INetFwV4OpenPort API to alter ICF rules.

Another problem is the fact that Microsoft won't be offering this service patch to those who hold pirated copies of Windows XP, which is reasonable enough, but there are a lot of illegal copies out there, especially in the Far East where a lot of worms get a quick foothold in the Internet. SP2 will apparently check Product IDs looking for known pirated copies and will not install on systems with bad Product IDs. This is understandable, but will reduce the overall effectiveness of the security upgrade.

A lot of the potential problems posed by SP2 are beyond the control of administrators. Some programming code for custom applications will have to be rewritten, but at least now you know what to look for when problems come up, rather than deploying XP SP2 and finding out that it breaks your most important line-of-business application.

• XP SP2 Preview
• XP SP2 RC1 Fact Sheet
• XP SP2 RC1 Release Notes
• XP SP2 Developer Notes

Nearly every security expert knew that, at some point, Microsoft would be forced to bite the bullet and take a big compatibility hit in order to solidify operating system soft spotsââ,¬"many of which are due to legacy code support. Plus, the XP SP2 changes will force developers to produce more secure applications and not just take advantage of a permissive Windows OS to write code that doesn't pay attention to security.

Of course, I would never recommend that anyone deploy such a major upgrade widely the day it hits the street. You should install SP2 on a testing network (or at least a single testing system) as soon as possible, and begin compatibility testing for your specific applications.

Those of you who have the budgetary luxury of being able to conduct even more extensive testing and want to get a leg up on evaluating XP SP2 even before final release should check out the Technical Preview Program, which makes SP2 RC1 generally available for testing by IT professionals (not just those on the beta list). The initial download, which doesn't include any support other than some Microsoft-sponsored newsgroups, requires Windows XP to be installed already. English and German versions of the update are now available and are about 270 MB in size.

As soon as you feel comfortable that Windows XP SP2 will not cause a significant interruption for users (or you have fixed the issues that would lead to a potential interruption), then you should deploy SP2 company-wide. It is an important upgrade that can only improve the security of your network.

TechRepublic is the online community and information resource for all IT professionals, from support staff to executives. We offer in-depth technical articles written for IT professionals by IT professionals. In addition to articles on everything from Windows to e-mail to firewalls, we offer IT industry analysis, downloads, management tips, discussion forums, and e-newsletters.

©2004 TechRepublic, Inc.

Copyright © 2009 CBS Interactive, a CBS Company. All Rights Reserved.
ZDNET is a registered service mark of CBS Interactive. ZDNET Logo is a service mark of CBS Interactive.