E-mail security services square off

E-mail is the primary conduit of information for many organizations, both internally and with the outside world. Unfortunately, e-mail is also a prime channel for annoyances such as spam, as well as security threats in the form of viruses, spyware, phishing attacks, and more. Some companies choose to defend their e-mail systems in-house by deploying e-mail security products. Others, however, look to outside assistance in the form of hosted e-mail security services.

As a follow-up to my review of nine e-mail security appliances, I looked at six hosted e-mail security solutions from AppRiver, MessageLabs (now owned by Symantec), Microsoft, MX Logic, SECNAP Network Security, and Trend Micro. Postini (now owned by Google), another major player in the hosted e-mail security space, declined to participate.

[ Compare these e-mail security services by features. Compare their spam filtering results. See the Test Center guide to e-mail security appliances including reviews of offerings from Barracuda, BorderWare, Cisco, Mirapoint, Proofpoint, Secure Computing, Sendio, Symantec, and Tumbleweed. ]

Before delving into the review itself, however, it's useful to understand just why an organization might want to hand over e-mail-security duties to a third-party provider. After all, many IT managers may be uncomfortable with having a critical application such as e-mail outside their control. However, hosted e-mail services hold several advantages over their in-house counterparts. There are drawbacks as well.

Appliance versus services

One advantage that a service holds over an in-house solution is that when you're using an outside provider, the volume of mail coming to your internal network is greatly diminished -- by 80 to 90 percent in most cases. Moreover, because you don't receive the unwanted mail at your location, you have no need to worry about archiving it. Such might not be the case if you're doing the filtering in-house.

A second advantage of a hosted service is that most providers have more robust networks than even large organizations, with multiple sites that have at least two separate Internet connections and multiple servers. Thus, going with an outside provider greatly decreases the chances that your e-mail service will be unavailable.

Third, hosted services offer a buffer for your e-mail system. If your internal e-mail server fails or if your Internet connection goes down, your mail will continue to accumulate on the hosted service's server until your in-house problem is resolved.

Yet another point for hosted services: They offer all features you'll find in appliances, such as content management, but they also boast services that don't come with an in-house box, including archiving, disaster recovery, and encryption. Bear in mind, however, that if you're going to implement certain features among internal users -- for example, to enforce policies on e-mail content between users in the same department -- you'll have to ensure that all messages are forwarded through the hosted service. This can be complex to set up and could increase delivery times.

E-mail security appliances aren't without their advantages over hosted services, however. Take, for example, directory synchronization. If you want to ensure that e-mail addressed to invalid users is turned away (which you should), you need to export your Active Directory information or user information from another source, be it LDAP, NIS, or something else. Although this is easy to accomplish with appliances, it becomes more difficult with a service. There are two alternatives: Option one is to open a port in your firewall for LDAP (usually port 389). This creates a security hole, however, to which your network admins may object. Option two is to export the data using an application provided by the hosting service. Although I was able to get this process to work during my testing, it took much longer than it did with my previously tested appliances -- up to a couple of hours more.

There's another important drawback to hosted services: When you sign on with one, you'll need to change your DNS records so that mail addressed to your domain goes to the service rather than your internal mail server. The service then forwards the non-spam to the internal server. Any e-mail server that performs a DNS lookup before sending mail to your users should be going with the new address within 72 hours; however, some servers, both spam and legitimate, send messages directly to an IP address and don't resolve the hostname beforehand. These e-mail messages will continue coming directly to your e-mail server, bypassing filtering, unless you configure your firewall to block all incoming e-mail from addresses other than the service. The problem here is that some of the services have multiple IP addresses from which e-mail may be sent, and depending on the firewall, setup can be complicated.

The final drawback with services lies in how user accounts are set up. Users must access the service Web site to view the quarantine, from which they can release messages and (in most cases) whitelist or blacklist senders. Some services can pull account information from Active Directory so that the user logs in with the same password recognized by his or her Windows Domain account. Others offer self-enrollment, forcing users to create an account the first time they log in.

By contrast, appliances generally work with plug-ins to Outlook so that users can review the quarantined messages within their familiar e-mail app or via a local Web site that takes the same log-in and password as their standard Windows account.

E-mail security at your service

I tested the six services in this roundup with a real e-mail stream over 15 days, averaging 16,000 to 19,000 total messages. Of those, about 2,500 were legitimate. The services tracked all incoming messages; thus, I didn't witness the reporting disparity I saw with appliances. How each service counts messages does vary slightly, however. The greatest variable is the number of messages assumed to have been delivered per connection. If a mail server connects to your domain and sends an SMTP message, it may be for one user or multiple users. Most reporting tools assume a message count higher than one, but the actual number assumed varies.

Comparing the filtering rates among the services is not terribly important: As you can see from the results table, they all scored between 94 and 98 percent. (That figure might be more for users with high volumes per day.)

However, the numbers of false positives (legitimate messages mistakenly marked as spam) is far more important to monitor: If users don't trust the system to forward all their important mail, they'll spend more time perusing the quarantine than they would have spent dealing with the full volume of spam. For my testing, most of the false positives were bulk mail, either newsletters or marketing e-mails from legitimate senders who were given permission to send e-mail to the test account. Critical false positives were messages sent from a single legitimate user via a normal e-mail program, which shouldn't ever be identified as spam.

As noted, these services don't just squelch spam. In my testing, they all succeeded in stopping viruses. Anti-phishing was less reliable, with only 60 to 80 percent of phishing messages identified as such, although most were stopped as spam. Features such as archiving, content management, and data recovery all worked as advertised.

When all was said and done, MessageLabs came out with the highest marks in my tests, not only because the service's false-positive performance was best overall, but because it had no critical false positives. Its overall feature set, ease of use, and interface all contributed to the win as well. The second- and third-place vendors, respectively Microsoft and Trend Micro, each had one critical false positive, along with slightly higher overall false positive scores. In the real world, the differences between first and third are nearly indistinguishable. To its credit, Trend Micro is much less expensive (US$2.16 per month per user versus $1.60 per month per user), and it offers more features at this price point, along with a guaranteed SLA (service level agreement).

AppRiver SecureTide

The SecureTide services offer a decent number of features, including spam filtering, virus filtering (drawing on four systems), content filtering, and unlimited queuing. Though easy to set up and use, SecureTide proves lacking in some areas. General spam-stopping performance, for example, is at the bottom of the six services I tested, although still acceptable, with 95 percent of spam blocked. It also suffered 94 false positives and three critical false positives (see test results). The policy engine isn't as robust as some enterprises might like. Also, the service has some irritating quirks, such as requiring admins to whitelist messages one by one.

SecureTide proves simple to set up, as do all the services in this roundup. When setting up users, you have a couple of options: You can import users from Active Directory or another LDAP directory, or you can enter the information manually or through a comma-delimited file.

Once the service is configured, a held mail report goes to each user. This leads to one of the shortcomings of the service: If a user discovers a false positive, he or she must request whitelisting. Each request goes to the exception requests filter for the administrator to review. If approved, the requests are added to the whitelist. This means that the admin must review each and every whitelist request separately. If you take the 94 false positives I got during the first two weeks of testing and multiply it by several hundred users, you're looking at an inordinate amount of the admin's time during the first few weeks.

Both the admin interface (screenshot) and user interface are clean and easy to use, with drop-down boxes rather than text boxes for specific entries. The administrator can copy messages, bounce them, add an identifier to the subject line, delete, forward, allow, or hold (quarantine) spam. The service allows admins to set policies for individual users, but not the creation of groups.

The Web console used for setup and maintenance does have one annoying feature: If you leave it open, it times out after a short while. When you click on a link, you don't get a message saying the console has timed out, nor do you get a log-in box. You just get a message: "Login Failed: Username not found."

Pricing for SecureTide starts at $1.50 per mailbox per month and includes 24/7 U.S.-based support. The first month is free. Emergency E-mail Service (EMS) provides a backup e-mail server, either POP3/IMAP or Exchange Hosted Service on demand if your on-site server fails, for an extra 50 cents per user per month.

MessageLabs E-mail Anti-Spam and Anti-Virus Services

The MessageLabs service had the best performance in my testing, catching 97 percent of incoming spam with only eight false positives and zero critical false positives. It also boasts a stellar feature set, including anti-virus, control over images received, excellent administrative controls, a self-service portal, superior auditing tools, and the best LDAP synchronization software of the test. All of this comes at a price: The service runs $2.16 per user per month at 1,000 users, a little more expensive than the others in the roundup. Notably, MessageLabs was acquired by Symantec during the course of this test, though the brand will be retained and the service will remain the same.

Setting up the MessageLabs service is generally quite straightforward. The Active Directory/LDAP synchronization software works easily without requiring IT to open a hole in the firewall or export files to a comma-delimited format. However, the log-in creates a random user name, not username@domain or anything memorable. In my case, it was MED8559, and neither IE nor Firefox recognized or remembered the log-in. The unalterable password policy is also incredibly irritating: Passwords must contain capitals, lowercase, numbers, and symbols, which makes them difficult to type and nearly impossible to remember. Thus, many users will either write down log-ins and passwords on yellow stickies or call IT because they've forgotten them.

These unalterable security measures are annoying, especially in contrast to the service's policy engine, which is flexible in every way. Admins (screenshot) can, for instance, devise policies and settings by time of day, group, user, and more.

In addition to best-of-class anti-spam performance (see test results), MessageLabs offers a porn detection capability, scanning images for "excessive" bare skin, number of people, and such.

Additionally, images of photos or documents can be uploaded and used to create a signature to ensure that specific intellectual property is not sent or received without authorization. For example, if you have a confidential document titled ProprietaryInfo.doc, you can create a signature of that file that records not only the name by its length, content, and so on. MessageLabs will detect and stop any attempt to send that document to an outside recipient.