The scope of the problem is that blacklists--one of the most widely deployed solutions to spam--are ineffective at blocking spam and they promulgate false-positives that result in legitimate and important e-mail never reaching its intended destination.
Blacklists--a fancy name for pre-determined filters--typically live in one of three places. First, some are replicated into the databases of ISPs that subsequently refuse any e-mail traffic that satisfies the criteria of the blacklist. These are most often founded on a community-based vigilante system that identifies spam sources by the Internet address of the system from which the mail is transmitted.
Second, blacklists are deployed for e-mail servers that service users of business and public e-mail systems. Generally, these blacklists are formulated on spam forensics, which attempt to identify the offending e-mail based on examining the source and destination addressing as well as analysing the header and content of the e-mail.
Third, blacklists reside in client systems as plug-ins or tools for e-mail clients like Outlook and are also based on generating lists of culprits using the spam forensics.
Unfortunately, filtering is not a black or white issue. The more aggressive a filter is, the more likely that innocent e-mail will be treated as spam. The less aggressive is the filter, the more spam squeaks through into our inboxes. As a result, end-users and e-mail administrators end up without a reliable solution. We get angry when spam fills up our inboxes, exposes material that we'd otherwise censor, and eats up valuable network and storage resources. Unfortunately, we also lose our cool, or maybe a job or some business, when our e-mail doesn't make it to its final destination.
This intolerable situation calls for a new approach to the problem. It's time for the same technology and service providers that made spam possible to put their heads together to figure out how to make it impossible or at least improbable.
Any solution will probably have something to do with whitelists. Unlike blacklists that are based on exclusion and result in false positives, whitelists are inclusionary, focusing on certifying legitimate e-mail sources.
So important are whitelists, Scott Banister, founder of bulk e-mail appliance maker IronPort Systems, has started a whitelist initiative called Bonded Sender. Currently in a beta phase, Bonded Sender is not Banister's brainchild to combat the scourge of spam. With high profile clients like CBSMarketWatch, MTV, and The Motley Fool relying on IronPort technology to send, in aggregate, close to a billion e-mails per week-- with many of those e-mails wrongly being denied safe passage--Banister realised that the blacklists were threatening the viability of his business. His perspective is not unlike my contention that blacklists, as they're currently deployed, could help destroy the Internet (or at least the e-mail part of it).
While Bonded Server is self-serving to the extent that it deals with a threat to IronPort's business, participants need not be an IronPort customer. That's because in order for the whitelisting principle behind it to work, Bonded Sender requires a critical mass that exceeds the IronPort's current base of 80 customers.
So, how does Bonded Sender work? Philosophically, it's similar to the application of digital certificates that I suggested in my previous column. In both cases, a degree of accountability is placed upon the senders of e-mail. Senders who meet some criteria earn the privilege of being on the whitelists , which are deployed in the same places that blacklists have been inhabiting. A breach in that responsibility would result in consequences to the sender.
In my digital certificate proposition, the consequences for abusing the system would be removal from the whitelist. In other words, e-mails carrying specific digital certificates would be removed from the whitelists, and therefore would no longer be guaranteed safe passage. With digital certificates employed as the unique identifier, the subjectivity and problems associated with spam forensics are removed from the process
In the Bonded Sender world, legitimate bulk e-mailers like MTV, CBSMarketWatch, or even ZDNet would put up as much as a US$100,000 bond. Upon receiving spam complaints, the initial consequences would be financial in nature. The financial penalty would depend on how many complaints are received.
For example, if through the vigilante system that's currently in place, 10 complaints are received for every 10 million e-mails, a senders bond would be docked a nominal fee of one dollar per complaint. But it could scale up from there. Ten to twenty complaints would result in charges of $20 per complaint. On the top end, 10,000 complaints per million e-mails (1 percent) could cause the forfeiture of the entire $100,000. Any proceeds from such penalties would flow to non-profit causes like e-Trust that help combat spam.
The goal of the program, according to Banister, isn't to punish spammers. It's to create a list of legitimate bulk e-mailers that ISPs can put some faith in. Banister's thinking is that if companies put up a sizeable bond, then ISPs would be willing to incorporate IronPort's whitelists of bonded senders into their reporting and filtering algorithms in a way that makes blacklist exceptions for senders on the whitelist. IronPort will even provide the PERL scripts that are necessary to implement the whitelists free of charge to the ISPs. Ultimately, this would help Banister's customers get the most out of their investment in IronPort's products.
To be successful, the Bonded Sender program would need broad participation on behalf the same ISPs that now subscribe to the blacklists. So far, admits Banister, Bonded Sender's beta phase has only included the smaller, regional ISPs and IronPort's existing customers (which have yet to post any bond).
The beta phase did reveal an interesting statistic for readers who thought I was exaggerating the scope of the false-positive problem. After a one and half months of testing, IronPort identified hundreds of thousands of false-positives. At that rate, the mail generated by IronPort's customers alone, which make up a small percentage of the total amount of e-mail that traverses the Internet, is resulting in over one million false-positives per year.
Could Banister be exaggerating his figures? Maybe. But I'm not sure what the motives would be. Banister will make no money off the Bonded Sender program and, in terms of preserving the value of his offerings, his competitors will benefit equally from the Bonded Sender effort should it succeed.
In the bigger picture, Banister's Bonded Sender idea is validation of the notion that whitelists could play an increasingly important role on smacking down spam. But in order for whitelists to reliably extend their utility to the rest of us (and not just companies that can afford to put up thousands of dollars in bond money), we'll need established methodologies and techniques that don't reproduce the kind of problems caused by blacklists-- ones that don't throw the baby out with the bath water. What might those be? Stay tuned.
If you feel victimised in some way by spam, feel free to commiserate with your fellow readers using Talkback below, or if you just want to vent your frustrations, send an e-mail to edit@zdnet.com.au.
Another weapon - or another means to get your legitimate opt-in newsletters delivered is to pre-screen it against widely used spam-filters or systems.
The service I use for both my web hosting and newsletter publishing comes at around $300 per year and includes a very neat SpamCheck feature.
Before sending out my newsletter I just click one button, and my newsletter is 'scored' against the SpamAssassin spam-filter.
If the score is too high I have the chance to fix it before sending out the newsletter.
With this, the likelihood of reaching all my subscribers increase.
A way to test this system without using their hosting services is to do the following:
1) Send your newsletter to affiliateprogram-spamcheck@sitesell.net
2) Make sure you put the word TEST at the beginning of the Subject field. That way the service know you are trying out the SpamCheck testing and are not just spamming it.
A few minutes later you'll get back a report that is similar to the report I get when I'm using the built-in system.