COMMENTARY--A look at the fight against technologies designed to bring you more and more spam.

Spam is a hot topic at RMIT at the moment. Our ITS department is ramping up its war against spam and, like any war, there are some "innocent" casualties. As ITS ramps up the "aggressiveness" of their anti-spam campaign, more and more legitimate e-mails are being tagged.

It's probably worse for staff at the Lab as we subscribe to quite a bit of online news and some of these (for example "SearchWin2000.com" newsletters) are tagged as spam when they are not. This is fine, because as ITS receives feedback from users they tweak the software to allow particular sites through and all is OK.

Now I don't know about you but I hate unsolicited junk mail--on principle I will not purchase anything that is brought to my attention this way.

So how come such a significant proportion of the Internet bandwidth is being wasted by spam? Most off e-mail addresses that spawn junk mail are killed at a great rate by their respective service providers, but generally after the fact. You would need quite a large, and consequently expensive, human workforce to create the huge number of e-mail addresses used to send spam, and this is not viable. So spammers employ "bots" to create new e-mail addresses.

A bot is simply a small program that connects to a service provider and applies for an e-mail address providing bogus but perfectly believable personal details in the relevant fields--an automated form filler.

When you think about it, writing a program to fill in personal detail forms is not a big ask so the practice is prevalent. But many content providers such as Yahoo are starting to fight back.

Bots are great at automated tasks but so far humans are the only "intelligence" with cognitive skills, show any human an abstract picture and they will make something of the image, a bot will simply see a mess.

And this is how many of the content providers are catching out bots--by employing a type of reverse Turing test called CAPTCHA, which is an abbreviation for "Completely Automated Public Turing test to tell Computers and Humans Apart".

Essentially, the early version of CAPTCHA sourced text from a relatively small 850-word dictionary, distorted it slightly, and placed it on a busy background. The human (or bot) filling out the form was asked to read the distorted text and then type the answer in the field provided.

Now bot programmers are instilling more "smarts" into their creations, such as the small dictionary that the bot searches to obtain the closest match statistical to the distorted text. Sometimes the bot will get it wrong but often the literal "guess" will be correct.

The new generation of CAPTCHA designed to correct this weakness is called Baffletext. Baffletext uses non-English character strings, so the bot's simple word dictionary is no help. But the greatest strength of Baffletext is the methods used to "degrade" the text. The text may be represented with fragments of the characters missing, as if corroded by acid, or with "ink blots" over the characters and then placed on a busy background. The human mind is exceptionally good at recognising patterns and intuitively filling in the blanks in the fragmented characters.

Machine intelligence on the other hand has been struggling for years to recognise slightly miss-oriented shapes let alone incomplete complex shapes, so for the moment the bots are losing.

Naturally this represents a fun intellectual challenge to bot programmers who will try their darndest to come up with a way around this problem. To my mind, if they succeed it will be a great leap in AI, which has been drifting in the doldrums for many years with the promised leaps in intelligence never eventuating.

And if the bot programmers solve the problem, the "good guys" are already investigating audio and video cues instead of static text characters to fool the next generation of smarter bots.

Subscribe now to Australian Technology & Business magazine.

Copyright © 2009 CBS Interactive, a CBS Company. All Rights Reserved.
ZDNET is a registered service mark of CBS Interactive. ZDNET Logo is a service mark of CBS Interactive.