Of course, by definition, community filters cannot reach 100% accuracy as someone has to be getting spam to be voting it as such! Theoretically, a Bayesian filter may be able to eventually get quite close to 100% accuracy, so at least there is hope there. Content based filters (those that look for certain words, phrases or other indicators in a message to identify it as spam), will almost certainly not get much higher accuracy figures than best of them can achieve today. Adapting to changing spam requires new filters to be created on an ongoing basis.

And finally, we come to holy grail of spam filtering:

It is transparent Strangely enough, not enough work seems to be done in trying to achieve this goal. Some of best filters on market today identify spam with impressive accuracy and then simply place them in a “killed mail” folder for your later perusal. Now, forgive me if I’m missing something here, but isn’t point to save you having to wade through junk mail? Isn’t that what you bought filter for? With “SpamSplatter 3000”, you don’t need to do that.

As we haven’t achieved 100% accuracy yet (and probably never will), only way to free us from checking killed mail folder is a challenge/response system. This is where a message is automatically sent back to sender requiring them to take some action for their message to actually be delivered.

Some systems tend to go overboard with challenge/response system. These systems - often called “Whitelist” systems - block messages from anyone that isn’t in user’s friends list. Guaranteed 100% effective, but too drastic a measure for most users.

Now, it seems that most intelligent use of this system would be to send challenges only to messages that were flagged as “questionable”. Good message can be delivered, definite spam can be deleted and questionable ones would earn themselves a challenge message.

So, to sum up, let’s rewrite qualities of our perfect filter and get a shopping list of what to look for while we wait for “SpamSplatter 3000” to arrive:

1. Simple, minimal setup and maintenance. 2. Extremely low rate of false positives and as few false negatives as possible. 3. A transparent “fail-safe” mechanism whereby victims of those false positives can force message through to you.

It’s simple really. Now, who’s going to build me this “SpamSplatter 3000”…?

Alan Hearnshaw is owner of http://www.WhichSpamFilter.com, a site which provides weekly in-depth spam filter reviews, user help and guidance and a community forum. alan@whichspamfilter.com

While this may seem like a good idea from outset, a whitelist methodology is too restrictive for most people and, as virtually all spam e-mails carry a forged “from” address, there is little point in collecting this address to ban it in future as it is very unlikely to be same next time. There are bodies on internet that maintain a list of known “bad” sources of e-mail. Many filters today have ability to query these servers to see if message they are looking at comes from a source identified by this Internet-based blacklist, or RBL. While being quite effective, they do tend to suffer from “false positives” where good messages are incorrectly identified as spam. This happens often with newsletters.

Challenge/Response Filters “Open sesame!”

Challenge/Response filters are characterised by their ability to automatically send a response to a previously unknown sender asking them to take some further action before their message will be delivered. This is often referred to as a "Turing Test" - named after a test devised by British mathematician Alan Turing to determine if machines could “think”.

Recent years have seen appearance of some internet services which automatically perform this Challenge/Response function for user and require sender of an e-mail to visit their web site to facilitate receipt of their message.

Critics of this system claim it to be too drastic a measure and that it sends a message that "my time is more important than yours" to people trying to communicate with you.

For some low traffic e-mail users though, this system alone may be a perfectly acceptable method of completely eliminating spam from their inbox - one step above "Whitelist" system outlined above.

Community Filters “A united front”

These types of filters work on principal of "communal knowledge" of spam. When a user receives a spam message, they simply mark it as such in their filter. This information is sent to a central server where a “fingerprint” of message is stored. After enough people have “voted” this message to be spam, then it is stopped from reaching all other people in community.

This type of filtering can prove to be quite effective, although it stands to reason that it can never be 100% effective as a few people have to receive spam for it to be “flagged” in first place. Just like its similar cousin Internet black list (RBL), this system also can suffer from “false positives”, or messages incorrectly identified as spam.

Hopefully you are now armed with a little more information to be able to make an informed decision on best spam filter for you. For further information, consider reading reviews and articles found at http://www.whichspamfilter.com

Alan Hearnshaw is owner of http://www.whichspamfilter.com, a web site which conducts weekly in-depth reviews of current spam filters, provides help and guidance in fight against spam and provides a useful community forum. alan@whichspamfilter.com