Continued from page 1
Of course, by definition, community filters cannot reach 100% accuracy as someone has to be getting spam to be voting it as such! Theoretically, a Bayesian filter may be able to eventually get quite close to 100% accuracy, so at least there is hope there. Content based filters (those that look for certain words, phrases or other indicators in a message to identify it as spam), will almost certainly not get much higher accuracy figures than best of them can achieve today. Adapting to changing spam requires new filters to be created on an ongoing basis.
And finally, we come to holy grail of spam filtering:
It is transparent Strangely enough, not enough work seems to be done in trying to achieve this goal. Some of best filters on market today identify spam with impressive accuracy and then simply place them in a “killed mail” folder for your later perusal. Now, forgive me if I’m missing something here, but isn’t point to save you having to wade through junk mail? Isn’t that what you bought filter for? With “SpamSplatter 3000”, you don’t need to do that.
As we haven’t achieved 100% accuracy yet (and probably never will), only way to free us from checking killed mail folder is a challenge/response system. This is where a message is automatically sent back to sender requiring them to take some action for their message to actually be delivered.
Some systems tend to go overboard with challenge/response system. These systems - often called “Whitelist” systems - block messages from anyone that isn’t in user’s friends list. Guaranteed 100% effective, but too drastic a measure for most users.
Now, it seems that most intelligent use of this system would be to send challenges only to messages that were flagged as “questionable”. Good message can be delivered, definite spam can be deleted and questionable ones would earn themselves a challenge message.
So, to sum up, let’s rewrite qualities of our perfect filter and get a shopping list of what to look for while we wait for “SpamSplatter 3000” to arrive:
1. Simple, minimal setup and maintenance. 2. Extremely low rate of false positives and as few false negatives as possible. 3. A transparent “fail-safe” mechanism whereby victims of those false positives can force message through to you.
It’s simple really. Now, who’s going to build me this “SpamSplatter 3000”…?
Alan Hearnshaw is owner of http://www.WhichSpamFilter.com, a site which provides weekly in-depth spam filter reviews, user help and guidance and a community forum. firstname.lastname@example.org
Alan Hearnshaw is a computer programmer and the owner of http://www.WhichSpamFilter.com, a site which provides weekly in-depth spam filter reviews, user help and guidance and a community forum.