So What Makes a Good Spam Filter Anyway?

Written by Alan Hearnshaw

Continued from page 1

Of course, by definition, community filters cannot reach 100% accuracy as someone has to be gettingrepparttar spam to be voting it as such! Theoretically, a Bayesian filter may be able to eventually get quite close to 100% accuracy, so at least there is hope there. Content based filters (those that look for certain words, phrases or other indicators in a message to identify it as spam), will almost certainly not get much higher accuracy figures thanrepparttar 105889 best of them can achieve today. Adapting to changing spam requires new filters to be created on an ongoing basis.

And finally, we come torepparttar 105890 holy grail of spam filtering:

It is transparent Strangely enough, not enough work seems to be done in trying to achieve this goal. Some ofrepparttar 105891 best filters onrepparttar 105892 market today identify spam with impressive accuracy and then simply place them in a “killed mail” folder for your later perusal. Now, forgive me if I’m missing something here, but isn’trepparttar 105893 point to save you having to wade throughrepparttar 105894 junk mail? Isn’t that what you boughtrepparttar 105895 filter for? Withrepparttar 105896 “SpamSplatter 3000”, you don’t need to do that.

As we haven’t achieved 100% accuracy yet (and probably never will),repparttar 105897 only way to free us from checkingrepparttar 105898 killed mail folder is a challenge/response system. This is where a message is automatically sent back torepparttar 105899 sender requiring them to take some action for their message to actually be delivered.

Some systems tend to go overboard withrepparttar 105900 challenge/response system. These systems - often called “Whitelist” systems - block messages from anyone that isn’t inrepparttar 105901 user’s friends list. Guaranteed 100% effective, but too drastic a measure for most users.

Now, it seems thatrepparttar 105902 most intelligent use of this system would be to send challenges only to messages that were flagged as “questionable”. Good message can be delivered, definite spam can be deleted and questionable ones would earn themselves a challenge message.

So, to sum up, let’s rewriterepparttar 105903 qualities of our perfect filter and get a shopping list of what to look for while we wait forrepparttar 105904 “SpamSplatter 3000” to arrive:

1. Simple, minimal setup and maintenance. 2. Extremely low rate of false positives and as few false negatives as possible. 3. A transparent “fail-safe” mechanism wherebyrepparttar 105905 victims of those false positives can forcerepparttar 105906 message through to you.

It’s simple really. Now, who’s going to build me this “SpamSplatter 3000”…?

Alan Hearnshaw isrepparttar 105907 owner of, a site which provides weekly in-depth spam filter reviews, user help and guidance and a community forum.

Alan Hearnshaw is a computer programmer and the owner of, a site which provides weekly in-depth spam filter reviews, user help and guidance and a community forum.

Spam Filters Explained

Written by Alan Hearnshaw

Continued from page 1

While this may seem like a good idea fromrepparttar outset, a whitelist methodology is too restrictive for most people and, as virtually all spam e-mails carry a forged “from” address, there is little point in collecting this address to ban it in future as it is very unlikely to berepparttar 105888 same next time. There are bodies onrepparttar 105889 internet that maintain a list of known “bad” sources of e-mail. Many filters today haverepparttar 105890 ability to query these servers to see ifrepparttar 105891 message they are looking at comes from a source identified by this Internet-based blacklist, or RBL. While being quite effective, they do tend to suffer from “false positives” where good messages are incorrectly identified as spam. This happens often with newsletters.

Challenge/Response Filters “Open sesame!”

Challenge/Response filters are characterised by their ability to automatically send a response to a previously unknown sender asking them to take some further action before their message will be delivered. This is often referred to as a "Turing Test" - named after a test devised by British mathematician Alan Turing to determine if machines could “think”.

Recent years have seenrepparttar 105892 appearance of some internet services which automatically perform this Challenge/Response function forrepparttar 105893 user and requirerepparttar 105894 sender of an e-mail to visit their web site to facilitaterepparttar 105895 receipt of their message.

Critics of this system claim it to be too drastic a measure and that it sends a message that "my time is more important than yours" torepparttar 105896 people trying to communicate with you.

For some low traffic e-mail users though, this system alone may be a perfectly acceptable method of completely eliminating spam from their inbox - one step aboverepparttar 105897 "Whitelist" system outlined above.

Community Filters “A united front”

These types of filters work onrepparttar 105898 principal of "communal knowledge" of spam. When a user receives a spam message, they simply mark it as such in their filter. This information is sent to a central server where a “fingerprint” ofrepparttar 105899 message is stored. After enough people have “voted” this message to be spam, then it is stopped from reaching allrepparttar 105900 other people inrepparttar 105901 community.

This type of filtering can prove to be quite effective, although it stands to reason that it can never be 100% effective as a few people have to receiverepparttar 105902 spam for it to be “flagged” inrepparttar 105903 first place. Just like its similar cousinrepparttar 105904 Internet black list (RBL), this system also can suffer from “false positives”, or messages incorrectly identified as spam.

Hopefully you are now armed with a little more information to be able to make an informed decision onrepparttar 105905 best spam filter for you. For further information, consider readingrepparttar 105906 reviews and articles found at

Alan Hearnshaw isrepparttar 105907 owner of, a web site which conducts weekly in-depth reviews of current spam filters, provides help and guidance inrepparttar 105908 fight against spam and provides a useful community forum.

Alan Hearnshaw is a computer programmer and the owner of, a site which provides weekly in-depth spam filter reviews, user help and guidance and a community forum.

    <Back to Page 1 © 2005
Terms of Use