And How to Stop Them

Effectively stopping spam over long-term requires much more than blocking individual IP addresses and creating rules based on keywords that spammers typically use. The increasing sophistication of tools spammers use coupled with increasing number of spammers in wild has created a hyper-evolution in variety and volume of spam. The old ways of blocking bad guys just don’t work anymore.

Examining spam and spam-blocking technology can illuminate how this evolution is taking place and what can be done to combat spam and reclaim e-mail as efficient, effective communication tool it was intended to be.

There are several widely-used methods for filtering spam, each of which can be defeated by spammers to some degree. Understanding strengths and weaknesses of each approach and methods spammers use to defeat them is basis of an effective, comprehensive anti-spam strategy.

Signature-based Filters

Signature-based filters examine contents of known spam, usually derived from honey pots, or dummy e-mail addresses set up specifically to collect spam. Once a honey pot receives a spam message, content is examined and given a unique identifier. The unique identifier is obtained by assigning a value to each character in e-mail. Once all characters have been assigned a value, values are totaled, creating spam’s signature. The signature is added to a signature database and sent as a regular update to e-mail service’s subscribers. The signature is compared to every e-mail coming in to network and all matching messages are discarded as spam.

The benefit of signature-based filters is that they rarely produce false-positives, or legitimate e-mail incorrectly identified as spam.

The drawback of signature-based filters is that they are very easy to defeat. Because they are backward-looking, they only deal with spam that has already been sent. By time honey pot receives a spam message, system assigns a signature, and update is sent and installed on subscribers’ network, spammer has already sent millions of e-mails. A slight modification of e-mail message will render existing signature useless.

Furthermore, spammers can easily evade signature-based filters by using special e-mail software that adds random strings of content to subject line and body of e-mail. Because variable content alters signature of each e-mail sent by spammer, signature-based spam filters are unable to match e-mail to known pieces of spam.

Developers of signature-based spam filters have learned to identify tell-tale signs of automated random character generation. But as is often case, spammers remain a step ahead and have developed more sophisticated methods for inserting random content. As a result, most spam continues to fool signature-based filters.

Rule-based (Heuristic) Filtering

Rule-based filters scan e-mail content for predetermined words or phrases that may indicate a message is spam. For example, if an e-mail administrator includes word “sex” on a company’s rule-based list, any e-mail containing this word will be filtered.

The major drawback of this approach is difficulty in identifying keywords that are consistently indicative of spam. While spammers may frequently use words “sex” and ‘Viagra” in spam e-mails, these words are also used in legitimate business correspondence, particularly in healthcare industry. Additionally, spammers have learned to obfuscate suspect words by using spellings such as “S*E*X”, or “VI a a GRR A”.

It is impossible to develop dictionaries that identify every possible misspelling of “spammy” keywords. Additionally, because filtering for certain keywords produces large numbers of false positives, many organizations have found they cannot afford to rely solely on rule-based filters to identify spam.

Blacklists

The goal of blacklisting is to force Internet Service Providers (ISPs) to crack-down on customers who send spam. A blacklisted ISP is blocked from sending e-mail to organizations. When an ISP is blacklisted, they are provided with a list of actions they must take in order to be removed from blacklist. This controversial method blocks not just spammers, but all of ISP’s customers. Blacklisting is generally considered an unfriendly approach to stopping spam because users most affected by blacklist are e-mail users who do not send spam. Many argue blacklisting actually damages utility of e-mail more than it helps stop spam since potential for blocking legitimate e-mail is so high.

In addition to ethical considerations, there are other problems with blacklists. Many blacklists are not updated frequently enough to maintain effectiveness. Some blacklist administrators are irresponsible in that they immediately block suspect servers without thoroughly investigating complaints or giving ISP time to respond. Another downside is that blacklists are not accurate enough to catch all spam. Only about half of servers used by spammers, regardless of how diligent blacklist administrator may be, are ever cataloged in a given blacklist.

Blacklists are used because they can be partially effective against spammers who repeatedly use same ISP or e-mail account to send spam. However, because spammers often change ISPs, re-route e-mail and hijack legitimate servers, spammer is a moving target. Blacklist administrators are forced to constantly revise lists, and lag-time between when a spammer begins using a given server and when blacklist administrator is able to identify new spam source and add it to blacklist allows spammers to send hundreds of millions of e-mails. Spammers consider this constant state of flux a part of doing business and are constantly looking for new servers to send spam messages.

Blacklists, therefore, have some utility in stopping known spammers. Because of their limitations, however, this data should only be used in conjunction with other sources to determine if a given message is spam.

Whitelists

Whitelists are databases of trusted e-mail sources. The list may contain specific e-mail addresses, IP addresses or trusted domains. E-mails received from a whitelisted source are allowed to pass through system to user’s email box. The list is built when users and e-mail administrators manually add trusted sources to whitelist. Once built, catch-rate for spam can be close to 100%, however, whitelists produce an inordinate number of false positives.

It is virtually impossible to produce an exhaustive list of all possible legitimate e-mail senders because legitimate e-mail can come from any number of sources. To get around this difficulty, some organizations have instituted a challenge-response methodology. When an unknown sender sends an e-mail to a user’s account, system automatically sends a challenge back to sender. Some challenge-response systems require sender to read and decipher an image containing letters and numbers. The image is designed to be unreadable by a machine, but easily recognizable by a human. Spammers would not spend time required to go through a large number of challenge-response e-mails, so they drop address and move on to those users who don’t use such a system.

Whitelists are only partially successful and impractical for many users. For example, problems can arise when users register for online newsletters, order products online or register for online services. If user does not remember to add new e-mail source to their whitelist, or if domain or source is entered incorrectly, communication will fail. Additionally, whitelists impose barriers to legitimate e-mail communication and are viewed by some as just plain rude.

Traditional anti-virus software only stops known computer viruses – stopping undefined computer viruses requires a different approach.

In past, network administrators scrambled to apply new virus signatures whenever new computer viruses were discovered. While these signatures will stop a known threat, it takes time for anti-virus vendors to develop them. Unfortunately, newest and most damaging viruses are able to spread so quickly that damage is done before a signature can be developed and distributed.

In fact, independent testing laboratory AV-test.org found response times for major anti-virus software publishers to range from just under 7 hours to almost 30 hours , with four leading vendors (Sophos, McAfee, Symantec and Trend Micro) clocking in at no less than 12 hours.

In January 2004, computer virus known as “MyDoom” created mass disruption to corporate resources and reputations as it quickly spread through e-mail networks worldwide. At its peak, MyDoom infected one in every five e-mails transmitted over Internet. The worm broke records set by previous malware, such as Sobig.F, to become fastest-spreading virus ever. This incredible propagation speed left many networks vulnerable - despite presence of anti-virus software - because of lag time between when virus outbreak began, and when a virus definition became available.

As a result of recent malware threats, corporations and organizations have learned a painful but important lesson: simply deploying a signature-based solution is no longer enough. Detecting and eliminating computer viruses requires a multi-faceted, rapid-response approach that traditional anti-virus protection cannot provide. Even a single unprotected computer on an enterprise network can bring down entire system in just minutes, rendering even most expensive and up-to-date software useless.

Why E-Mail is Particularly Susceptible

In many organizations, e-mail has replaced telephone as most useful business tool available. Unfortunately, e-mail has also been a victim of its own success and presents a unique threat to enterprise network as a whole.

Detecting and eliminating threats has traditionally been combined responsibility of firewalls, virus scanners, and intrusion detection systems (IDS) set up by enterprises to defend against attacks. Firewalls prevent unauthorized programs from accessing network, virus scanners scan each PC in network for malicious code, and gateway servers lock down extraneous ports to protect against unauthorized access.

But key Internet-facing applications, including e-mail are unguarded by firewalls. In order to function, e-mail must expose firewall ports, including port 25, port used by SMTP (Simple Mail Transfer Protocol) and port 110, port used by POP (Post Office Protocol).

When a firewall receives a connection on port 25, it generally assumes that transmission is e-mail and allows it to flow through to e-mail server. The transmission may very well be a valid e-mail; however, it could also be a virus, spam or something much worse. Firewalls are not able to distinguish between “good” mail and “bad” mail and therefore they are unable ot protect e-mail application.

Stop E-Mail Threats at Gateway

Therefore, some sort of protection is needed specifically for e-mail and, since best place to stop a threat is before it gets inside network, protection should be at e-mail gateway. Protecting e-mail gateway requires a coordinated effort to combat a host of issues, including spam, viruses, corporate policy infringements, directory harvest attacks, denial of service attacks, phishing, spoofing, and snooping. As e-mail threats evolve, distinction between each of these types of threats becomes blurred.

Furthermore, accuracy in identifying “bad” e-mails is crucial. Extreme care must be taken to avoid filtering out legitimate e-mails (false positives), which could contain important information from customers or partners.