Canned Spam Filters

The ACCC uses anti-spam filtering with SpamAssassin for everyone at UIC, regardless of where your account is. (Or isn't -- we also do anti-spam filtering for people who just use our uic.edu re-direction service, regardless of whether the email is actually delivered to an ACCC email server or not.)

The default settings are described in this Web page, along with information on how to change them if you wish. To change anti-spam filters settings, go to our Email Filters utility page, login with your UIC netid and ACCC common password, select the machine that your account is on if necessary, and click on ANTI-SPAM Settings/Filter.

Note added November 9, 2010: On Tuesday morning, November 9th, 2011, the ACCC will begin phasing in new Barracuda brand Anti-Spam appliances for our incoming email sent to uic.edu. We believe that this new system will improve our incoming email spam and virus protection.

Most email users will not notice any difference difference as we switch over, other than better spam and virus detection. But there will be a few changes in how you can or should set up your spam filters. To keep from confusing people too much during the switchover process, we have not updated this page yet. The upgoming changes are described in Upcoming Changes to Spam Filtering.

• Spam and What We Do with It
• The Spam Report Email Message
• Email Filter Tool
• Select What Should Be Done With Your Spam
• Spam-Filter Sensitivity
• Your Whitelist -- A List of Email Addresses that You Know Don't Send Spam
• Enter Whitelist addresses directly using the Email Filter Tool
• Other Ways to Whitelist
• -- Webmail address book
• -- From the Spam email message
• Foreign Language Character Sets
• Message Modifications
• Greylisting
• Click the Button to save your Anti-Spam Filter Settings
• Oops, Did You Forget Something?
• Spam Messages Headers and Creating Local Filters for Spam Headers
• Attachment Filters and Customizable Filters
• How can I personalize my anti-spam filter to make it more accurate?

Are you interested in an "Incredible Satellite TV offer?" Would you like to "Consolidate your debts in an offshore Visa card?" No? Well then, certainly you'd like to "Dig up Dirt on your Coworkers!" Still not interested? Good, because that's what the ACCC's automatic spam filtering system is turning away.

You've probably seen phrases like those above in email messages you received but didn't request, that is, spam. We discussed the spam problem in the October/November/December 2000 issue of the A3C Connection in the article called "Slamming Spamming." That article covered the basics, what spam is, how it works, and some possible options to minimize the amount of spam you receive.

One such option is to filter spam out before it reaches your Inbox. But anti-spam filters can be very complicated to set up, and they can dispose of valid email if not crafted carefully. (For example: The word "Specialist" has "Cialis" in the middle of it. And yes, that has happened.)

We are now remedying this for you in an automatic and simple way: we've set up effective anti-spam email filters, i.e., Canned Spam Filters, and we're running them for everyone, automatically. The most egregious spam we're throwing away before it even gets to you.

The Mailtools anti-spam filters won't catch all of of your spam -- a perfect anti-spam filter is impossible. And there is some stuff that really looks like it's spam but we just can't be sure. This spam we deliver to you, marking it as spam.

Why? Although we have taken care to make this unlikely, it is possible that a piece of valid email will somehow be sorted in with the spam. You should check the spam messages we deliver to you just in case there is any real email that gets caught among the spam. We will help you with this too, by sending you an email message with a list of your probably-spam email -- its From address and its Subject. This way you can make sure you didn't miss anything that you wanted to see -- that notification of your lottery winnings, for example.

What We Do Now

This is what happens by default now:

• Our anti-spam filter applies to all ACCC email accounts and any @uic.edu address (even if it doesn't go to an ACCC account) is also now automatically scanned for spam.
  
  
• Messages that are identified as having a very high spam score, that is, messages that most certainly are spam, are thrown away before ever reaching you. Comparing the number of messages I used to have in my spam folder to the number that I have now, I'd say that about 60% of the spam that I used to get is now getting thrown out.
  
  
• But there are always some messages that we can't be 100% sure are spam. In fact, now and then, a legitimate message will get filtered out as spam. So we deliver the lower scoring spam to you, but we tag it as spam for you.
  
  
• By default, we will tag the spam messages as being spam by putting ***SPAM*** in their Subject:.
  
  
• By default, if your account is on an ACCC server, we also move them the spam messages into to a spam mailbox on the server.
  
  
• Also by default, we will change each spam message into an attachment, and send it to you as an attachment, with the SpamAssassin report. This is good because it saves you from viruses that might automatically execute or other malicious or intrusive items in the message.
  
  
• Every day, if your account is on an ACCC server, we send you a Spam Report Email message listing today's new spam messages, giving you a chance to redeliver them or whitelist their senders.
  
  
• Every day, if your account is on an ACCC server, we delete your the messages in your spam mailbox (on the server) that are three days old.
  

You can change all these settings and decide what exactly you want to be done with your spam once it gets to your account. You might not want your spam moved to a spam mailbox on the server. You can also make exceptions to the spam filter rules, or set the filter to be more or less aggressive, or various other options. Login to the Email Filters Utility to view and adjust your current settings. These settings are discussed below in more detail.

If your account is on an ACCC server, your main interaction with the ACCC SpamAssassin anti-spam service will be with the Spam Report email messages that it sends, for example, Figure 1.

Figure 1: Sample Spam Report Email Message The bold number before the From: address is the message's spam score. Do not reply to the message; no one will see your reply. If you have questions about ACCC spam filtering, send them to spamfiltering@uic.edu.

The Email Message Lists

There are two groups of spam email messages listed;

• the top messages with spam scores less then 10,
• the bottom messages with scores other greater than 10.

(Any messages with a spam score of less than 5 are not considered to be spam. Though lots of email with lower scores are spam; but lots with lower scores aren't also.)

For each message,

• the From: address
• and the Subject:

will be listed.

If you see a message that isn't spam, you can retrieve it with the links in your Spam Report email message.

Beside the Subject: is a (redeliver) link; click it to have the message re-delivered to your Inbox. You might also want to forward it, with full headers, to spamfiltering@uic.edu ; we are interested in seeing false-positive messages; sometimes we can even fix false-positive problems.

And to prevent a message from this sender being mis-identified as spam again,

Beside the From: address is a (whitelist) link; click it to automatically add the From: address to your whitelist.

By default, these spam messages will be deleted automatically from your spam mailbox on the server after three days if you don't do anything else about them.

The Text at the Bottom of the Message

The text at the bottom of the Spam Report message has links to various Web pages that will be useful for you in working with your spam.

• First there is a link to WebMail. If your regular email program is set up to use POP -- if you login, download all you new incoming email, and then logout -- then you can't work with your spam email folder on the server with your regular email program. Click on this Webmail link if you want to directly access your spam email folder.
  
  
• Next there is a link to the ACCC Email Filters Utility, http://mailfilter.accc.uic.edu
  Then click ANTI-SPAM Settings/Filter. Use this Web page to check out your ACCC anti-spam filter and change details if you wish to.
  
  
• Next are links to this page and to the ACCC Web home page.
  
  
• Finally, there is a link that allows you to toggle between text and HTML format for your Spam Report email messages.

The ANTI-SPAM Settings/Filter are introduced below.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid and ACCC common password. To work on your anti-spam filters, click ANTI-SPAM Settings/Filter.

Changing Your "Account"

If your uic.edu email is sent to an ACCC email server, the Email Filters utility will automatically open for that server. However you can do filters for all ACCC servers you have accounts on and for your uic.edu email alone, if you wish. At the top of the Email Filters Utility Page, just below the heading, it says what netid and "account" -- the server -- that you're working on the filters for. If you want to work on the filters for another machine, click on (change account). That displays a screen which allows you to select another of your ACCC accounts to do filters for or for No Account -- your uic.edu email.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid and ACCC common password. To work on your anti-spam filters, click ANTI-SPAM Settings/Filter.

These choices will only be offered when you are setting options for an ACCC account.

There are two choices as to what should be done with spam once detected. With the new Spam Report email messages, the default moving to spam mail folders on the server, should work just fine for everyone. Having the spam messages on the server used to be a problem for people using POP, because they didn't have contact with email on the server with their email program, but the Spam Report email message (Figure 1) solves that problem now.You can check to see whether there is any legitimate email in your spam folder by checking the listing of your spam email messages in the Spam Report email message (Figure 1) that we'll send you listing of your spam email, and use the links in the email message to redeliver any messages you want to keep to your Inbox. So it's no longer as important to use the Tag only option with POP.

However, there still are two choices. They are: (1) Move to a spam mailbox on the server or (2) tag the message as spam. Filter spam into your spam mailbox on the server is the default.

• The Tag only option was designed for people who use POP. Here's how it works:
  
• When spam comes into your email Inbox, it will stay there unless YOU make a filter to move it to another mailbox.
• A hidden tag is added to each spam email message: X-Uicclass: UICClass Spam
  (This is an X-header ; see the online version of Figure 2: Headers of a Legit Email Message, from the October/ November/ December 2000 A3C Connection.) You can use this hidden tag as a criterion in a Eudora local filter so that Eudora will recognize the email message as spam and move it into a local spam mailbox on your personal computer. See Creating a Eudora Filter with the SPAM Tag. To see an example of the X-Uicclass header, see Figure 3 , which is the full headers of a spam message identified by the ACCC SpamAssassin filters.
• It also tags the Subject: as ***SPAM***
  You can also use this tag as a criterion for a local email filter so that your email program will recognize a spam email as spam and move it into your local spam mailbox. For example, see Figure 4 , which is a spam message identified by the ACCC SpamAssassin filters.
• Make sure that you check your spam messages to see if any are legitimate messages. (It's a lot easier to do this with the other option.)
  
  
  
• The default option Filter spam into your spam folder was designed for people who use IMAP, but with the new Spam Report message email (Figure 1) can be used by everyone. Once an email message is identified as probably being spam, it will automatically be sorted into another folder on the server called spam. When you use this option the system will send you a summary of the titles of the probable spam messages you receive and will delete older spam on a regular basis. The choices are for the scheduling are two:
• Daily summary of filtered spam, spam deleted when it's three days old. This is the default.
• Weekly summary of filtered spam, spam deleted when it's 10 days old. You might want to choose this if you don't check your email every day.
This option also tags the Subject: as ***SPAM***
Make sure that you check the spam messages listed in the Spam Report message email message (Figure 1) for legitimate messages before they are will be deleted, so that you can redeliver any legitimate messages that were falsely identified as spam.

For example, see Figure 4 , which is a spam message identified by the ACCC SpamAssassin filters.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid and ACCC common password. To work on your anti-spam filters, click ANTI-SPAM Settings/Filter.

This and all the following choices are "Anti-Spam Settings" and will be offered for all types of accounts.

The anti-spam filter works on a number scale. Each email message is given a numeric score, the sum of the points for various infringements are given specific numbers of points depending on how spam-like they are. All email messages with a spam score worse than a specific number is marked as spam.

• Clicking Aggressive will lower the bar, causing more spam to be filtered out, but it also raises the risk of accidentally filtering out your legitimate email.
• Clicking Low Sensitivity will raise the bar, causing less spam to be filtered out, but also lowering the risk of accidentally filtering out your legitimate email.

I recommend that you leave it at Normal. It works well at normal, with a minimum of false positives (real messages that you want to see that are marked as spam).

Your whitelist is a list of email address that you know won't send your spam. The ACCC's SpamAssassin will give an automatic pass to any email that is sent to you from any address that's in your whitelist.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid and ACCC common password. To work on your anti-spam filters, click ANTI-SPAM Settings/Filter.

Email messages from email lists are often identified as spam when they aren't; to prevent messages from your email discussion lists from being counted as spam, type the email address of all lists the you are subscribed to and other email addresses that you trust in the box. For instance, do you subscribe to bubblegumweekly@stickylists.com? Then you put that address into the box in this section.

If you subscribe to a number of lists from the same domain, you can cover all of them by just entering the domain name, @yahoogroups.com, for example.

By default we already exempt any mail that was sent from within UIC or was sent through a UIC emailing list (Listserv).

We automatically whitelist all addresses in your Webmail address book on a daily basis, so you shouldn't have to add any addresses you have there.

We'd be glad to whitelist your personal address books too (Eudora, Outlook, etc), but we don't have access to them. If you would like all your address book entries to be whitelisted, just upload/transfer them to your Webmail address book. By the next day they will all be whitelisted.

You can do this easily with our address book conversion utility.

If your account is on an ACCC server, every day, the ACCC anti-spam filters will send you an email message listing the messages that have been identified as spam.

For each message,

• the From: address
• and the Subject:

will be listed.

• Beside the From: address is a (whitelist) link; click it to add the From: address to your whitelist.
• Beside the Subject: is a (redeliver) link; click it to have the message re-delivered to your Inbox.

This option will be removed in the new Barracuda system.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid and ACCC common password. To work on your anti-spam filters, click ANTI-SPAM Settings/Filter.

This is opt-in not opt-out.

These days, a lot of the spam email comes not only in foreign languages, but also in different character sets. This sections has a list of foreign language character sets (Japanese, Korean, Cyrillic, etc.).

If you ever get legitimate email in any of these character sets, select them. If you don't, don't. They will be used to identify probable spam.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid and ACCC common password. To work on your anti-spam filters, click ANTI-SPAM Settings/Filter.

There are two additional message modification options:

Rewrite the Subject: line of spam messages by putting ***SPAM*** before the message's original subject. You can choose to have this done or not; the default is add the ***SPAM***. It's best to select this; it's much easier to write a filter in your email program to sort on ***SPAM*** in your Subject: than the X-header.

The choice to not change the subject will be removed in the new Barracuda system.

Change the spam message into an attachment and send it to you that way. This will disable any auto-execute viruses or Web bugs. Or you can leave the messages as is. The default is to change the spam messages into attachments. The message in A Look at SpamAssassin Spam Messages Headers and Creating a Eudora Filter with the SPAM Tag is an example of why you might want to turn this option on.

This option will be removed in the new Barracuda system.

This option will be removed in the new Barracuda system.

To configure your anti-spam filters, visit our Email Filters utility page and login with your UIC netid ANTI-SPAM Settings/Filter.

When normal email messages are routed through the internet, it often happens the machine on the receiving end isn't able to accept the message right away. In such cases, the receiving machine replies to the sending machine with a "temporary error", which tells the sending machine to try again in a little while. After a few minutes, or perhaps even as much as an hour, the sending machine will try again. Normally it will continue to try for a few days before giving up completely. When the receiving machine is finally ready, it will accept the message and it will be delivered to the target user's mailbox.

Greylisting takes advantage of the fact that spammers don't usually keep trying, they tend to give up after one try. If the incoming message is from a spam server, and we give the incoming messages such a temporary error as described above, the result is that they never try to send the message again. So giving a temporary error to any message that we're seeing for the first time has the effect of keeping a whole lot of spam from ever being delivered, while also not stopping real email, as legitimate email is re-delivered shortly after the first rejection.

Normally the whole process takes under 10 minutes, but the time it takes to retry a message ultimately depends on the particular remote server.

That's the basic principal of greylisting, although there are many details. The main disadvantage to such method is that it has the potential to slow down some incoming messages, whose immediate reception might be important to some people. However, we take many steps to make this very minimal:

• All mail from within UIC is whitelisted (meaning it bypasses greylisting, see above for more on whitelisting),
• Messages from senders in your whitelist or your Webmail address book are whitelisted,
• After a message is successfully delivered once, the sender is whitelisted, so that the delay only happens once per sender. Messages from the same sender after that are not delayed again. (It's actually a little more complex than that, but that's the general idea.)
• You can also add any addresses to your permanent whitelist if you want to make sure that delivery from those senders is always immediate.

Additionally, we have provided an interface so that you can view exactly what messages are in which states of greylisting. Normally you shouldn't need to use this, but if you suspect there is a problem you can check this list. Unfortunately, however, the only information we have is the sender's address -- no subject, no message, etc.

The interface is available from the anti-spam settings utility where you set the greylisting option.

A second problem with greylisting is that it's possible, although very rare, that the remote server, if not properly configured, might NOT redeliver a legitimate message.

Because of this possibility, when we find servers like this, we globally whitelist them to prevent any future problems. Likewise, the interface mentioned above can be of use in finding those types of situations. If you ever come across such a situation, please inform us so that we may globally whitelist that server for all other UIC users.

Would you like to know more about the details of how greylisting works? You can read a greylisting "whitepaper" on the topic here.

Greylisting FAQ

Question: I see that a legitimate piece of email has been rejected, can I get it back?

Answer: There isn't anything that YOU can do, but if the message really is legitimate, then you won't have to do anything. The remote server will re-send it to you shortly, at which time it will not be rejected again.

Question: How long will the delay be for rejected messages?

Answer: That really depends on the remote server from which the message is coming. Normally legitimate email servers try to re-send messages within a few minutes after receiving such rejections. Nonetheless, it is rare but possible that the remote server in question is using a larger delay period.

Question: I've 'whitelisted' the address, so why doesn't the mail come back?

Answer: Whitelisting an address can only affect all FUTURE delivery attempts, it can't undo the rejection that we've already given to a particular message. It means that, from this point forward, a whitelisted address won't be subjected to greylisting rejections or any other type of spam filter rejections.

When you are done configuring your anti-spam filter, click the Save Settings button.

The neat thing is you're not turning on the anti-spam filter, you're just adjusting the settings. It's already been turned on for you!

Of course, if you forget to identify any email discussion lists or other email senders that you trust, the filter might identify messages from then as spam, so you'll want to be sure to check your spam folder frequently at first to see if any messages got through from addresses you forgot to include. You can imagine that, if you subscribe to many lists, it may take a few tries before you get it quite right.

Did you forget to add an email discussion list or another email address that you always trust to send you email that you want to see? No problem. Just return to Email Filters page, click ANTI-SPAM Settings/Filter, then move down to the Whitelist section. You'll see that the whitelist addresses that you entered before in the box where you typed them. Simply edit the list and click the Save Settings button again and your changes will take effect immediately.

Spam message headers will be different in the new Barracuda system, but you can still -- always -- sort on the ***SPAM*** in the spam message's subject.

If you use POP with a personal computer email program such as Eudora, Thunderbird, or Outlook, you might not want to use the anti-spam filter's move spam to your spam folder option because that spam folder will be online and you don't have direct access to online folders when you use POP. Although you certainly can use the online spam folder when you use POP; you can manage the email in the online spam mailbox with the (redeliver) and (whitelist) links in your Spam email summary that the ACCC's spam filters send you every day (or every week).

But if you prefer, you can have your spam delivered to your Inbox, "tagged" as spam. The Tag only: action option was designed to be used by people who use POP, and should be used with local Email Filters. If you use this option, you have to then filter the tagged messages with a local filter -- one you make with your personal computer email program..

The Tag only option action adds a hidden tag: X-Uicclass: UICClass Spam to each spam email message. (What they actually add is an X-header ; see the online version of Figure 2: Headers of a Legit Email Message, from the October/ November/ December 2000 A3C Connection.)

Using X-Uicclass to Sort Spam

An Easier Way To Sort Spam: ***SPAM*** in the Subject:

Figure 4: An easier way: use the ***SPAM*** in the Subject: This is the whole message that goes with the headers above. The ACCC Spam Filters added the Spam Report Card. This is a strange take off of the "419 spam", where you give someone money so they can get their money, and they will give you lots more money in return. Notice that it is not even addresses to me -- no To: address.

Figure 4: Filter For Mailtools Tag only: Action -- ***SPAM*** This is a filter in Thunderbird that selects email that the Mailtools anti-spam filter has added ***SPAM*** to the Subject: and moves them to a separate mailbox named junk.

The anti-spam email filter is only one of the types of filters the ACCC email filters tools allows you to make. There are also specialized filters for attachments and customizable filters for just about anything else. There is a Web page help file for the ACCC email filter tools; there are links to it on all of the email filters setup pages.

Here is one strategy, but it will require some work on your part. If you're willing to keep up with it, you can increase the accuracy rate of your filter to be close to 100%.

First set your filter to be "aggressive". Or, for even better results, you can set it to be even more aggressive than the "aggressive" setting -- I'm still getting gtoo much spam! What can I do?

Next, you've got to put all the email addresses from which you normally receive email into your whitelist:

You shouldn't have to worry about adding any UIC or UIUC addresses, they should come through without being on your whitelist.

Any addresses in your Webmail addressbook will automatically be counted in your whitelist as well, even though they won't appear in your whitelist listing on the anti-spam filter page. If you don't use Webmail, you can still take advantage of this by uploading your addressbook to Webmail. Click the EMAIL button at the top of this page and then find the link "Email - Address Book Tool".

If there are still address from which you get mail besides those, you need to add them to your whitelist manually to make sure mail from them still gets through. It may help to go through the legitimate mail that you've sent or received in the last month or two and make a note of all the addresses there.

Doing this will probably keep almost all spam out of your inbox. But it will probably cause some of your legitimate messages to end up in your spam folder. Carefully check your Spam Summary email message for such messages. If you continue to add addresses to your whitelist as you find them, your filter will become more and more accurate over time.