Content Filtering

These days everyone seems to be worried about getting sued. I've heard of companies spending countless man-hours and hundreds of thousands of dollars (even millions) attempting to prevent the possibility of a lawsuit.

There are now whole courses, usually mandatory for the Human Resources department, on the subject of preventing various kinds of lawsuits, including sexual and racial harassment. Upper management is also usually very concerned with leaking trade secrets (such as customer lists) and IT people are typically concerned with the massive amounts of spam their email system receives every day. People are even concerned about the company email system being used for personal endeavors such as job hunting, romance seeking and reading jokes.

In recent years all of these concerns have led to the creation of a new industry and a new type of product. Some very bright people have figured out how to turn these concerns, whether or not they are valid, into dollars.

Many companies are now installing what is known as a content filter. These filters can be used to filter out a variety of message types, some of which are listed below.

• Spam - Many content filters are set up to look for specific keywords which typically are found in spam messages.

• Inappropriate content - Most companies (at least those that have some common sense) have rules about what types of materials can be viewed at the workplace. There are lots of good reasons for these rules, including sexual and racial harassment lawsuits. Content filters can be set up to help detect these materials upon attempts to receive them. In fact, these inappropriate messages can be routed directly to the human resources department.

• Resumes - Some companies set up their content filters to catch any attempts to send or receive resumes. This can be useful to get a warning about someone who is looking for a new job. On the other hand, it does seem like an invasion of privacy. Of course, it's not generally a good idea to search for another job while at your current job...

• Proprietary information - Sometimes employees send out information that should not be sent. A good example is someone sending out customer mailing lists or plans to the competition. Content filters can be set up to detect this information and inform the appropriate managers.

• Profanity - Attempts to send or receive messages with curse words can be trapped and discarded.

Unfortunately, content filtering is, by the very nature of the beast, not very precise. Typically, these things work on scoring system. A single instance of a swear word, for example, might be worth a couple of points. A racial slur might be worth three, and a spam keyword might be worth one point. When it all adds up to a certain value, the message is flagged as questionable and either deleted or forwarded depending upon the rules.

My day job has installed content filters on our email server, and it's interesting how many times the filters choose incorrectly. For example, I receive NTBUGRAC, a mailing list which discusses bugs and issues with Windows 2000 and Windows NT. Every single day at least one message (and often many more) are stopped by our filtering system.

Content filtering is not perfect and it's never going to be perfect. It's extremely annoying to most employees and even managers. However, it's become pretty standard in larger companies. They are so popular that it's probably only a matter of time until you see these annoying inventions in your own company.