Spam Vigilante - Mail Filter Virtual Appliance

[VMTN_Admin]

http://www.vmware.com/vmtn/appliances/directory/255

A mail proxy based on FreeBSD with spam (SpamAssassin) and virus (ClamAV) scanning. Can be used with any existing mail system.

[jian1]

have using it for couple month, everything run very well.

just found out the server time is wrong, how to setup ntp server and localtime zone info?

[Grifter75]

The next would be increased rule sets in

/etc/rulesdujour/config, assuming you enabled

RulesDuJour. Here are the ones that I use, but there

are many more available:

TRUSTED_RULESETS="TRIPWIRE SARE_EVILNUMBERS0

SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_URI

SARE_FRAUD SARE_BML SARE_OBFU SARE_HTML0 SARE_STOCKS

SARE_RANDOM SARE_ADULT"

Check out

http://www.psoft.net/HSdocumentation/sysadmin/rulesduj

our.html and http://www.rulesemporium.com/ for more

info on those, and additional options.

First of all I just want to say that this product is great, I know this because I am a total child of Windows and I'm finding it fairly easy to configure. However I'm have a few challenges in terms of it's ability to catch spam. I'm finding that it's roughly catching about 50% of the spam that gets sent to our organisation. I've followed the sugestions in Telackey's post here, and I notice that even though I've configured the rulesets as laid out by Telackey, they don't show up in my X-Spam headers. Have I missed a step?

Also I'm wondering about the exchange script that Telackey posted. and I know this is a total child of windows question, but I'm wondering where this script should be saved in the file system, does it need a special file extention, and how would I set up the chron job to get it running in a scheduled fashion.

Sorry for so many questions. Any help would be appreciated.

[groetschel]

Could you find a solution for that utf-8 problem?

Best regards

Gunnar

[Harald_Bessels]

Dear Jian@ldd.ca

You can use the command sysinstall to set the correct time zone (use menupath configure => timezone)

To use ntp to automatically keep the time up to date add the following:

create the file /ect/ntp.conf with the following:

\# Use random ntp.org public timeservers:

server 0.pool.ntp.org

server 1.pool.ntp.org

server 2.pool.ntp.org

server pool.ntp.org

\# or use a local (preferably your own ISP) NTP server and

\# remove the lines above.

#server asia.pool.ntp.org

#server europa.pool.ntp.org

#server north-america.pool.ntp.org

#server south-america.pool.ntp.org

driftfile /var/db/ntp.drift

logfile /var/log/ntp.log

restrict default kod nomodify notrap nopeer

\# end of ntp.conf file

Create the driftfile /var/db/ntp.drift. This is a text file with just a 0 (number zero) on line 1.

Add the following line to /etc/rc.conf

ntpd_enable="YES"

After a reboot the server wil automatically start ntpd and keep the appliance on time.

(The ntp daemon needs UDP port 123 open to the internet if you are behind a firewall)

[nam37]

Thanks for your work on this project. I have two quick questions:

1) Is the current .TORRENT up-to-date, or do I need to apply the updates/patches to it?

2) I'm I current in my understanding that the "Spam Viewer" web interface is not useful to me if I am running in the "SMTP" mode (not the "fetchmail" mode)? It is my understanding that in SMTP mode the server simply filters all the SMTP traffic passing through it and forwards the results on to the actual SMTP server. Is this correct? So since the accounts are involved there is nothing to view on the web interface. Correct?

[Grifter75]

Hi I'm wondering if anyone has had sucess getting telackey's sa-learn-exch.py script to work?

I tried to run it but I got indentation errors. Any help would be apreciated.

Thanks!

[Harald_Bessels]

@ Jian@ldd.ca,

I made a small typing error in my previous message. The ntp.conf file should be in the /etc directory and not /ect.

@ nam37,

1) The torrent is not up-to-date. You should apply the updates/patches.

2) In the SMTP mode, the SV server receives incoming email, applies the anti-virus and anti-spam rules and then puts the email either in the local spam quarantine on the SV server or forwards it to the actual SMTP server. With the "spam viewer" website You can view the blocked spam on the SV server. You can also release blocked false-positives on the "spam viewer" website.

[Harald_Bessels]

I'm also trying to get the sa-learn-exch.py to work to get spam/ham from exchange. It however doesnot remove the email from the Ham folder. Also my sa-learn-exch logfile states the following:

Training SpamAssassin on 2007-02-20 at 17:00:00

Learned tokens from 63 message(s) (65 message(s) examined)

Done training from 1 spam and 0 ham messages at 2007-02-20 17:01:12

I suspect I have the identation of the script wrong. Any help in correcting the identation of the script would be appreciated.

Beste regards,

Harald.

[netmavrik]

I have one setup where I am using SV, and I am not using LDAP or Fetchmail to maintain a list of valid user accounts. I wrote my own app that runs on several different Exchange Servers and one Kerio Mail server that uses SCP to copy a recipient list to SV, and SV then concatenates those different recipient files into one recipient list that Postfix uses. Postfix then "transports" messages for specific domains to the correct destination mail server. That part is working perfectly. I only mention that, because it appears as though the WWW Admin Console will only maintain a list of user accounts when using SV the way most users are using it. So as a work around, I manually created a few user accounts in the Admin Console and setup the correct e-mail addresses for those accounts. All appeared to be working fine, for awhile.

What I have noticed is that after 1 week or so, the user accounts that I create from within the Admin Console vanish. I can recreate them only to have them vanish later. I have been unable to locate the cron job that is causing this.

The reasoning behind the way I have SV deployed is because of a multi-domain, multi-client setup, with several different destination servers. SV's built-in setup script doesn't address that scenario. As mentioned earlier, the filtering and processing of e-mail is working perfectly. It is only the WWW Admin Console that doesn't seem to work with my config.

I know that my current usage isn't standard, but I'm hoping that Teleacky can at least point me in the correct direction. I will fix the problem my self if you can narrow down where to start looking.

Thanks, and Kudos for the best appliance on the VMTN!

[brightdog]

Hi,

I downloaded this appliance last night, and have been working to get it configured and integrated with my exchange server, and so far I'm pretty impressed.

I do have a question however. I have the Exchange integration working and when I go into the Spamviewer webpage I can see the users and all of their smtp aliases from exchange listed. However, I'm having difficulty getting it to recognize new aliases that I've just added to the user's accounts in Exchange.

I've run the maintain-spamviewer.sh -c command, and after doing that I'm able to see the new smtp aliases listed in the spamviewer webpage. However, when I try to send mail to one of those new addresses, I get an error saying

"Recipient address rejected: User unknown in relay recipient table"

I don't understand this, because the spamviewer page shows the aliases.

Is there something else that I need to do in order to update the alias database? I've tried rebooting the appliance, but that didn't help.

Once I get this sorted out, I think I'll be about ready to actually put the appliance into production so see how effective it is at stopping spam and viruses (which is of course why I'm looking to use it). I'm currently using a Symantec Mail Security anti-spam solution which works well, but my license runs out soon, so I need to find another alternative. I'm hoping this will perform as well as the Symantec product.

Any help you can offer is much appreciated.

Thanks,

--brightdog

[brightdog]

I've run the maintain-spamviewer.sh -c command, and

after doing that I'm able to see the new smtp aliases

listed in the spamviewer webpage. However, when I

try to send mail to one of those new addresses, I get

an error saying

"Recipient address rejected: User unknown in relay

recipient table"

I don't understand this, because the spamviewer page

shows the aliases.

Is there something else that I need to do in order to

update the alias database? I've tried rebooting the

appliance, but that didn't help.

Ok, I found the solution. It was in a previous post in this thread. I needed to run the following script:

/etc/periodic/daily/474.m-postfix-accounts

I'm assuming that this gets automatically run each day (this assumption is based only on the directory name). But once I ran it manually, I was able to successfully send messages to the new aliases.

So far so good, I think I've got it up and running now. I'm going to run it in production overnight to see how much spam it detects.

[brightdog]

Hi telackey,

Thanks so much for starting this project, you've done a great job building and supporting Spam Vigilante. Its exactly what I've been looking for.

I've got SV up and running now live and it seems to be identifying approx 75-80% of incoming spam, which isn't too bad, but I believe it can do better with some tweaks to configuration or leveraging some of the other components like Razor, etc.

However, before I start spending a lot of time configuring additional components, I'd to get a feel for how close you are to releasing the next version of Spam Vigilante.

Back in early January you said that you were just weeks away and that you were waiting for FreeBSD 6.2 to be released. It looks like 6.2 was released in mid January, so I'm hoping that you're getting close to the next SV release.

If so, then I'm going to hold off on investiging too much time into the current version (because my understanding is that this won't be an upgrade, but rather a new appliance...is that correct?).

Thanks, and keep up the good work!!

-Brightdog

[newpond]

Hi,

I have just installed this server and can see the mails being downloaded into the spam server.

The users in the vadmin are there so ldap is working OK but no mails are being delivered to my exchange mailboxes.

I am using the fetch mode from my pop serving ISP. I have configured my exchange server as an SMTP device.

Any help would be much apreciated.

cheers

Jon

[phoenixsecure]

This product did start good but its lack of update make it less appealing. Last software update date from 2006-09-12, over 5 months ago, last time telackey reply to a message on this forum was over one month ago. I also had many perfermance problem related to the web interface. I personaly went with SpamTitan, sure you need to pay but the web interface rock.

[telackey]

I suspect I have the identation of the script wrong.

Any help in correcting the identation of the script

would be appreciated.

Harald,

Yes, it does sound like something is wrong with the identation.

I've put up a copy of it at http://www.redbudcomputer.com/howtos/spam/pfolders/spam_collect.py.

One will need to set the SERVER, USER, and PASSWORD variables, and optionally the names of the spam and ham folders.

I should mention, this script uses straight IMAP, so it should work with any IMAP server, not only Exchange.

Hope that helps!

[telackey]

I think it has been noticed that I have been silent for a while. In brief, personal, and honestly more critical, matters arose that diverted my attentions away from SV. I won't go into the details of them, but I do believe I will be able to start putting some focus back onto SV again.

My original intentions were to release an update to SV, SV2 as it were, around the start of the year. The matters alluded to above prevented that; but I will be resuming work. I'd made a good start on the v2 work, but I have more yet to do. I am also slowly beginning to work through my backlog of e-mail and posts. I do apologize for the delays in answering your questions.

Regards,

Thomas

[telackey]

Hi,

I have just installed this server and can see the

mails being downloaded into the spam server.

The users in the vadmin are there so ldap is working

OK but no mails are being delivered to my exchange

mailboxes.

I am using the fetch mode from my pop serving ISP. I

have configured my exchange server as an SMTP

device.

Any help would be much apreciated.

cheers

Jon

Were you able to get it working? If not, the first thing I would do is check /var/log/maillog (cat /var/log/maillog | less) to see what is becoming of the messages. If I had to guess, there may be a DNS or firewall issue which is frustrating delivery.

[Harald_Bessels]

Dear telackey,

I'm happy to hear You have time again for supporting SV. Thank You for the link for the spam collect script. I indeed had the indentation wrong.

I have 2 remarks for Spam Vigilante:

1) I receive email where the email address is written in capitals. In the spam-viewer website these aren't displayed for the user. Only vadmin can see them by viewing all spam/blocked messages.

2) We receive a lot of picture spam. I ran into an add-on for spamassassin called FuzzyOcr. I installed this on our machine and it stopped most of the picture spam. If you're interested, I can send You the howto I used.

Regards,

Harald Bessels.

[brightdog]

Hey Telackey,

Glad to hear you're back on board, and I hope whatever critical issues you've had to deal with are in the past now and that they turned out well.

Hi Harald,

I'm interested in your How To on setting up FuzzyOCR. I've been struggling with how to deal with the Picture Spam, and it sounds like you've got a beat on it.

Thanks,

-Matt

[Grifter75]

Forgive the windows guy in advance.

Does anyone know how to automate the spam_collect.py script so it will run on a scheduled basis?

also just wanted to say thank you Telackey for this project. I support a very small non profit organization that would have no option to purchase an anti-spam product (like some other guy on this forum recently sugested). For us a tool like this is invaluable, thank you for providing us with a solution that has so many features that the for pay services provide and also something that is open source.

You rock.