VMware Virtual Appliances Community
VMTN_Admin
Enthusiast
Enthusiast

Spam Vigilante - Mail Filter Virtual Appliance

http://www.vmware.com/vmtn/appliances/directory/255

A mail proxy based on FreeBSD with spam (SpamAssassin) and virus (ClamAV) scanning. Can be used with any existing mail system.

Reply
0 Kudos
553 Replies
dinop
Contributor
Contributor

Is there a way to download it?

Reply
0 Kudos
nshastings
Contributor
Contributor

Has anyone gotten the exchange integration to work? After multiple tries, i still get "can't connect to ldap server".

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

I've got the exchange integration to work after a little playing around.

I had already setup a basic ldap querying account in active directory (from using the adldap php class) and i just pulled the information from there so I don't really remember what I originally did to get it to work. However, I seem to recall playing around with the base dn before i got it to work (mine is DC=[i]companyname[/i],DC=local).

Also I have people with hypens in their last names (myself included), so I had to add proper character escaping in /usr/home/spamviewer/elr.py i.e. replace one single quote with two single quotes:

I added the line

user.dn = user.dn.replace("'", "''")[/i]

right before the sql insert command:

sql = "INSERT INTO usernames(username) VALUES('%s')" % user.dn[/i]

*Note: This fix doesn't help the web interface escape the single quotes and if I want to view quarrantined messages for people with hyphens in the name I have to add the extra single quote to the GET value of the url.

(I wish I had posted this right after i did it because I kinda forget, but...)

One last thing I had to change for the spam headers to be added to messages and the spam subject line was to change (I think) /usr/local/etc/amavisd.conf. The $my_domain variable was setup as my local domain (.local) and not what all the mail was being received and forwared as (.com). I had to change $my_domain from companyname[/i].local to companyname[/i].com.

I think that was it.

It's working great now!!!!

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

Has anyone gotten the exchange integration to work?

After multiple tries, i still get "can't connect to

ldap server".

Could it be a DNS issue or maybe a user account issue. What user account are you using to login to the ldap server (AD root domain controller)? Like I mentioned in my previous post, I don't remember much of what I did to setup my ldap query account but I can check it out and get back to you.

For testing purposes you could attempt to use the IP Address of the root domain controller and use an administrator account as the ldap query account. (Of course it would be a terrible idea to keep it this way for any period of time.)

Reply
0 Kudos
telackey
Contributor
Contributor

nshastings:

Could you post more details on your configuration? Specifically the version of Windows and Exchange, the base DN you are using. I would be glad to help.

Reply
0 Kudos
telackey
Contributor
Contributor

Mr O Brien,

Excellent catch! I'll look into adding the appropriate changes to the initial configuration.

Until the contest is complete I will not be able to update the appliance directly. However, I will be making patches available to any who ask as they become available.

Currently patches are available for the two issues mention in section 4 (Troubleshooting) on the appliance description page.

I'll be making patches for the issues you describe as well.

Edit 06/30/06, 14:29:

I have a preliminary patch available that addresses the issue of apostrophes, and other characters, in usernames. This fixes it both on the elr.py side, and in the UI. I haven't had the opportunity for extensive testing of the changes as yet. If you would like to test the patch, please contact me at the address listed on Spam Vigilante VM page in section 4.

Message was edited by:

telackey

Reply
0 Kudos
nshastings
Contributor
Contributor

Thank you for the help and sorry I didn't respond back on here sooner. I am trying it on Windows 2003 Server with Exchange 2003. This is a test setup at home and the domain is hastings.home (DC=hastings,DC=home). I have tried this using my main administrative account to no avail after it failed with a lesser privileged account. I have trying using the hostname, fqdn, and ip of my single domain controller. I don't have a clue where the problem lies.

Edit: forgot to say thanks to telackey for creating this vm, which (once i get it working) will be quite handy

Message was edited by:

nshastings

Reply
0 Kudos
telackey
Contributor
Contributor

nshastings,

My pleasure! And don't worry, you can save your thanks till it is completely working for you. Smiley Happy

My first step would be to verify basic network connectivity and DNS. To do that I would first try pinging the IP address of the Windows domain controller, and secondly execute:

nslookup -D "administrator@mydomain.dom" -W -s sub -LLL -b "DC=mydomain,DC=dom" '(&(|(objectclass=user)(objectclass=contact)(objectclass=group))(proxyAddresses=smtp:*))' dn

Once that query is executing successfully, you should try it as a less privileged user and check that it still works as well.

Please tell me how this goes for you, or if there is any additional information I can provide. I’m pleased to help.

Reply
0 Kudos
telackey
Contributor
Contributor

Web Site:

I am pleased to announce that I have brought up a website where the documentation can be viewed and updates downloaded.

Updates have been coming at a fairly quick pace, so you may wish to check back now and again:

Spam Vigilante Page:

http://www.redbudcomputer.com/spamvigilante.htm

Documentation:

http://www.redbudcomputer.com/vmdocs/spam/mailfilter-documentation.htm

Download Page:

http://www.redbudcomputer.com/downloads.htm

Enjoy!

Edit:

In addition to the page, new updates were added today to fix some recent issues:

1. Fix a problem downloading messages other than spam as a .eml file.

2. Prevent caching of messages with banned or infected attachements.

3. Display a warning before viewing an infected message.

The full list of changes since the release is available at: http://www.redbudcomputer.com/updates/spam/CHANGES.txt

Reply
0 Kudos
nshastings
Contributor
Contributor

Thank you, telackey.

I know the network configuration is correct and working because I was already able to ping the hostname of the ad server. I will check the ldap when I have a chance.

I appreciate your assistance.

Edit:

Both queries were successful using the same info I used in the configuration script. I used both an administrative user and the lower-privileged user I have for this purpose, both working. Yet, at the end of the script (using same info) it still says can't contact ldap server, and subsequently connecting to the web interface shows no users.

I don't know where the problem lies.

Message was edited by:

nshastings

Reply
0 Kudos
telackey
Contributor
Contributor

nshastings:

The next thing to look into is that all the informationfrom the error you are encountering, particularly the server namewas saved correctly to the configuration file.

To do so, you'll want to check /usr/home/spamviewer/config.py

The server name is stored in the variable LDAP_SERVER, and the user auth info is stored in LDAP_SEARCH_USER and LDAP_SEARCH_USER_PW.

If you would like, you can e-mail me that file as well. I'd be glad to take a look. My address is listed on section 4. of the appliance page.

Another option is to step back through setup and check it that way. To do so, execute 'touch /reconfigure' and reboot. When you log back in you'll be stepped back through setup. All your previous answers will be saved (except for the ldap user password), so it is pretty quick to step through tapping "Enter."

Message was edited by:

telackey

Reply
0 Kudos
telackey
Contributor
Contributor

Update Notice:

A new update has been released for Spam Vigilante. This addresses the second issue discovered by Mr O'Brien: that the domain name in amavisd.conf may not be correct if the internal mail domain name is not actually the domain that mail is being delivered to. This is an uncommon case. Users with more than one relayed domain can also be affected, and should apply the update to ensure all domains that are handled are scanned.

The solution employed is to use a map file, /usr/local/etc/postfix/relay_domains, that sets the allowed domains for Postfix and is used by Amavis for its local_domains list.

To activate the change, it is necessary to restep through setup, followed by either rebooting the appliance (recommended) or restarting Postfix and Amavis. If the issue addressed is not affecting your deployment, there is no need to re-run setup. This fix has not yet received extensive testing, and though there are no known issues, any feedback is appreciated.

The update package also contains all previous updates.

Instructions are available at:

http://www.redbudcomputer.com/updates/spam/INSTALL.txt

Download at:

http://www.redbudcomputer.com/downloads.htm

Message was edited by:

telackey

Reply
0 Kudos
jobber_jobber
Contributor
Contributor

Hi

Firstly, Telackey - thank you so much for producing an appliance I have been waiting for - my spam problem has been driving me mad!

Now down to business, I've been running this appliance for a couple of days, and would like to know a few things, whether they are possible or not. Note, I'm using D_BOUNCE instead of D_PASS for spam - because I'm trying to reduce the number of spams received by users, rather than just tagging them.

1. I am using the appliance to filter mails for a number of domains, and would like for a single user account in the viewer to be able to view all the currently quarantined messages for all mail addresses, on one single web page . Is this possible, and how do I do it?

2. The main reason I am running mail server, is because my previous provider was losing "ham" mails for me, as well as the spam! Therefore I want to be able to whitelist certain sending addresses and also certain domains. Is this possible please, and how is it done?

3. In the viewer, I set up a new user called "postmaster" using the Vadmin account, however, it says Bad Username or Password when I try to login? I tried setting up another, with the same problem, what may I have done wrong?

4. Is there anyway using the viewer to "release" a quarantined e-mail and have it sent on to the recipient, with an option to whitelist sender?

5. In addition to 4 above, perhaps for SPAM mails that have been classified in the maybe, maybe not category (rating 6.31 to 15...), could the user also get notified of the quarantined mail, with a link to a webpage to giving an option to delete or release?

6. And finally for now, is there any way of having statistics presented on a webpage, such as number of spam and/or virus e-mails blocked by domain, by day? Also, number of messages passed. And perhaps extra details by sender and recipient address?

Sorry for such a long list, but I'm so enthusiastic about this, I'd like to make maximum use of the tool. Hope I'm not asking too much.

Thank you once again telackey.

Regards,

jobber.

Reply
0 Kudos
telackey
Contributor
Contributor

1. I am using the appliance to filter mails for a number of domains, and would like for a single user account in the viewer to be able to view all the currently quarantined messages for all mail addresses, on one single web page . Is this possible, and how do I do it?

No, there wasn’t a way. But it is a good idea, so I have added it. I haven’t released the patch for it yet, but it will be in the next update, which I’ll release soon.

2. The main reason I am running mail server, is because my previous provider was losing "ham" mails for me, as well as the spam! Therefore I want to be able to whitelist certain sending addresses and also certain domains. Is this possible please, and how is it done?

Yes, but not through an automated mechanism. To do so, you need to add a whitelist map for Amavis. Check out http://www.ijs.si/software/amavisd/amavisd-new-docs.html for some good info. The documentation provides the proper information, but the basic idea is pretty simple, a list of e-mail addresses or partial addresses, one per line. If soft white listing (score_sender_maps), it is an address followed by white space and then the score.

3. In the viewer, I set up a new user called "postmaster" using the Vadmin account, however, it says Bad Username or Password when I try to login? I tried setting up another, with the same problem, what may I have done wrong?

Yes, I have an idea. I have introduced a bug in a very recent update where the password was no stored correctly when adding a new user. It is set correctly if you change the user’s password. I have fixed the issue, and will include it in the next update.

4. Is there anyway using the viewer to "release" a quarantined e-mail and have it sent on to the recipient, with an option to whitelist sender?

Perhaps. I have checked into it as it was one the initial requirements I listed when designing the product. This is actually much more difficult than one would expect. The reason is that since the mail has already been delivered, the true information on where it was coming from and where it was going that is part of the SMTP transaction is gone. There are a couple of ways around this, one of which would be to use a BSMTP transport for Amavis. That is a significant change to the workings, however. Another is to execute amavis-release on the appropriate message. I have worked on something to that end, but I am not ready to include it in the base product as yet. If you are interested I can send you some details. My second best solution to this problem was the download link that downloads the message in rfc822 format, which most, though admittedly not all, e-mail clients can handle and open.

5. In addition to 4 above, perhaps for SPAM mails that have been classified in the maybe, maybe not category (rating 6.31 to 15...), could the user also get notified of the quarantined mail, with a link to a webpage to giving an option to delete or release?

Possibly, this is a little tricky however, as it is really more on the delivery side than on the viewer side. Something might be possible to work out, but I would need to think on it.

6. And finally for now, is there any way of having statistics presented on a webpage, such as number of spam and/or virus e-mails blocked by domain, by day? Also, number of messages passed. And perhaps extra details by sender and recipient address?

Like number 5, this is a more on the delivery side, but I think it is an excellent idea. I’ll definitely be looking into ways to do this, but I can’t make any positive commitments as to when it would be ready.

Sorry for such a long list, but I'm so enthusiastic about this, I'd like to make maximum use of the tool. Hope I'm not asking too much.[/b]

My pleasure! I hope it works well for you!

Reply
0 Kudos
telackey
Contributor
Contributor

Update Notice:

A new update has been released for Spam Vigilante. This is a minor update. It adds one new feature, a "View All Spam" link for the "vadmin" user in the Admin Console. This allows the admin to view all spam messages for all users at once. There is one bug fix, correcting an issue introduced in an earlier update pack that prevents the password from being stored correctly when creating new users in the Viewer (local auth only, not pertinent when using Exchange auth).

The update package also contains all previous updates.

Instructions are available at:

http://www.redbudcomputer.com/updates/spam/INSTALL.txt

Download at:

http://www.redbudcomputer.com/downloads.htm

Reply
0 Kudos
jobber_jobber
Contributor
Contributor

Telackey,

Thanks very much for your comprehensive response - and also new release.

Have had a quick look at it, and it's a big step forward. Though I'm not sure I explained my requirements correctly for vadmin to review all blocked spam mails.

I need vadmin to be able to see quarantined mails to all accounts, whether there is a "viewer" user set up for them or not. I don't want to have to specify the recipient e-mail addresses anywhere to be able to view them. Would that be possible please?

Thanks for the link re whitelisting, I'll have a read and see what I can make of it.

Also, I appreciate some of my requests aren't simple to implement, but anything you can do to implement them would really very much be appreciated.

Regards,

Jobber

Reply
0 Kudos
feld
Contributor
Contributor

I'm so glad I found this Smiley Happy but let me get something straight....

you're using fetchmail if you want to pull your email down from somewhere and then forward it to a mail server

and the smtp setting is if you are having the mail sent directly to you and you want to filter and forward to a mail server

correct?

Just want to make sure this is doing what I think it is..... because the default description is a little misleading (for SMTP, anyway)

Thanks

Reply
0 Kudos
telackey
Contributor
Contributor

feld:

Sorry for the delay in replying. Yep, you have it exactly right. I'll look into clearing up the documentation on it.

Feel free to contact me if you have any more questions!

Reply
0 Kudos
telackey
Contributor
Contributor

jobber:

Quote:

"I need vadmin to be able to see quarantined mails to all accounts, whether there is a "viewer" user set up for them or not. I don't want to have to specify the recipient e-mail addresses anywhere to be able to view them. Would that be possible please?"[/b]

That makes sense.

I am planning to release a patch this weekend that will include this change. Regular work prevents doing many releases during the week, but I'll try to get things like this in when I have the time to work on them on the weekends.

My most immediate plans (though not necessarily for inclusion this week) are:

1. "View All" without entering email addresses

2. Delete a single message

3. Inject/release messages for delivery

Message was edited by:

telackey

Reply
0 Kudos