VMware Virtual Appliances Community
VMTN_Admin
Enthusiast
Enthusiast

Spam Vigilante - Mail Filter Virtual Appliance

http://www.vmware.com/vmtn/appliances/directory/255

A mail proxy based on FreeBSD with spam (SpamAssassin) and virus (ClamAV) scanning. Can be used with any existing mail system.

Reply
0 Kudos
553 Replies
Arken_420
Contributor
Contributor

I know that this forum is really for troubles with Spam Vigilante, but I just didn't know where else to turn. I don't like sifting through tons of mailing lists. Anyways, I'll shoot and see what happens:

I have installed all prereqs for FuzzyOCR and copied the files needed, FuzzyOcr.pm and FuzzyOcr.cf to /usr/local/etc/mail/spamassassin, and /etc/mail/spamassassin. Of course nothing happens with I run spamassassin on the test samples, oh SA works just fine, but I don't get any output from FuzzyOcr. The log file is writtable and the db is as well, I have three entires in the log file talking about my word list, but that was simply renamed to FuzzyOcr.words and haven't seen a thing since then. I've restarted spamassassin over and over with no luck. I'm really wondering if this is an easy fix and if someone could let me know. Thanks.

BTW, Spam Vigilante is freaking awesome, I use at work and home!!!

Reply
0 Kudos
netmavrik
Contributor
Contributor

I use FuzzyOCR myself. I can tell you that with DCC, Pyzor, Razor2 and a properly tuned ruleset, bayes and AWL working FuzzyOCR only assists with maybe 1 out of 1000 of my SPAMS. When I first started using FuzzyOCR it was more useful but I wasn't using enough SARE Rules. I am debating removing it from my SV. Just thought I would share that.

Are you running this under SV or FreeBSD? You don't need any files copied to /etc/mail/spamassassin. Make sure you edit the FuzzyOCR.cf file for the correct paths to the bin files. The default file isn't FreeBSD friendly.

To test, go to /var/virusmails and run:

spamassassin -D FuzzyOCR -t < spam-(some-spam-message).

It might not add to the resulting score, but the debug output should mostly be FuzzyOCR related.

Reply
0 Kudos
dajabo
Contributor
Contributor

Arken, I also had to change the paths to the helper applications in the FuzzyOcr.pm[/b] file, which I think is in the same directory as the FuzzyOcr.cf file. This was supposed to be done during set-up but wasn't. Check they are right.

FuzzyOcr works well for me. A good deal of the spam I am getting caught by SV gets hits by FuzzyOcr. Though there is a setting in the cf file I think or the amavisd.conf file which only fires up Fuzzy when the score from regular rules is below a certain amount. Like netmavrik, the SARE rules also helped my scores, but stuff does slip through them which gets caught by fuzzy.

Reply
0 Kudos
dajabo
Contributor
Contributor

netmavrik, have you managed to get blacklist tests working? I have enabled mine but all they ever do is time out. Razor is working, and DCC, but I have tried maxing out the timeout and the BLs still don't return a result.

Anyone?

Reply
0 Kudos
netmavrik
Contributor
Contributor

The timeout that you are referring to might be misleading. I still get that when I review spamassassin -D --lint, as well.

In my case, both the local.cf and amavisd.conf files needed to be set to reflect the ip address block that the clients isp uses for their smtp servers.

That specific SV configuration is unique among my others. Where I have deployed SV as a true front-end server with a MX record pointing to it, I haven't had to do anything to get the RBLs to work.

When you edit those files, make sure to add 127.0.0.0/8 as well. Not sure why, but you will need it.

Let me know.

Reply
0 Kudos
pperkins
Contributor
Contributor

I'm installing for a SBS2003 client. I have managed to get Exchange Integration working & LDAP lookups are OK.

Using fetchmail config if I pull the mail from their ISP it seems to disappear into a black hole.

I did get 550 relay not allowed messages, this was an error in the fetchmail.cf which I corrected, then I got postfix errors

<accounts@xxx.local>: Host or domain name not found. Name service error

for name=srvr.xxx.local type=A: Host not found

which were resolved by using the SBS as the primary DNS server.

After that though I dont get any messages about relay issues, or domain not found but mail doesnt get to the users' exchange mailbox.

can anyone help please?

thx

paul

Reply
0 Kudos
telackey
Contributor
Contributor

Using fetchmail config if I pull the mail from their

ISP it seems to disappear into a black hole.

<accounts@xxx.local>: Host or domain name not found.

Name service error

for name=srvr.xxx.local type=A: Host not found

ich were resolved by using the SBS as the primary DNS

server.

Yes, this is an important point, which I may stress in the manual or setup later. If you use a hostname for the destination mail server, SV needs to be able to resolve it. That makes sense, naturally, but it is easy to forget if one is more accustomed to entering the ISP's DNS server.

After that though I don't get any messages about relay

issues, or domain not found but mail doesnt get to

the users' exchange mailbox.

One quick thing is that I hope you are using the "keep" option in fetchmail, so that this mail isn't being deleted.

For a possible answer, check that SV can access the Exchange server over SMTP by:

telnet 25

It should be able to connect. My guess is that it cannot. If the name cannot be resolved, it will bounce on delivery. If it can be resolved, but cannot be reached, it will queue the messages. You can check the size of the queue with:

postqueue -p

It will normally be empty; if it isn't, you have a likely culprit. Since that is a network issue, you will probably need to track it down elsewhere than SV, maybe a firewall on the Exchange box, for example.

If that does not help, can you post some parts of the maillog for a delivery? Best of would be for a blocked delivery and for a clean delivery.

Reply
0 Kudos
pperkins
Contributor
Contributor

THX

I have left the keep option on.

the postqueue -p showed that the FQ server name was resolving to the internal IP address not the external. I reconfigured to use the IP on the external interface and the queue reduced from 48 to 10 in a matters of minutes.

i have now had test mails bounced back with 550

SMTP error: 550 <user@domain.local>: Recipient address rejected: User unknown in relay recipient table

next clue plse?

paul

EDIT

main.cf, transport.cf & exchange_recipients in /usr/local/etc/postfix show both the external (.com) & internal (.local) domains

Reply
0 Kudos
telackey
Contributor
Contributor

i have now had test mails bounced back with 550

SMTP error: 550 <user@domain.local>: Recipient

address rejected: User unknown in relay recipient

table

next clue plse?

Well, not exactly a clue as it is not exactly a puzzle, but there are a few options:

First thing, naturally, is to make sure the recipient exists on the destination server and is entered correctly in fetchmail.cf--no typos, etc.

Once that is confirmed, the next step is to make sure:

A. The address exists in LDAP.

B. The address has been synced to the recipients list* (/usr/local/etc/postfix/exchange_recipients).

If the answer to A is no, you should add it if possible. After it has been added, or if it is already there but missing from the list, try syncing by running: /etc/periodic/daily/474.m-postfix-accounts.

If it \_still_ isn't there, or if for some reason you cannot add it to the LDAP (ie, AD) server, I would recommend not using that feature. It really doesn't have the same level of import when used with fetchmail that it does when used as an external SMTP server, so there is little harm in turning it off. To disable the relay recipient checking, comment out the the relay_recipient_maps line in /usr/local/etc/postfix/main.cf (the last line) thus:

#relay_recipient_maps = hash:/usr/local/etc/postfix/exchange_recipients

Then restart Postfix:

/usr/local/etc/rc.d/postfix restart

\* This sync step should be eliminated in the next release, but is needed now.

Message was edited by:

telackey

Reply
0 Kudos
pperkins
Contributor
Contributor

hi

still not getting mail into the exchange server, both domains listed in relay_domains.

checking maillog shows postfix trying to deliver to localhost

paul

Reply
0 Kudos
telackey
Contributor
Contributor

still not getting mail into the exchange server, both

domains listed in relay_domains.

checking maillog shows postfix trying to deliver to

localhost

Let me explain the parts a bit, fetchmail version, which might help you track it down.

1. fetchmail. Fetchmail will recieve the mail over POP3 or IMAP and inject it into the local SMTP server (Postfix).

2. Postfix. The SMTP server. It receives the mail from fetchmail, and forwards it to amavisd-new. Amavisd-new filters the mail and then forwards it back to another Postfix listener. This second one looks at a few parts to determine what to do with the message. A. The relay recipients. This is a list of e-mail addresses which are allowed to receive mail. B. The relay domains. These are the domains which are allowed to receive mail. C. The transport maps. This tells Postfix what to do with the mail for a given domain, that is, where it should go. This is the ultimate destination of the mail.

A very small amount of mail is delivered to localhost, these are messages for root, such as the daily system status reports, RulesDuJour update notices, etc. No mail for any of the domains listed in the transport maps will be delivered to localhost, however. I am skipping over amavisd-new, spamassassin, etc. as they are not directly related to delivery.

I can't determine exactly where the problem you are experiencing is located based on your description, but my recommendation is this:

1. Make sure all the info is fetchmail.cf is correct, specifically the mapping between the external account being checked and the internal account that is the destination.

2. Comment out the relay_recipient_maps line in main.cf to disable the recipient checking. It isn't really required when using fetchmail.

3. Check that the domains are listed in /usr/local/etc/postfix/relay_domains properly. Sounds like you have already checked this.

4. Check that the domains and destinations are listed correctly in /usr/local/etc/postfix/transport. To avoid any problems, use the IP address of the destination server, such as, "smtp:\[192.168.100.100]".

If you make any changes to transport or relay_domains files, remap them by running "postmap ".

Re-running setup is also an option. Assuming no network issues, one should have a working appliance after setup is complete as it sets all these based on user input.

Reply
0 Kudos
phoenixsecure
Contributor
Contributor

What is this OSS project?

Reply
0 Kudos
pperkins
Contributor
Contributor

hi

have re-run setup again, i have verified that fetchmail.cf is OK, using telnet and the user account info to connect to the remote server.

I then telnet to the Exchange server and can send a message to each user's mailbox in turn.

main.cf - # at start of relay_recipient_maps line

relay_domains has the external & internal domains (.com & .local) listed

the transport has the exchange IP address for both domains

can telnet and send mail to the exchange smtp using the FQDN or ip address

/var/log/maillog has

fetchmail connecting to the remote server

verifying the number & size of messages

then get smtp connect to localhost failed

smtp transaction error when collecting from mailbox@domain[/i] and delivering to smtp host localhost

i can telnet to the ip address of the applicance on port 25 ok

paul

Reply
0 Kudos
phoenixsecure
Contributor
Contributor

Hi, I edit the clean-spam.sh and put a smaller amount for the database. So far nothing append, do I have to run a script or is it suppose to shrink the database by itself?

Thanks.

Reply
0 Kudos
pperkins
Contributor
Contributor

hi

think its sorted, found this

http://www.catb.org/~esr/fetchmail/fetchmail-FAQ.html#R1

the default entry for localhost points to a host at redbudcomputers

edit /etc/hosts to make ip & host for appliance localhost too and it communicates OK

thx

paul

Reply
0 Kudos
telackey
Contributor
Contributor

hi

think its sorted, found this

http://www.catb.org/~esr/fetchmail/fetchmail-FAQ.html#

R1

the default entry for localhost points to a host at

redbudcomputers

edit /etc/hosts to make ip & host for appliance

localhost too and it communicates OK

thx

paul

Paul,

Excellent catch! I am glad you got it working. I am surprised that was set that way. I'll make sure that is corrected for the next release.

Reply
0 Kudos
telackey
Contributor
Contributor

Hi, I edit the clean-spam.sh and put a smaller

amount for the database. So far nothing append, do I

have to run a script or is it suppose to shrink the

database by itself?

Thanks.

Yep, run the clean-spam.sh script and it will delete the stuff. Later, /etc/periodic/daily/476.m-spamviewer will run and pare down the DB. If you want it to cut down right away, just execute it as well after clean-spam. Normally clean-spam is also run automatically through /etc/periodic/daily/475.clean-spam.

There will be changes on both these fronts for the next release that should make it much simpler.

Reply
0 Kudos
Nukemizer
Contributor
Contributor

telackey,

No matter what you do, I for one heap LARGE amounts of praise upon you

for creating such a wonderful tool !!

We have never experienced such reduced levels of spam.

thank you thank you

Reply
0 Kudos
phoenixsecure
Contributor
Contributor

Ok tried clean-spam.sh (after putting in databse size to 20 megs) but it does noting, if I look into the script it does not do anything with the database, unless I am mistaken.

If I look in /usr/home/spamviewer/db I can see that the spam.db is almost 500 megs in size. Even after running clean-spam.sh and 476.m-spamviewer the size is still 500 megs. Right now my user cannot even look at there quarentine because its way too slow, I have almost 20,000 mails in the db. I really need to bring down the db size. Any idea why its not working.

Thanks.

Reply
0 Kudos
telackey
Contributor
Contributor

Ok tried clean-spam.sh (after putting in databse size

to 20 megs) but it does noting, if I look into the

script it does not do anything with the database,

unless I am mistaken.

Nope, you are 100% correct. The first script only deletes old messages from /var/virusmails. The size for the script doesn't directly have to do with the DB size, rather the total size of the files in that directory.

The way the viewer works in this version is to index all the messages beneath /var/virusmails into a SQLite database. When the indexing script runs, it removes from the DB messages no longer on disk, and adds any new ones.

If I look in /usr/home/spamviewer/db I can see that

the spam.db is almost 500 megs in size. Even after

running clean-spam.sh and 476.m-spamviewer the size

is still 500 megs. Right now my user cannot even

look at there quarentine because its way too slow, I

have almost 20,000 mails in the db. I really need to

bring down the db size. Any idea why its not

working.

Well, I can't really tell, but since you are in a rather serious spot, this is what I would do:

A. Find out how many messages are under /var/virusmails.

ls /var/virusmails | wc -l

B. Run the clean-spam.sh script.

C. (opt.) Check the number of messages again.

D. Turn off the Spam Viewer. This is just in case there are any locks on the DB which cannot be resolved.

/usr/local/etc/rc.d/apache2.sh stop

E. Re-index the DB. This is the same script as m-spamviewer.sh runs, but we are going to use more agressive options. This will delete all existing messages from the index and re-index from scratch. When done, the count of items in the 'messages' table should be nearly the same as the output from step C, if not identical.

python /usr/home/spamviewer/maintain_db.py --clean-msgs

F. Restart the viewer.

/usr/local/etc/rc.d/apache2.sh start

Thanks.

Np, hope this works for you!

Reply
0 Kudos