VMware Virtual Appliances Community
andy_mac
Enthusiast
Enthusiast

ESVA 1.6

This is the forum to discuss ESVA 1.6 matters.

VMTN Appliance listing: http://www.vmware.com/vmtn/appliances/directory/542

Website: http://www.global-domination.org/ESVA/16

-Andy

Reply
0 Kudos
118 Replies
_Real
Contributor
Contributor

Andy, Awesome product![/b] This tool you created is perfect for those of us who run their own personal small time mail servers. Please tell us you will keep up the good work and keep it free for us little people!

The only problem I'm having seems to be the DCC checking, it almost always says "3.00 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)" even for special one time custom generated emails which can't be listed in the DCC. Any idea why this would be happening and the best way to resolve the issue?

Also, is there any way to see statistics for greylisting? I was running v1.6.4.5 where greylisting was not running and seen a lot of Spam being caught, but with v1.6.4.6 MailWatch sees almost no Spam which means greylisting is working VERY well. Just wondering how well.

Reply
0 Kudos
LogIQ
Contributor
Contributor

julian_o_brien

Can you be a little more specific on where in /etc/init.d/postgrey to insert the delay line?

Also is the number 1 in seconds?

Thx.

Ulrich

Reply
0 Kudos
LogIQ
Contributor
Contributor

I also think it would be nice to be able to see what is delayed...

I have allready encountered a friend whose emails to me returned as rejected from his mailserver.

Also it would be nice to overview the delayed messages, so one could manually whitelist those that are definately not spammers.

\- Ulrich

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

Ulrich,

Here is the section in /etc/init.d/postgrey:

DBPATH=/var/spool/postfix/postgrey

SOCKET=$DBPATH/socket

OPTIONS="--inet=127.0.0.1:60000 --delay=1"[/b]

\# Source an auxiliary options file if we have one, and pick up OPTIONS,

As for how long the delay is... i'm no postgrey expert or nuthin' so I don't know if it's minutes of seconds, but i'm pretty sure its seconds. It has allowed more spam through than previously, though, but so far I'm still happy about the trade off.

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

as far as viewing the postgrey delays...

I have a little php script that parses the maillog and shows whats been delayed and what has been sent for each user. I'll put the code up here when I get to work tomorrow. It's really very primitive now, but its still useful.

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

Cheers.

I think there will always be a need for a free product - just like HedHat have Fedora, I think I'll always have a free version of ESVA...

Anyway. Not sure about the DCC check - sounds like a strange one to me... Maybe someone else has seen something like that?

For stats, the best way is to point your browser at esva (http - not https for webmin) and click on the mailgraph link. The rejected messages are from postgrey and postfix checks (mostly postgrey though)

-Andy

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

Nice one - I'd like to see that one - With your permission I could host it on the site for others to download and install.

-Obviously credit would be given for this and any other contributions!

-Andy

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

Unfortunately there are some poorly implemented (RFC wise) MTAs out there that:

a) Don't interpret the 450 message correctly

b) Don't retry at all or have such a long retry that it's as good as useless.

If anyone comes across any domains that don't cope with Greylisting, please let me know so that I can update the default postgrey whitelist and make it available.

Clever regex is always appreciated in these whitelist rules rather than screens full of IP addresses...

-Andy

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

I think that this forum is too limited for the wide ranging discussions that are held in here, so I'm thinking about adding a forum to global-domination.org.

Does anyone have any suggestions?

(it's a shared (linux based) service, but can utilise PHP/CGI and MySql...)

-Andy

Reply
0 Kudos
yjchung
Contributor
Contributor

phpbb is what I'd recommend for free board... but I've had some problems with it not coping very well for high traffic sites... shouldn't be a problem for what's been discussed here so far.

Reply
0 Kudos
LogIQ
Contributor
Contributor

Andy

That is why I would like to manually be able to browse the Postgrey logfile, so I can sleep a night, knowing that my customers mails gets through.

Well for a start gmail.com. Sending from my google mail resultet in a returned failure message instantly.

\- Ulrich

Reply
0 Kudos
LogIQ
Contributor
Contributor

julian_o_brien

I am looking forward to that script. I must admit that I am a little worried/sceptic about the Postgrey feature as it removes 99.9% og all my spam. I do get a lot of "real" mails through, but I would like to see what is beeing rejected and what is not, and then form a large whitelist, just to be sure.

\- Ulrich

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

I'm cleaning up my script a bit so I won't be terribly embarrassed when the public at large looks at it (blush). Also, I have to change permissions on /var/log/maillog, and I'm not sure if logrotate will reset these permissions. (if anybody has any ideas on this then I'm all ears). Hopefully I'll post up a basic version later today. Thanks for the interest.

- Andy: All the credit goes to you my man. This is a great virtual machine.

Oh, and one more thing. Any real email that doesn't get through because of postgrey, is only because of a poorly configured mailserver on the other end. I've already called a couple ISPs to have them fix their mailservers response to 450 errors. They are always responsive when you say "Your mail server is broken and here's why..."

Message was edited by:

julian_o_brien

Reply
0 Kudos
LogIQ
Contributor
Contributor

julian_o_brien

Looking forward to trying your script.

I know how postgrey works, and why some mailservers fail, but that does not change the risk of a very business critial mail, or a mail from a potential big new customer goes to the trash because of postgrey.

\- Ulrich

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

Agreed on the missing email thing - That's why Julian's script will be so useful.

-Have you considered running a cron job hourly that just updates a web page containing the postgrey details?

That way the cron job can run as root and not have issues with permissions, but wouldn't be as up to date as your php script.

Also (thinking aloud now) maybe put a job in cron.houly to reset the permissions on the rotated log file???

-Andy

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

o.k. so a very basic script here. i use one that's a little more complex (searches by user name) but this one gives the basic idea. i'll be working on it to make it more useful and i'll keep everybody updated.

two files->

1st file is scanLog.php

it searches the maillog for delays and sends

<?php //open the maillog file, I set mine to world readable (chmod o+r /var/log/maillog) $file = '/var/log/maillog'; $file = file($file); $delays = array(); $sents = array(); foreach ($file as $key => $line) { if (strpos($line, "NOQUEUE")) { $pos1 = strpos($line, 'from='); $pos2 = strpos($line, 'to='); $delays\[substr($line, $pos1 + 6, $pos2 - $pos1 - 8)] = substr($line, 0, 15); } elseif (strpos($line, "hold")) { $pos1 = strpos($line, 'from='); $pos2 = strpos($line, 'to='); $sents\[substr($line, $pos1 + 6, $pos2 - $pos1 - 8)] = substr($line, 0, 15); } } foreach ($sents as $key => $sent){ if (isset($delays\[$key])){ unset($delays\[$key]); } } unset($key, $file); //display the data echo '<pre><body style="font-family: courier">'; echo 'h2. Delays: '; echo '<p>These email address were told to resend, but have not done so successfully yet.<br/>'; echo 'They may still try to resend in the future.</p>'; foreach ($delays as $key => $delay) { echo $delay . ' - ' . '[' . $key . ' | searchLog.php?search=' . urlencode($key) . ']</br><//br>'; } unset($key); echo '<br/>'; echo 'h2. Sents: '; echo '<p>These emails were succesfully sent.</p><br/>'; foreach ($sents as $key => $sent){ echo $sent . ' - ' . '[' . $key . ' | searchLog.php?search=' . urlencode($key) . ']</br><//br>'; } echo '</body></pre>'; ?>

2nd file is searchLog.php

it simply searches the log for the email address in the first script, so you can get a better idea as to what exactly happened. you'll see.

you can also use searchLog.php to search for any string you want.

<?php if (!isset($_GET\['search'])){ echo '<form action="searchLog.php" method="GET">Search String:<input type="text" name="search" id="search"><input type="submit" value="search"></form>'; exit(); } $file = '/var/log/maillog'; $file = file($file); echo '<xmp>'; $delays = array(); $sents = array(); foreach ($file as $key => $line) { if (strpos($line, $_GET\['search'])) { print_r($line); } } echo '</xmp>'; ?>

copy this text into two files and upload them to esva at /var/www/html with webmin and point your browser to http://esva.ip.address/scanLog.php.

as far as the permissions thing goes, it appears logrotate will keep the original permissions (whatever that means for sure i'll find out). if it does reset the permissions on maillog then logrotate can be fixed to run chmod right after it rotates.

Message was edited by:

julian_o_brien

Reply
0 Kudos
LogIQ
Contributor
Contributor

julian_o_brien

Thank you for the script, it works perfectly!

Someone will eventually help you crack the permissions issue, I am sure.

May I propose a change to the script? I am no programmer, and therefore have no idea as to how difficult it is to do.

Would it be possible to read out the filter for the user logged on to MailWatch, and make your script sort results by these filters?

If user@domain.com is logged on MailWatch and is only able to see mails to user@domain.com, scanLog.php would also sort to only show entrys for this e-mail address.

If possible one could just make a link to it from Tools/Links and all users could react if any errors should occur.

Thanks again!

\- Ulrich

Reply
0 Kudos
julian_o_brien
Enthusiast
Enthusiast

Yeah, to have the search limited by user was my original idea and will be the final goal.

I've changed my approach at this point. My new idea is to use a cron job that runs postgreyreport (which shows all emails that havent gotten through) and then parse it's output file. It seems to work a little cleaner than having me parsing each line in the maillog (though I still want to do this for searchLog.php).

I'll keep this place updated.

Also, I agree that a phpbb forum would be great for this.

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

Great work - and thanks again!

There seems to a bit of a community springing up around ESVA so I will mostlikely setup a phpbb based forum on global-domination.org.

\|'ll keep you posted re: this.

At some stage I think that I'll create a maintenance release (aim is to upgrade existing 1.6 installs inplace) to include this sort of reporting as well as a few other minor enhancements.

-Andy

Reply
0 Kudos
andy_mac
Enthusiast
Enthusiast

I've tried this out, but all I get back is blank screen once i've set the perms on /var/log/maillog to 644. (same with 777).

Any idea?

-Andy

Reply
0 Kudos