Thursday, February 28, 2008
How Do I Enable remote access to MySQL database server?
First, login over ssh to remote MySQL database server
Step # 2: Enable networking
Once connected you need edit the mysql configuration file my.cfg using text editor such as vi.
* If you are using Debian Linux file is located at /etc/mysql/my.cnf location
* If you are using Red Hat Linux/Fedora Linux file is located at /etc/my.cnf location
* If you are using FreeBSD you need to create a file /var/db/mysql/my.cnf
# vi /etc/my.cnf
Step # 3: Once file open, locate line that read as
[mysqld]
Make sure line skip-networking is commented (or remove line) and add following line
bind-address=YOUR-SERVER-IP
For example, if your MySQL server IP is 65.55.55.2 then entire block should be look like as follows:
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/English
bind-address = 65.55.55.2
# skip-networking
....
..
....Where,
* bind-address : IP address to bind to.
* skip-networking : Don’t listen for TCP/IP connections at all. All interaction with mysqld must be made via Unix sockets. This option is highly recommended for systems where only local requests are allowed. Since you need to allow remote connection this line should removed from file or put it in comment state.
Step# 4 Save and Close the file
Restart your mysql service to take change in effect:# /etc/init.d/mysql restart
Step # 5 Grant access to remote IP address
# mysql -u root -p mysqlGrant access to new database
If you want to add new database called foo for user bar and remote IP 202.54.10.20 then you need to type following commands at mysql> prompt:mysql> CREATE DATABASE foo;
mysql> GRANT ALL ON foo.* TO bar@'202.54.10.20' IDENTIFIED BY 'PASSWORD';
How Do I Grant access to existing database?
Let us assume that you are always making connection from remote IP called 202.54.10.20 for database called webdb for user webadmin, To grant access to this IP address type the following command At mysql> prompt for existing database:mysql> update db set Host='202.54.10.20' where Db='webdb';
mysql> update user set Host='202.54.10.20' where user='webadmin';
Step # 5: Logout of MySQL
Type exit command to logout mysql:mysql> exit
Step # 6: Test it
From remote system type command:
$ mysql -u webadmin –h 65.55.55.2 –p
Where,
* -u webadmin: webadmin is MySQL username
* -h IP or hostname: 65.55.55.2 is MySQL server IP address or hostname (FQDN)
* -p : Prompt for password
You can also use telnet to connect to port 3306 for testing purpose:
$ telnet 65.55.55.2 3306
SMTP and POP with SSL (ucspi-tls-qmail-howto)
Why You Want To Use UCSPI-TLS
UCSPI-TLS is a protocol for adding "delayed encryption" to Dan Bernstein's Unix Client/Server Program Interface protocol. "Delayed encryption" means a session starts off in plaintext, then a command is issued to turn on encryption, encryption is negotiated, and the session restart. This has become a very common way to handle encryption, because it simplifies client configuration and requires only one TCP port.
These are the goals of UCSPI-TLS:
- Simple, UCSPI-compatible use.
- Support for both traditional SSL and delayed encryption through STARTTLS or similar.
- All SSL/TLS code in one place.
- Minimal changes required for each server.
- Support for "privilege separation", so that encryption can happen in a low-privilege subprocess.
In particular, I believe the privilege separation feature increases your system's security significantly. It creates a dedicated process to handle each encrypted connection, and this process can change its root directory and switch to a low-privilege user and group. Because of its complexity, OpenSSL has had its share of security bugs. Doing encryption in a low-privilege process ensures that the impact of any security bugs is minimized.
How to install UCSPI-TLS for netqmail-1.05
UCSPI-TLS is currently implemented as patches to existing programs. To install it, first download the programs you want to use, apply the UCSPI-TLS patches, install the programs as you normally would, and finally make the appropriate changes to your configuration files.Here are more details on doing this with netqmail version 1.05.
- Become root, using su or sudo bash. Some of these steps require you to have superuser privileges, and its easier just to get them now than to switch back and forth throughout the process.
Do a base qmail install
- Install qmail according to Life with qmail , and make sure everything works with a standard setup.
Install a patched ucspi-ssl+tls
- Make sure you have a recent version of OpenSSL installed on your system, along with the files required for development with the library. If you don't already have it installed, this should be available from your OS install CDs, with a name like libssl-dev or openssl-devel. If all else fails, you can download OpenSSL and compile it yourself.
- Download ucspi-ssl 0.70, and unpack it. It will create a directory host/superscript.com/net/ucspi-ssl-0.70; cd into this directory. Here's some commands to cut-n-paste:
wget http://www.superscript.com/ucspi-ssl/ucspi-ssl-0.70.tar.gz
gunzip -cd ucspi-ssl-0.70.tar.gz |tar xf -
cd host/superscript.com/net/ucspi-ssl-0.70 - Download ucspi-tls patch to ucspi-ssl (you can also read the ucspitls for ucsp-ssl patch README), and apply the patch:
wget http://www.suspectclass.com/~sgifford/ucspi-tls/files/ucspi-ssl-0.70-ucspitls-0.1.patch
patch -p1 < ucspi-ssl-0.70-ucspitls-0.1.patch - If anything is already using an older copy of sslserver, shut it off now.
- Compile and install the patched ucspi-ssl+tls, by running package/compile base then package/install base:
You now have a patched copy of sslserver in /usr/local/bin/sslserver. Congratulations!package/compile base
package/install basePatch qmail
- The next step is to patch qmail to add TLS support to its SMTP and POP servers. If you followed Life with qmail religiously, your qmail source code will be in /usr/local/src/netqmail-1.05/netqmail-1.05. cd to that directory, or wherever you have the qmail source:
cd /usr/local/src/netqmail-1.05/netqmail-1.05
- Now download the ucspi-tls patch to netqmail (and read the ucspitls patch for netqmail README), and apply the patch:
wget http://www.suspectclass.com/~sgifford/ucspi-tls/files/netqmail-1.05-ucspitls-0.3.patch
patch -p2 < netqmail-1.05-ucspitls-0.3.patch - If qmail is running, shut it down. On a Life with qmail setup, you do this by running:
/var/qmail/bin/qmailctl stop
- Recompile and re-install qmail by running make setup check:
You now have a copy of qmail-smtpd and qmail-popup with TLS supportmake setup check
Make your certificates
- Next you'll need SSL certificates. These certificates contain the encryption keys used to communicate with your servers, and possibly a signature from a trusted authority confirming your server's identity. The first thing you'll need is a place to store them. Let's put them in /var/qmail/ssl:
mkdir /var/qmail/ssl
chown root /var/qmail/ssl
chmod 700 /var/qmail/ssl
cd /var/qmail/ssl - Create the SSL keys for your system. We'll store the certificate in a file named cert, and the key in key. If you already have certificates you'd like to use, separate out the certificate part (the key part will generally be labeled RSA PRIVATE KEY, and the cert part CERTIFICATE. You can just copy these sections from your certificate file, and paste them into the appropriate files.
You'll have to decide whether you want to use an unsigned certificate or a certificate signed by a Certificate Authority (CA). If you hire a CA to verify your identity and sign your certificate, clients are more likely to accept this certificate without prompting the user. If you use your own unsigned certificate, clients may have to confirm that they trust your certificate.
If you'd like to use a signed certificate, choose a CA, and follow their directions to obtain a signed certificate. Save the certificate in a file called cert, and the key in a file called key. If you just have one file containing both of these, see the beginning of this step for suggestions on how to split it up.
If you'd like to use an unsigned certificate, follow these steps:
- Make sure openssl is installed on your system.
- Set a umask that will keep your files protected, by running:
umask 077
- Generate a certificate by running:
Answer the questions you're asked. Your password is temporary, so it doesn't matter what it is as long as you remember it for a few minutes; you can even write it down. Make sure you use the server's host name, as clients will be configured to connect to it, as the "Common Name" for the certificate. Note that this certificate will expire in 360 days, and you'll have to create a new one before then.openssl req -new -x509 -keyout key.enc -out cert -days 360
You'll now have a file called key.enc containing your encrypted key, and a file called cert containing your certificate.
- Remove the password from your certificate, so the server can start automatically. You can do this by running:
Enter the password you chose above, and you'll have a file called key containing an unencrypted copy of your key.openssl rsa -in key.enc -out key
- Create a Diffie-Hellman parameter file. I usually create 1024-bit dhparam files, but I'll admit I don't know exactly what they do. I should probably find out and put it here. You can use the command
to generate a 1024-bit Diffie-Hellman parameter file.openssl dhparam -out dhparam 1024
- Add a user and group for SSL to drop privileges too; name them both ssl. How to do this depends on your OS and what tools it provides; you can find some examples in the qmail source directory in a file called INSTALL.ids. For example, on Linux you would run:
If you can't figure out any OS-specific commands, edit the /etc/group and /etc/passwd files directly, using the file format described in the system manpages for group(5) and passwd(5).groupadd ssl
useradd -g ssl -d /var/qmail ssl - Create a file to set shell variables in /var/qmail/ssl/env. Put these lines in that file:
# Set these three
SSL_USER=ssl
SSL_GROUP=ssl
SSL_DIR=/var/qmail/ssl
# Enable UCSPI-TLS
UCSPITLS=1
# The rest are set based on the above three
SSL_CHROOT="$SSL_DIR"
CERTFILE="$SSL_DIR/cert"
KEYFILE="$SSL_DIR/key"
DHFILE="$SSL_DIR/dhparam"
SSL_UID=`id -u "$SSL_USER"`
if [ $? -ne 0 ]; then echo "No such user '$SSL_USER'" >&2; exit; fi
SSL_GID=`id -g "$SSL_GROUP"`
if [ $? -ne 0 ]; then echo "No such group '$SSL_GROUP'" >&2; exit; fi
# Export the variables used by other scripts
export SSL_CHROOT SSL_UID SSL_GID UCSPITLS CERTFILE KEYFILE DHFILE - Now set your umask back to something more usable, like 022:
umask 022
Set up qmail-smtpd
- Edit the qmail-smtpd run file, in /var/qmail/supervise/qmail-smtpd/run. There are three changes required:
- The top of the file has several variable settings. Below these lines, include the SSL environment variable script we created above, using the shell's "dot" command, typed as a single period:
. /var/qmail/ssl/env
- On the line that contains softlimit, add 10MB (10,000,000) the number after the -m flag. This allows qmail-smtpd to use the extra memory required for SSL. For example, if it's currently 2000000, you would have for that line:
exec /usr/local/bin/softlimit -m 12000000 \
- On the line that contains tcpserver, change tcpserver to sslserver -e -n, leaving all of the other flags in place. The line will now look something like:
/usr/local/bin/sslserver -e -n -v -R -l "$LOCAL" -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \
- The top of the file has several variable settings. Below these lines, include the SSL environment variable script we created above, using the shell's "dot" command, typed as a single period:
- Restart qmail to use the new configuration:
/var/qmail/bin/qmailctl restart
- Now we'll try a few tests to make sure everything works. If any of these steps fails, double-check the steps above, look for errors in the error logs, and use your common sense to see what's going wrong. Look in the logs to see if there are any errors:
You should see something like this:tail /var/log/qmail/smtpd/current
@40000000431fb29d10e09c2c sslserver: cafile 32655
@40000000431fb29d10e120fc sslserver: ccafile 32655
@40000000431fb29d10e14bf4 sslserver: cadir 32655 /usr/local/ssl/certs
@40000000431fb29d10e176ec sslserver: cert 32655 /var/qmail/ssl/cert
@40000000431fb29d10e1a9b4 sslserver: key 32655 /var/qmail/ssl/key
@40000000431fb29d10e1d4ac sslserver: param 32655 /var/qmail/ssl/dh 512
@40000000431fb29d10e226b4 sslserver: status: 0/20 - Make sure the server is now offering TLS. To test this, you can connect to the server on port 25 and issue an "extended HELO" command (EHLO) and see what extensions the server offers. To do this, start at a shell and type in the lines marked below with *:
* telnet localhost 25
The line that says STARTTLS after you type EHLO means that the server has TLS configured.
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 host.example.com ESMTP
* EHLO there
250-host.example.com
250-PIPELINING
250-8BITMIME
250 STARTTLS
* QUIT
221 host.example.com- Now test with a client to make sure things basically work. First start watching the logfile for errors:
tail --follow=name /var/qmail/supervise/qmail-smtpd/log/main/current
Unfortunately the client included with sslserver doesn't support TLS. If you have a copy of stunnel version 3 lying around, you can do something like this:
Otherwise, just skip this test and try the test in the next step.stunnel -r localhost:25 -f -c -n smtp -D debug -P none
- Finally, try turning on TLS in the mail clients you'll be using and sending a test message. Make sure there are no errors from the client or in the logs, and that the messages arrives successfully.
If this step works, you've set everything up correctly. Congratulations!
Set up qmail-pop3d
If you want to run a POP3 server, make sure you've set up qmail-pop3d as described in Life with qmail, then continue to follow these steps. If you're not running a POP3 server, you can skip this entire section.- Edit the qmail-pop3d run file, in /var/qmail/supervise/qmail-pop3d/run. There are three changes required:
- Near the top of the file, between the #!/bin/sh line and the line that begins with exec, include the SSL environment variable script we created above, using the shell's "dot" command, typed as a single period:
. /var/qmail/ssl/env
- On the line that contains softlimit, add 10MB (10,000,000) the number after the -m flag. This allows qmail-smtpd to use the extra memory required for SSL. For example, if it's currently 2000000, you would have for that line:
exec /usr/local/bin/softlimit -m 12000000 \
- On the line that contains tcpserver, change tcpserver to sslserver -e -n, leaving all of the other flags in place. The line will now look something like:
/usr/local/bin/sslserver -e -n -v -R -H -l 0 0 110 /var/qmail/bin/qmail-popup \
- Restart qmail to use the new configuration:
/var/qmail/bin/qmailctl restart
- Now we'll try a few tests to make sure everything works. If any of these steps fails, double-check the steps above, look for errors in the error logs, and use your common sense to see what's going wrong. Look in the logs to see if there are any errors:
You should see something like this:tail /var/log/qmail/pop3d/current
@40000000431fb29d10e09c2c sslserver: cafile 32655
@40000000431fb29d10e120fc sslserver: ccafile 32655
@40000000431fb29d10e14bf4 sslserver: cadir 32655 /usr/local/ssl/certs
@40000000431fb29d10e176ec sslserver: cert 32655 /var/qmail/ssl/cert
@40000000431fb29d10e1a9b4 sslserver: key 32655 /var/qmail/ssl/key
@40000000431fb29d10e1d4ac sslserver: param 32655 /var/qmail/ssl/dh 512
@40000000431fb29d10e226b4 sslserver: status: 0/20- Make sure the server is now offering TLS. To test this, you can connect to the server on port 110, then ask for a list of its capabilities. To do this, start at a shell, and type in the lines marked below with *:
* telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
+OK
* CAPA
+OK capability list follows
STLS
.
* QUIT
+OK
Connection closed by foreign host.The STLS line after you type CAPA indicates that the server supports TLS.
- Now test with a client to make sure things basically work. First start watching the logfile for errors:
tail --follow=name /var/qmail/supervise/qmail-pop3d/log/main/current
Unfortunately the client included with sslserver doesn't support TLS. If you have a copy of stunnel version 3 lying around, you can do something like this:
Otherwise, just skip this test and try the test in the next step.stunnel -r localhost:110 -f -c -n pop3 -D debug -P none
- Finally, try turning on TLS in the mail clients you'll be using and receiving some test messages. Make sure there are no errors from the client or in the logs, and that the messages arrives successfully.
If this step works, you've set everything up correctly. Congratulations!
- Now test with a client to make sure things basically work. First start watching the logfile for errors:
Optional Step: If desired, set up SSL servers
If you'd like to support SSL (not delayed encryption through TLS), you can still use the modified sslserver, and get the security advantages of chroot and privilege separation.These steps will help you create an SSL service from a TLS service. The instructions are for qmail-smtpd; they will work for qmail-pop3d if you simply replace smtpd with pop3d everywhere.
- Change directory to /var/qmail/supervise:
cd /var/qmail/supervise
- Create a service directory and log directory:
mkdir -p qmail-smtpd-ssl/log
- Copy the run file from the original service, and make it executable:
cp qmail-smtpd/run qmail-smtpd-ssl/
chmod 755 qmail-smtpd-ssl/run - Edit the run file (qmail-smtpd-ssl/run) in the following ways:
- On the line after . /var/qmail/ssl/env, add unset UCSPITLS
- On the line that contains sslserver line, remove the -n flag.
- On the next line, which will contain something like 0 smtp, change smtp to smtps; that tells sslserver to listen on the appropriate port for the SSL version of this service.
- Set up a logging directory for this new service:
mkdir /var/log/qmail/smtpd-ssl
chown qmaill /var/log/qmail/smtpd-ssl - Set up the logging program for this new service, by creating a file in qmail-smtpd-ssl/log/run with these contents:
Make sure the script is executable:#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t \
/var/log/qmail/smtpd-sslchmod 755 qmail-smtpd-ssl/log/run
- Link the new service into the /service directory, to have it start automatically on boot:
ln -s /var/qmail/supervise/qmail-smtpd-ssl /service
- Add the following to qmailctl's "start" section:
if svok /service/qmail-smtpd-ssl ; then
svc -u /service/qmail-smtpd-ssl /service/qmail-smtpd-ssl/log
else
echo qmail-smtpd-ssl supervise not running
fi - Add the following to qmailctl's "stop" section:
echo " qmail-smtpd-ssl"
svc -d /service/qmail-smtpd-ssl /service/qmail-smtpd-ssl/log - Add the following to qmailctl's "stat" section:
svstat /service/qmail-smtpd-ssl
svstat /service/qmail-smtpd-ssl/log - Add the following to qmailctl's "pause" section:
echo "Pausing qmail-smtpd-ssl"
svc -p /service/qmail-smtpd-ssl - Add the following to qmailctl's "cont" section:
echo "Continuing qmail-smtpd-ssl"
svc -c /service/qmail-smtpd-ssl - Add the following to qmailctl's "restart" section:
echo "* Restarting qmail-smtpd-ssl."
svc -t /service/qmail-smtpd-ssl /service/qmail-smtpd-ssl/log - To test, use an SSL-enabled mail client, or use stunnel version 3:
stunnel -r localhost:smtps -f -c -D debug -P none
Optional Step: If desired, set up other servers
If you have any other TLS or SSL servers you'd like to set up, such as an IMAP server, you should be able to use the above steps as a template to get you started.Required Step: Sleep better at night
Knowing your TLS- and SSL-protected mail server is secure.Thanks
Rbldns How to
What is this document?
I wrote this HOWTO so that others can have one place to stop and see how to setup their own private RBL list. The information is out there, but scattered all over the place.
The color convention I use is to have the headlines in gray and the code / shell scripts in yellow
What is rbldns?
Rbldns is a suite of programs in conjunction with dnscache to provide RBL service. In a nutshell IPs put into the rbldns data file show up as 127.0.0.1 responses to a specially crafted DNS query.
The IP address go into the file in the normal readable way, ie 192.168.0.1, and you query for them by reversing the address, like 1.0.168.192.example.com
How do I use rbldns?
Normally you don't use it directly, you have a program query an RBL server and based on the result of the query it either runs or doesn't run the program following it.
For example with Qmail you'll change your qmail-smtpd/run file from
tcpserver 0 smtp /var/qmail/bin/qmail-smtpd
to
tcpserver 0 smtp /usr/local/bin/rblsmtpd -a whitelist.example.com -r blacklist.example.com -r relays.ordb.org /var/qmail/bin/qmail-smtpd
According to the rblsmtpd man page, the -r and -a switches mean:
| Switch | If successful, do this |
|---|---|
| -r | Quit |
| -a | Don't do other lookups, and continue on |
In this case the steps are:
1. Do a lookup against whitelist.example.com, if sucessful then skip the other lookups and go on to accepting the smtp request
2. Do a lookup against blacklist.example.com, if that's successful then quit out printing a "RBL denied" message to the client.
3. Same as #2 but against relays.ordb.org
How do I make my own RBL?
You should know how to setup Dnscache before trying to do this. The steps are pretty easy, but its helpful if you know your way around
Setting up the rbldns is pretty easy, just follow the directions on the RBLdns-conf page:
rbldns-conf rbldns dnslog /etc/rbldns/black 127.0.0.2 rbl.example.com
rbldns-conf rbldns dnslog /etc/rbldns/white 127.0.0.3 whitelist.example.com
This says to setup a RBL on 127.0.0.2 that answers queries for *.rbl.example.com, and and RBL on 127.0.0.3 that answers for *.whitelist.example.com
Note: the 'rbl' and 'whitelist' names are arbitrary. RBLdns doesn't know if this is a 'good' or a 'bad' list of addresses -- it just knows that if someone asks for 1.0.168.192.bad.example.com on IP 127.0.0.2 it should answer with a 127.0.0.1 if it has it, or nothing if it doesn't.
Start these services up like all of DJB's other daemons:
ln -s /etc/rbldns/black /service
ln -s /etc/rbldns/white /service
To see if they are running, do a
lsof -n -i:53
And you should see listeners on 127.0.0.2 and 127.0.0.3
Why would I want to use a local RBL?
Why shouldn't you just relay on publically available RBLs? For a couple of reasons:
1. You know a couple of IPs are good and you don't want to waste bandwith by checking them against an RBL on the internet
2. Same thing for bad ones
3. You know your friend has a DSL line at home and an RBL blocks all home DSL users, you want to continue using the service but want to accept his mail.
4. You want to keep a 2nd source of information about who is sending mail to your server
So part of this is being a good netizen. If you know you're getting a lot of email from a handful of IPs, you should put those in your own whitelist so that you're not continously asking the public RBLs if they are okay
How to resolve *.(whitelist|rbl).example.com -- DNScache way
Now that we have the RBL service ready to go we have to be able to query it. Like we told dnscache to send all queries for *.example.com to the local tinydns IP address, we're going to do that for the RBL service.
This is the part that took me a while to think about: how do I serve both *.example.com and *.rbl.example.com? Well the same way you've already done it: through a dnscache setting.
Add two entries to your dnscache "forwarding" information:
cd /service/dnscache/root/servers
echo 127.0.0.2 > rbl.example.com
echo 127.0.0.3 > whitelist.example.com
svc -h /service/dnscache
This tells dnscache that when it gets a query for *.whitelist.example.com to send it off to the server on 127.0.0.3, *.rbl.example.com queries should go to 127.0.0.2. If you're setup TinyDns this should be familiar
How to resolve *.(whitelist|rbl).example.com -- TinyDns way
This might be a little cleaner if you're already serving up *.example.com results. You put in your tinydns file pointers to the (rbl|whitelist) lists:
cd /service/tinydns/root/
echo "&rbl.example.com:127.0.0.2:a" >> data
echo "&whitelist.example.com:127.0.0.3:a" >> data
make
svc -h /service/tinydns
So now when your tinydns server is queried for 1.0.168.192.rbl.example.com it will look at its file and say "That's handled by 127.0.0.2 and I'll ask it"
How do I add entries to my RBL?
Scenerio: a spammer is continously sending us email from 192.168.5.100 while our friend is sending from 192.168.6.56. We would like to avoid using public RBL servers for these two addresses.
Step 1: add the spammer to your spam list:
cd /service/rbl/black/root
echo 192.168.5.100 >> data
make
Step 2: Add your friend to your good list:
cd /service/rbl/white/root
echo 192.168.6.56 >> data
make
Now query to see if they are in there:
dnsip 100.5.168.192.rbl.example.com
dnsip 100.5.168.192.whitelist.example.com
dnsip 56.6.168.192.rbl.example.com
dnsip 56.6.168.192.whitelist.example.com
Notice I queried both sets of databases to make sure we didn't screw up who is a spamer versus who is our friend.
If you don't get a 127.0.0.1 result from a query when you think you should, HUP the appropriate daemon:
svc -h /service/rbl/black
svc -h /service/rbl/white
How do I now use an RBL service?
Now that you have the local RBLdns server(s) working through simple queries, modify your other scripts to use them. With qmail you do that like this:
exec /usr/local/bin/tcpserver -R -v -p \
-x /var/qmail/control/tcp.smtp.cdb \
-u $QMAILUID -g $QMAILGID \
mail.example.com smtp \
/usr/local/bin/rblsmtpd \
-a whitelist.example.com \
-r rbl.example.com \
-r relays.ordb.org \
-r bl.spamcop.net \
/var/qmail/bin/qmail-smtpd 2>&1
This is what your implementation will probably look like: an initial test against your goodlist, if that passes the continue on to the qmail-smtpd part. If that fails, then test against your local bad list. If that is sucessful then quit out. If not, continue on to your 2nd and 3rd RBLs.
How do I test my RBL?
You test an RBL by sending it a query. If it is in the database then it should return with a 127.0.0.x (1 or 2) answer. If it isn't in there, then nothing comes back.
The trick is to remember to reverse your IP address, so that 192.168.0.1 becomes 1.0.168.192.myrbldomain.com
Here's a simple program to check names fed on stdin:
#!/usr/bin/perl
use Net::DNS;
use strict;
my ($res, @rbls);
@rbls = qw (rbl.example.com whitelist.example.com);
$res = Net::DNS::Resolver->new;
while (<>) {
my ($rev, $ip);
chomp;
$ip = $_;
$rev = (join ".", reverse (split /\./, $ip) ); # ie 4.3.2.1.
foreach my $rbl (@rbls) {
my $query = $res->search("$rev.$rbl");
next unless ($query);
my $hit;
foreach my $rr ($query->answer) {
next unless $rr->type eq "A";
$hit = $rr->address;
last;
}
next unless $hit;
printf "%-20s %-20s $hit\n", $ip, $rbl;
}
}
How do I contact you?
If you have any questions or comments, or just want to say thanks, drop me a line at anand.shah@coolaquarius.comWednesday, September 19, 2007
Easy steps to preventing spam
The following 10 steps offer a very common sense (though often overlooked) approach to avoiding spam. If you follow these guidelines, you’ll notice a drastic decrease in the number of annoying messages that make it into your inbox…
01. Give your primary email address to friends and family only. Give a different address to others on the Internet. Although this second address will be more likely to receive unwanted emails, it is more disposable and can allow you to better control the emails you receive. Your ISP almost certainly offers multiple email addresses, and Yahoo, Microsoft and Google all offer free web based email accounts. These are great to use as disposable addresses.
It is NOT uncommon for a brand new email address to start receiving spam almost immediately after it has been created. This happens because even though the address is new, spammers will take a previously used user name (eg. jdoe@whatever.com) and apply it to a different domain (eg. jdoe@another_domain.com).
In addition, common email addresses may have been used previously and may still be on old mailing lists.
Block IP how to in Windows IIS and Apache
The following tutorial explainss how you can block IP addresses from viewing your website in Windows IIS and Linux/Unix Apache. Blocking IP addresses prevents users from seeing your website during construction, limits access to specific users, or to blocks users attempting to connect to your site maliciously.
Block IP how to in Windows IIS
To block an IP address from viewing your website, please follow these steps:
1. Login to your server through Terminal Services or Remote Desktop Connection. 2. Click Start, select Programs, and then click Administrative Tools. For IIS 5.0 click Internet Services Manager. For IIS 6.0 click Internet Information Services.3. In the left column you will see the Server Name. In IIS 5.0, expand the Server Name to find the domain name. In IIS 6.0, expand the Server Name and then Web Sites to find the domain name.4. Right-click on the domain name and select Properties.5. On the Directory Security Tab under IP Address and Domain Name Restrictions click Edit.You have two options, you can grant access to all computers and restrict individual IP's or you can deny access to all computers and grant access to specific IP's. By default, all users will be granted access to your site except the IP addresses you specify.To add an IP address to the restriction list, please follow these steps: 1. Click Add.2. Select the type: Single Computer - add a single IP Address. Group of Computers - add a block IP Address using the starting IP address of the block and the appropriate subnet mask. Domain Name - add a domain name (this method is not recommended). 3. Click Ok.
Block IP how to in Linux/Unix Apache
You can also block (or admit) users based on their incoming IP address:
You create my.block.db from a file (named blocklist) that looks like this: xxx.xxx.xxx.xxx block
and piping it to./db_create my.block.db < blocklist
You can add additional entries on the fly: echo "xxx.xxx.xxx.xxy block" ./db_create
Anand Shah
Blocking Spam with Sendmail
Spam is unlike regular junk mail from the post office for several reasons:
Normal junk mail costs money to send. Spam does not. The cost is born by the recipients and by the servers in between. While normal junk mail is a revenue source for the post office and actually helps to pay for delivering regular mail, spam pays for nothing. Spam can hence be viewed as theft of services.
Spam has detrimental effects. Historically Internet mail servers have passed along email between third parties as a gesture of friendship and good will. Much like tossing your neighbor's paper over the fence if the paperboy misses his throw. This is happening less and less because spammers take advantage of it.
Spam reduces the utility of email. People become discouraged about checking their mailboxes if they are always cluttered with spam.
Enough ranting. I decided to do something. Being the sort who favors technical fixes over legal ones, I started doing some research on the web, ordered a copy of the Bat book, and spent some time reading my sendmail configuration and scratching my head. I present here the result.
First, if you're not using sendmail I can't help you. Second, you need the latest version of sendmail or these tricks won't work. And finally, we had several conditions that had to be met:
We wanted to be able to block spam by domain name, network number, or by specific address for maximum flexibility.
We needed to be able to allow spam to selected mailboxes for customers who do not want spam blocked. I may disagree with them, but they are paying for a service and it is, after all, their mail. Spam blocking had to be a value added service that could be turned off.
We host a number of "virtual domains" and needed to be able to route email for them to the proper mailboxes. We had already been doing that, but it was a factor that had to be considered in our antispam measures so that spam could be blocked or not as desired by the mailbox owners.
We wanted to stop "third party relay" going through our mail server while allowing for exceptions for customers with their own domains and mail servers or for roaming customers.
I think I have come up with a set of sendmail rules which accomplish this.
First, we need to add a few entries in the local configuration section:
LOCAL_CONFIG
Fw/etc/vdomain.cw
Kvdomain hash /etc/vdomain
# list of people who like spam
F{fools} /etc/WantSpam
# list of known spammers
Kjunk hash -a@JUNK /etc/spammers
# List of network addresses we will relay for
F{LocalIP} /etc/LocalIP
# List of domains we will relay to
F{RelayTo} /etc/RelayTo
Click on the filename of any of these files for an explanation of its purpose and contents.
Now we get to the rules themselves. First, an entry must be added to your local rule zero, like so:
LOCAL_RULE_0
R$* $: $>vmap $1
Not very interesting, is it? It just calls another rule set, named "vmap", which handles virtual domain address mapping. Note: I don't know what the "right way" is to do these things, but it works to just list all the rest of these rulesets right under LOCAL_RULE_0, so that's what I do. Here then is the "vmap" ruleset:
Svmap
R$+ < @ $+ . > $: $1 < @ $2 > .
R$+ < @ $+ > $* $: $(vdomain $1@$2 $: $1 < @ $2 > $3 $)
R$+ < @ $+ > $* $: $(vdomain $2 $: $1 < @ $2 > $3 $)
R$+ < @ $+ > . $: $1 < @ $2 . >
I made this a separate ruleset since I do it again in the rest of the rules, as you will see. I have obscure reasons for not just calling local rule zero as needed.
Next I define a "junk" ruleset to look up a domain name or email address in the /etc/spammers.db database:
Sjunk
R$* $: $(junk $1$) look for host in spammer list
R$+@JUNK $@ $1@JUNK return message if found
R@JUNK $@ Spam refused @JUNK generic message
R$-.$+ $: $1 . $>junk $2 retry skipping lead subdomain
R$-.$+@JUNK $@ $2@JUNK return message if found
Next, a "junkIP" ruleset to look up an IP address or network number in the /etc/spammers.db database:
SjunkIP
R$* $: $(junk $1$) look for host in spammer list
R$+@JUNK $@ $1@JUNK return message if found
R@JUNK $@ Spam refused @JUNK generic message
R$+.$- $: $2 . $>junkIP $1 retry without trailing number
R$-.$+@JUNK $@ $2@JUNK return message if found
R$-.$+ $@ $2.$1 fix order if not spammer
Now for the heart of it, the "check_rcpt" ruleset. Spam blocking is more often done in the "check_mail" ruleset, but we can't do it that way since we need to check the recipient to see if they want spam. Hence, this ruleset gets a bit long.
Scheck_rcpt
R$* $: $>vmap $>3 $1 normalize address
# Refuse to relay mail between nonlocal systems
R$* $: $(dequote "" $&{client_addr} $) $ $1
R0 $ $* $@ ok no client addr: directly invoked
R$={LocalIP}$* $ $* $@ ok from here
R$* $ $* $: $2 not from local, check recipient
R$*<@$=w.>$* $>3 $1 $3 remove our aliases, maybe repeatedly
R$*<@$*$={RelayTo}.>$* $>3 $1 $4 remove domains we relay to
# still something left?
R$*<@$+>$* $#error $@ 5.5.4 $: "554 we do not relay from " $&{client_name} " to " $1@$2$3
# Allow mail to fools who like spam, and otherwise block spammers
R$={fools} $@ ok recipient listed as wanting spam
# Block by host or domain name
R$* $: $(dequote "" $&{client_name} $)
R$* $: $>junk $1
R$*@JUNK $#error $@ 5.5.4 $: "554 " $1 ": " $&{client_name}
# Block by network or host IP address
R$* $: $(dequote "" $&{client_addr} $)
R$* $: $>junkIP $1
R$*@JUNK $#error $@ 5.5.4 $: "554 " $1 ": " $&{client_addr}
# Block by specific email address
R$* $: $(dequote "" $&f $)
R$* $: $>junk $1
R$*@JUNK $#error $@ 5.5.4 $: "554 " $1 ": " $&f
R$* @ $* $: $1 @ $>junk $2
R$* @ $*@JUNK $#error $@ 5.5.4 $: "554 " $2 ": " $&f
# Block mail from invalid addresses
R$* $: $>3 $1 make domain canonical
R$* < @ $+ .> $* $@ ok name resolved ok
# Killer case -- single token domain
R$* < @ $- > $* $#error $@ 5.5.1 $: "551 Invalid host name: " $2
# Delay case -- domain doesn't resolve
R$* < @ $+ > $* $#error $@ 4.5.1 $: "451 Unknown domain: " $2
And that's it. If you'd like, you can download a text version of this for easier editing.
Oh, one last thing. The rejection messages all get logged in /var/log/maillog (at least on our system). Here's a PERL script for maillog.scan that gives us a nightly report of spam blocks:
#!/usr/bin/perl
while($lt;$gt;) {
if(/rejection:.*\.\.\. ? ?(.*)/) {
$spam{$1} += 1;
}
}
print "\nSpam blocks:\n\n";
foreach $msg (sort keys %spam) {
printf "%5d %s\n", $spam{$msg}, $msg;
}
print "\n";
Anand Shah
Understanding QMAIL
Objective
SPAMCONTROL is an extension for qmail. It provides the following features:
Enhancements for qmail-smtpd:
ESMTP enhancements
Strict RFC 2821 conformance.
Reference 'Mail From:' parameter parser, supporting SIZE (RFC 1870) and AUTH options.
Customizable SMTP Authentication (RFC 2554) support for LOGIN, PLAIN, and CRAM-MD5.
Optional STARTTLS (RFC 2487) support in conjunction with sslserver. +)
Mail From Verification (MAV) allows relaying only of verified/authorized addresse. +)
SMTP envelope Anti-Spam-Tools
Wildmat Filters for the HELO/EHLO greeting and the 'Mail From:
DNS Lookup for the HELO/EHLO greeting (A/MX) and the domain part of the 'Mail From:' (MX).
Customizable HELO/EHLO greeting checks.
Tarpitting and Smart Rejection in case of too many invalid Recipients.
Enhanced badmailfrom support
Wildmat filter.
Additional 'badmailfromunknown' capabilities. #)
Anti-spoofing of own addresses. #)
Recipients extensions
badrcptto wildmat filter.
Restricting the number of allowed 'Rcpt To:' per SMTP session.
Whitelisting: Controlling the reception of mails not only on a rcpthosts base but rather on the complete
Customizable 550 or 450 return messages. #)
Virus prevention
Reference badmimetypes implementation.
Additional badloadertypes filter.
Qmail High Performance Scanner Interface (QHPSI).
Customizable SMTP 554 Reply Message.
Logging
Extensible logging format.
Logging for failed and accepted SMTP sessions.
Enhancements for qmail-remote:
SMTP Authentication
Supported are Auth types LOGIN and PLAIN.
Additional authsenders control file.
Fast delivery
Delivery to any DNS listed MX for that domain instead just the primary.
Increased read buffer for delivery.
Enhancements for qmail-pop3d:
STARTTLS support
Enhancements for qmail-queue:
High speed virus scanner by means of QHPSI
Additional QMAILQUEUE usage
Optional BIGTODO support
Enhancements for qmail-send:
Bounce control
Restricting the size of bounces.
Doublebouncetrim.
Additional recognition of local IP addresses
moreipme
notipme
With SPAMCONTROL, qmail-smtpd can stand the two most common threats:
Lexical and/or dictionary Spam attacks in particular to none-existing
Virus Bombing and resource exhaustion due to Virus Scanners.
Additionally, qmail-smtpd allows
positive verification and indication of authorized use of the
--------------------------------------------------------------------------------
0. Conventions
0.1 Definitions
Sender = SMTP envelope sender (Mail From:
Recipient = SMTP envelope recipient (Rcpt To:
Nullsender Mail = E-Mail with empty Sender (Mail From: <>)
Full qualified SMTP E-Mail address: "user@domain.com" (for Sender/Recipient)
0.2 qmail-smtpd run Script
Throughout this document, I assume that qmail-smtpd is under control of supervise (out of the Daemontools package) and served by tcpserver (part of the UCSPI package) or a patched version of sslserver (part of the UCSPI-SSL package).
A typical - minimal - so called run script looks like follows:
#!/bin/sh
# qmail-smtpd startup
QMAILDUID=`id -u qmaild`
QMAILDGID=`id -g qmaild`
HOSTNAME=`hostname`
exec tcpserver -vR \
-u $QMAILDUID -g $QMAILDGID \
-l $HOSTNAME 0 smtp \
/var/qmail/bin/qmail-smtpd 2>&1
0.3 Environment Variables
Qmail - and SPAMCONTROL - relies on the concept of environment variables which are available for a task (sharing the same environment). qmail-smtpd may be fed by environment variables in three different fashions:
(Gobal) Exported variables in the run script; eg. export RELAYCLIENT="".
(Gobal) By means of the envdir facility as part of the Daemontools package.
(Gobal) Using a "profile" configuration file called in the run script by means of the "dot" syntax (. pofile).
(Individual) Setting the appropriate variable in the tcpserver cdb database.
While the first three cases define static and "global" environments variables, the last case makes the environment variables client-dependent and - by means of tcprules - dynamically changeable. Any mixture is possible, though only the "last" setting of an environment variable is effective!
0.4 tcpservers' cdb
As a convention, I will call the tcperver's cdb, which rules the behavior of qmail-smtpd, tcp.smtp. A typical tcp.smtp would look like
=mydomain.com:allow,RELAYCLIENT="",LOCALMFCHECK=""
1.2.3.4:allow,RELAYCLIENT="",LOCALMFCHECK="otherdomain.com"
5.6.7.8:allow,REQUIREAUTH=""
:allow,MFDNSCHECK="",BADMIMETYPE="",BADLOADERTYPE="M"
The cdb is constructed on the fly:
tcprules tcp.smtp.cdb tcp.smtp.tmp < tcp.smtp
Caution: For use with tcpserver, the value of the environment variable has to be included in quotes.
0.5 rcpthosts/morercpthosts.cdb
Though qmail can live happily without the knowledge of domains to be responsible for as provided by rcpthosts/morercpthosts.cdb, it is highly adviceable to include all domains to receive emails for (as per DNS MX Records) into those control files. Otherwise, qmail-smtpd may act as an Open Relay. Further, some LOCALMFCHECKs will fail, as discussed below.
--------------------------------------------------------------------------------
1. qmail-smtpd Protocol Extensions
1.1 Size Extension
qmail-smtpd's "Mail From:" parameter parser is used to detect and evaluate the SIZE parameter and to eventually reject messages which initially exceed the databytes limit.
Nevertheless, qmail-smtpd checks the size of the incoming message anyway.
For incoming E-Mails which exceed the message size values (in Bytes) defined in
control/databytes
or via the $DATABYTES environment variable.
1.2 SMTP Authentication
SMTP Authentication requires a Client to authenticate and a Server to honor the authentication procedure. In this version of SPAMCONTROL, Qmail acts as an Authentication Server for qmail-smtpd and as an Authentication Client for qmail-remote.
Usually, a MTA (such as Qmail) will accept transmissions of E-Mails anyway as long as the "Rcpt To:
I have taken the SMTP-Auth Patch from Krzysztof Dabrowski and included this into SPAMCONTROL. However, SPAMCONTROL's implementation is compliant with the checkpassword API designed by Dan Bernstein (the Plugable Authentication Module PAM).
SPAMCONTROL provides the following features:
(E-)SMTP-Auth protocol extension for qmail-smtpd with the keywords AUTH PLAIN LOGIN CRAM-MD5.
"Mail From:" parameter parser/generator for the parameter AUTH (ie. AUTH=userid) as 'xtext'.
BASE64 en/decoding of the User-Id/Password/Digest.
checkpassword compatible call-API for an arbitrary PAM program.
For authenticated users, their Userid is treated as TCPREMOTEINFO and displayed in qmail-smtpd 'Received:' header.
Logging of successful and failed authentication attempts.
1.2.1 Pluggable Authentication Module PAM
While SASL is a generic concept, the information flow for authentication between e.g. qmail-smtpd and the PAM is defined by Dan Bernstein's checkpassword API. SPAMCONTROL provides the PAM on file descriptor 3 as an informational string composed of:
The BASE64-decoded Userid followed by a 0,
The BASE64-decoded Password (Login and Plain) or Digest (CRAM-MD5) followed by a 0, and
The plain Challenge followed by a trailing 0, thus the PAM can reconstruct and validate the Digest in the CRAM-MD5 case.
You are free to choose or even write your own PAM program, but in any case, the SASL Procedure of the client and the server has to match and the procedure has to be advertised. Compliant PAMs:
Dan Bernstein's checkpassword
Larry M. Smith's checkpassword.pl,
My cmd5checkpw version 0.30.
Vpopmail's (5.4) vchkpw
....
1.2.3 qmail-smtpd Setup for SMTP Authentication
qmail-smtpd including SMTP Authentication may be called by tcpserver/sslserver in a supervise run script. Here is an example (with some more features):
#!/bin/sh
# qmail-smtpd startup with SMTP Authentication
QMAILDUID=`id -u qmaild`
QMAILDGID=`id -g qmaild`
HOSTNAME=`hostname`
export SMTPAUTH=""
exec softlimit -m 2000000 \
tcpserver -vR -l $HOSTNAME \
-u $QMAILDUID -g $QMAILDGID 0 smtp \
/var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1
Beware! Unlike the original implementation, I omitted the inclusion of the Hostname as argument for qmail-smtpd.
Unlike the standard qmail-smtpd, now you have
define the environment variable SMTPAUTH to allow SMTP authentication and
to provide in addition a PAM program, here cmd5checkpw, which itself calls a shell named 'true' (/bin/true or /usr/bin/true) exiting simply with "0".
The environment variable SMTPAUTH may be left blank to allow Authentication types "PLAIN" and "LOGIN" or may be currently set to "cram..." (lower case) to enable CRAM-MD5 authentication in addition.
1.2.4 User Database for cmd5checkpw
For SMTP Authentication, a User Database has to be generated and maintained. The SMTP Authentication User may exist independently of any System Users, Qmail Users, or E-Mail Accounts. In case of the modified cmd5checkpw I decided to keep the User in the Qmail directory as
users/authuser
There exist other flavors, in particular the saslpasswd scheme or the Cyrus SASL Library you may want to use. Further, for users with POP3/IMAP4 Accounts on the system it is advisable to use a common User Database. For Vpopmail you may use vchkpw.
However, since you are free to use any other checkpassword compliant PAM, it's up to you whatever you apply. Please remember: In order to access the Unix /etc/passwd the respective program has to run as root.
1.2.5 Requiring SMTP Authentication
You can use the environment variable REQUIREAUTH to enforce authentication for particular clients. A typical run script to require SMTP Authentication for particular SMTP clients looks like:
12.34.56:allow,RELAYCLIENT="",SMTPAUTH="CRAM-MD5",REQUIREAUTH=""
:allow
1.2.6 SMTP Authentication and Vpopmail
SMTP Authentication works well with vpopmail, however, you have to use a checkpassword compatible PAM. Older versions of vchkpw have to be patched accordingly (see http://www.fehcom.de/qmail/smtpauth.html).
vchpkw offers a lot of authentication capabilities; it supports login, plain, and CRAM-MD5 and may authenticate the user against a mysql database and others. In start-up script for qmail-smtpd you have to make sure to access the user database with the correct user access rights:
#!/bin/sh
# qmail-smtpd startup with SMTP Authentication + vpopmail
QMAILDUID=`id -u vpopmail`
QMAILDGID=`id -g vpopmail`
HOSTNAME=`hostname`
export SMTPAUTH="crammd5"
exec softlimit -m 2000000 \
tcpserver -vR -l $HOSTNAME \
-u $QMAILDUID -g $QMAILDGID 0 smtp \
/var/qmail/bin/qmail-smtpd /home/vpopmail/bin/vchkpw true 2>&1
If you use Sqwebmail in addition, the user is free to set his/her own password.
1.3 STARTTLS support
SPAMCONROL's STARTTLS support for qmail-smtpd is aligned with Scott Gifford's approach and depends on the following:
Superscript's sslserver (0.70) instead of tcpserver
the enclosed patch against sslserver to allow STARTTLS in addition with standard TLS,
the availability of a valid X.509 cerificate, an appropiate key, and additionally a Diffi-Hellman parameter file,
the correct feeding (via environment variables) of those settings to sslserver to allow encryption.
My STARTTLS implemention conforms with RFC 3207.
1.3.1 STARTTLS implementation
Most current STARTTLS/TLS solutions depend on the existance and availability of the OpenSSL libraries -- so does SPAMCONROL. However, unlike other implementations, qmail-smtpd is insulated against OpenSSL by means of sslserver. In fact, all encryption and certificate verification is facilitated by sslserver. In this respect, Scott's and my STARTTLS implementation is very much OSI-like. The communication and presentation happens at a well defined environment, typically assigned to the user and group ssl. Any potential attacks or bugs are kept away from the application and don't harm.
The reading and response to client cerificates and the actual encryption happens in the assigned user spaces; which should never be root.
1.3.2 Prereqs
Install Superscript's ucspi-ssl (version 0.70). Apply the attached patch "ucspi-ssl-0.70-ucspitls-0.2.patch_" against the source, typically found at /package/host/superscript.com/net/ucspi-ssl-0.70/src and execute package/install base. This patch includes (delayed) STARTTLS support to sslserver and will allow to substitude tcpserver completely, even if no SSL/TLS communication is required.
Further, it is helpful to create a low privileged user and group ssl, which will be used by sslserver for SSL/TLS communication purposes. Please follow Scott Giffords' advices.
1.3.3 STARTTLS Settings
Substiute tcpserver with sslserver in the run script for qmail-smtpd. If you use softlimits, it might be necessary to raise those settings significantly due to the increased memory requirements. Here is my run script:
#!/bin/sh
QMAILDUID=`id -u qmaild`
QMAILDGID=`id -g qmaild`
HOSTNAME=`hostname`
export SMTPAUTH="crammd5"
export UCSPITLS=""
MAXCONCURRENCY=`cat /var/qmail/control/concurrencyincoming`
. /var/qmail/ssl/env
exec softlimit -m 180000000 \
sslserver -sevn -w 5 -R -l $HOSTNAME -c $MAXCONCURRENCY \
-x /var/qmail/etc/tcp.smtpd.cdb \
-u $QMAILDUID -g $QMAILDGID 0 smtp \
/var/qmail/bin/qmail-smtpd cmd5checkpw true 2>&1
It is absolutely necessary to use the "-n" flag for sslserver, since this will trigger the availability of encrypted communications channels between sslserver and qmail-smtpd.
The environment variables needed to feed sslserver are included in the "profile" /var/qmail/ssl/env. In my system, they have the following settings:
# Set
SSL_USER=ssl
SSL_GROUP=ssl
SSL_DIR="/var/qmail/ssl"
# Rest
SSL_CHROOT="$SSL_DIR"
CERTFILE="/var/qmail/ssl/cert"
KEYFILE="/var/qmail/ssl/key"
DHFILE="/var/qmail/ssl/dhparam"
SSL_UID=`id -u $SSL_USER`
if [ $? -ne 0 ] ; then echo "No such user '$SSL_USER'" >&2 ; exit; fi
SSL_GID=`id -g $SSL_USER`
if [ $? -ne 0 ] ; then echo "No such group '$SSL_GROUP'" >&2 ; exit; fi
export SSL_UID SSL_GID SSL_CHROOT
export CERTFILE KEYFILE DHFILE
Of course it is required, to have raised the directory /var/qmail/ssl before and generate via openssl the appropriate keys before.
Note: These settings are "global"; however, by means of sslserver and the settings in your tcp.smtpd file it is possible to use different certificates per connection.
Comment: Please read the documentation of UCSPI-SSL carefully w.r.t. the "mod-ssl" variables. It might in addition be necessary to define CAFILE, CADIR and other SSL options to your needs.
After you verified your stettings, restart qmail-smtpd. Whether qmail-smtpd will present "STARTTLS" in the EHLO dialogue, depends on the presence of the UCSPITLS environment variable. These can be set i.e. per IP in the tcp.smtpd control file.
In addition, you can enforce a TLS encrypted sessions defining REQUIRETLS="" for particular connections.
1.3.4 STARTTLS and SMTPAUTH
In case qmail-smtpd is instructed to use STARTTLS and SMTPAUTH, SMTP Authentication always takes place after the TLS session is active, but never reverse. Thus, all SMTP parameters like username and password are already encrypted. Of course, SMTP Authentication is still available for unencrypted SMTP connections, and STARTTLS does not require per se SMTP Authentication. However, STARTTLS and SMTP Authentication is a strong and powerful couple to secure the SMTP communication
1.3.5 STARTTLS Received Header Extensions
SPAMCONTROL displays the use of STARTTLS in the Received header (according to RFC 3207). The following information is added:
The TLS protocol in use.
The Cipher in use.
The presented encryption key length the and key length in place.
The client's "Subject" Destinguished Name (DN).
Here's a sample using Thunderbird als email client:
Received: (qmail 35450 invoked from network); 15 Mar 2006 20:22:09 -0000
Received: from dornroeschen.fehnet.de (HELO ?192.168.192.19?) (erwin@192.168.192.19)
encrypted via TLSv1: RC4-MD5 [128/128] DN=unknown
by orion.fehnet.de with ESMTPSA; 15 Mar 2006 20:22:09 -0000
1.4 MAV: "Mail From: Address Verification"
Mail From: Adress Verification (MAV) is a mean to enforce the use of the SMTP "Mail From:" address for particular Relayclients. Former versions of SPAMCONTROL used a "LOCALMF" check which allowed only a very limited granularity. However, with MAV you can control/enforce
the use of a particluar domain name (@example.com) per IP connection in the "Mail From:" - or -
the use of a particular "Mail From:" address (user@example.com) per Authenticated User.
MAV is in particluar very useful, if emails from your domains have to be undoubtly "officially" send.
1.4.1 Requiring MAV
Mail From: Address Verification is only be done if the flag 'relayclient' is set. This flag is set if
the environment variable RELAYCLIENT is provided through tcpserver/sslserver (i.e. by means of POP-before-SMTP) - or -
in case the user is verified by means of SMTP Authentication.
For these circumstances MAV can be enforced by means of the environment variable LOCALMFCHECK:
LOCALMFCHECK='"" means, the domain/host-part of the provided SMTP Originator address is checked against the contents of control/rcpthosts and control/morercpthost.cdb.
LOCALMFCHECK="example.com" requires that the domain/host-part of the Originator address matches the provided string (here: "example.com" - irrespectively of the case).
LOCALMFCHECK="!" checks the Originator address against the contents of the file control/mfrules.cdb.
1.4.2 Control files mfrules and mfrules.cdb
The file control/mfrules follows roughly the same syntax as the common file tcp.smtpd used for tcpserver/sslserver. It assigns either a complete SMTP address, a FQDN, an IP adress or a domain to a set of allowed Originator addresses. In practice control/mfrules allows
for relayclients qualified by SMTP authentication - to verify the used SMTP Originator address against the allowed/assigned address,
for relayclients identified by IP or domain - to verify the used Originator address against a list of assigned possible addresses,
for relayclients identified by IP or domain - to verify the used host/domain-part of the SMTP Originator address against the assigned domain-name.
Once you have populated control/mfrules run qmail-mfrules to derive control/mfrules.cdb from the input file. Additionally, define LOCALMFCHECK="!" either gobally or in the tcp.smtpd file.
1.4.2. Reply to the relayclients
Mail Clients may be setup wrongly or a user may want to use the relaying MTA to send emails for a different name. In case, MAV is in place and well configured, the particular user will not be allowed to send the mail over the gateway receiving the following SMTP reply:
553 sorry, invalid address specified (#5.7.1)
Since this might not be helpful for the (innocent) sender, you might use the environment variable REPLYMAV to add a qualification to that message.
1.4.3 MAV SMTP protocol extension
MAV puts the burden of SMTP Originator address verification ot the relaying MTA; that is the reverse scheme compared to SPF and others. Emails qualified through MAV are labeled with "ESMTPM" in the Received header, which is generated by qmail-smtpd.
--------------------------------------------------------------------------------
2. qmail-smtpd SMTP Envelope Filters
The SMTP Envelope consists of three parts:
The HELO/EHLO greeting from the SMTP Client.
The Originator: Mail From:
The Recipient: Rcpt To:
SPAMCONTROL allows to filter E-Mails according to the bad* criteria with a so-called wildmat search, which is a subset of the known Regualar Expressions (RegEx). The wildmat search works in order least significant to most significant and includes
a case-insensitive straight search - and -
a case-sensitive wildmat search
The following sets of wildmat control characters can be used:
Exclamation mark (!): The wildmat filter allows you to define exceptions for particular clients/addresses by simply putting an exclamation mark (!) as first character in the line.
Asterisk (*): General pattern matching character; one or more preceding.
Question Mark (?): Match zero or one preceding.
Backslash (\): Literal expression of following character, eg. \[.
Match one from a set ([...]): i.e. [Ff][Aa][Kk][Ee] matches FAKE, fake, FaKe, FAKe etc.
2.1 SMTP Envelope Addresses
Any E-Mail address, lets say
local part: user (left from the "@") and a
host part: host.domain.com (right hand side from the "@").
qmail-smtpd converts hostnames and domainnames taken from the tcp-env/tcpserver environment to lowercase (see man addresses).
qmail-smtpd reads the SMTP envelope address inside the recommended SMTP envelope brackets "<" and ">".
Originally, SMTP addresses without the brackets are accepted; and are terminated if a blank is seen " ",
however with SPAMCONROL (and RFC 2821) brackets are mandatory.
Double-quotes in SMTP addresses "forwarding path" are stripped.
E-Mail addresses for local accounts are considered case-insensitive and delivered irrespective of their case.
Lets say - if the local account is "admin" and the RCPT to: tells
2.2 Filtering HELO/EHLO Greetings (badhelo)
RFC 2821 says: "These commands (HELO/EHLO) are used to identify the SMTP client to the SMTP server. The argument field contains the fully-qualified domain name of the SMTP client if one is available. In situations in which the SMTP client system does not have a meaningful domain name (e.g., when its address is dynamically allocated and no reverse mapping record is available), the client SHOULD send an address literal (see section 4.1.3), optionally followed by information that will help to identify the client system. y The SMTP server identifies itself to the SMTP client in the connection greeting reply and in the response to this command."
Qmail records the HELO/EHLO greeting string for every received message in the E-Mail "Received:" header in case the provided HELO/EHLO string is different from the connecting hosts FQDN:
Received: from foo.bar.de (HELO foo) (192.168.192.11)
by heaven.bar.de with SMTP; 25 Apr 2003 15:01:42 -0000
The HELO/EHLO string is included as "(HELO foo)". The HELO/EHLO string is usually generated by the sending MTA without much control (MUAs often use their generic hostname).
SPAMCONTROL allows a flexible filtering of the clients HELO/EHLO greeting string, which depends on the setting of the environment variable HELOCHECK:
HELOCHECK="": evaluate contorl/badhelo control file
HELOCHECK="!": reject session, if no HELO/EHLO greeting is not provided/empty.
HELOCHECK=".": reject session, if no HELO/EHLO greeting is provided/empty and evaluate control/badhelo control file
HELOCHECK="=": require that the HELO/EHLO greeting corresponds to the FQDN (= TCPREMOTEHOST) of the host.
HELOCHECK="A": DNS A lookup for the HELO/EHLO greeting and evaluate control/badhelo.
HELOCHECK="M": DNS MX lookup for the HELO/EHLO greeting and evaluate control/badhelo.
The HELOCHECKs are only done, in case RELAYCLIENT is not set (split-horizon fashion). In my current setup, a useful setting is HELOCHECK="." and with the following input in control/badhelo
myIPAddress
myFQDN
localhost
localhost.localdomain
These settings exclude the spoofing of the MTA's own address, which is typical for spam senders, since they determine the EHLO/EHLO greeting from the initial IP/SMTP session parameter.
2.3 Filtering Mail From: SMTP Addresses (badmailfrom)
SPAMCONTROL allows four types of checks against the provided "Mail From:" SMTP envelope address (which I often call the "Originator"):
A DNS MX lookup whether the provided Domain (the part right from the "@", i.e. user@example.com) exists.
A wildmat check against the content of control/badhelo.
An anti-spoofing test, if the Originator addresses matches an entry in control/badhelo appended with a "+" (@mydomain.com+).
A badmailfromunknown test (in case TCPREMOTEINFO is "unknown", thus the IP address has no correspondance in the DNS) if the Originator address matches an entry in control/badmailfrom appended with a "-" (@yahoo.com-).
2.3.1 DNS MX Checks of the Envelope Sender
Invoking the environment variable MFDNSCHECK in the qmail-smtpd startup script enables globally the DNS check for the envelope's Sender.
Example:
#!/bin/sh
export MFDNSCHECK=""
.....
Additionally, the environment variable may be defined individually within a cdb of tcpserver/sslserver. Typically, this is done for "non-trusted" hosts within a tcpservers cdb:
127.0.01:allow,RELAYCLIENT=""
:allow,MFDNSCHECK=""
If environment variable MFDNSCHECK is not set, qmail-smtpd does not perform this DNS MX check.
Note: All DNS checks are either done by means of the libresolv library which comes with BIND, or my means of DJBDNS's routines, which can be included installing DJBDNS and using Nikola Vladov's enhancements for DJBDNS in addition with the modified Makefile.djbdns.
2.3.2 Standard badmailfrom Checks
control/badmailfrom was the only SMTP envelope filter Dan Bernstein originally implemented for qmail-smtpd. Here, only particular names or perhaps domains were listeted to be rejected in the SMTP dialogue. Since then, various flavours of badmailfrom have been brought out. However, the approach to reduce spam emails feeding control/badmailfrom with known spammer addresses is comparable trying to hit a moving target. Almost all Originator addresses spammer use today are fake and in this sense are meaningless.
2.3.3 badmailfromunknown Checks
There exist a special case, where you expect an email with a specific Originator address to be send via particular MTAs. For instance, if you see an email with Originator address "support@microsoft.com", it has to be send from a Microsoft MTA. qmail-smtpd has the knowledge of the sender's IP and FQDN (by means of the environment variables TCPREMOTEIP and TCPREMOTEHOST) in case you use tcp-env, tcpserver, or sslserver with the appropriate argument, i.e. tcpserver -h.
MTAs for which the FQDN can't be resolved are unqualified. In particular, emails from the large webmail providers (aol, hotmail, yahoo, gmx, t-online ...) have always to be send from qualified MTAs. Reversely, you can safely reject emails with those Originator hostparts, which can not be resolved tcpserver/sslserver records them as "unkonwn".
With SPAMCONTROL's badmailfrom implementation, you simply include the Originator addresses for which you enforce a qualified TCPREMOTEINFO into control/badmailfrom in the following way appending a dash ("-"):
@aol.com-
@hotmail.com-
@yahoo.com-
@gmx.de-
Note 1: Since tcp-env/tcpserver/sslserver relies on a qualified DNS lookup, it is certainly helpful to use DJBDNS' dnscache as frontend.
Note 2: Wildmat support is not provided; thus an entry "@*.yahoo.com-" won't work.
2.3.4 badmailfrom Anti-Spoofing
Another special case is given, rejecting none-Relayclient emails with Originator addreses spoofing your domain name or email addresses. Email can be rejected if the "responsible domains" are included with a trailing plus ("+") in the following way into control/badmailfrom:
god@mydomain1.com+
@mydomain2.net+
@mydomain3.org+
Note: Wildcard support is not provided in this case.
2.4 Controlling the Rcpt To: SMTP dialogue and Filtering the Recipient Address
Apart for the RECIPIENTS mechanism, which is detailed later, you can reject SMTP Recipient addresses (Rcpt To:
reject emails included in control/badrcptto,
reversely accept only those recipients whitelisted in control/badrcptto,
control the number of accepted Recipients by means of the environment variable MAXRECIPIENTS,
enforcing TARPITTING if too many Rcpt To:'s have been encountered.
Note: The provided Rcpt To:
2.4.1 badrcptto
By populating control/badrcptto you reject emails to Recipients listed there in already in the SMTP session. Wildcards are allowed. If you don't wont to receive emails for root (from the Internet) include in control/badrcptto:
root@*
2.4.2 badrcptto Whitelisteing
Alternatively to the Recipients mechanism, as a side-effect of the wildmat filtering, you can use the control/badrcptto file as an effective whitelisting mechanism. The trick is, to initially reject everything while later to allow specific Recipients:
*
!*@otherdomain.com
!user1@maydomain.com
!user2@mydoamain.con
Note: The evaluation of control/badrcptto is done independent from the setting of the RELAYCLIENT environment variable.
2.4.2 Restricting the number of Recipients
The environment variable
MAXRECIPIENTS
can be used to restrict the number of counted "Rcpt To: "s in the SMTP session. By default, no restriction is facilitated.
2.4.3 Tarpitting
I have included Chris Johnson's TARPITTING patch into SPAMCONTROL:
"What is tarpitting? It's the practice of inserting a small sleep in a SMTP session for each "Rcpt To:" after some set number of "Rcpt To:"s. The idea is to that spammers who would hand your SMTP server a single message with a long list of RCPT TOs. If a spammer were to attempt to use your server to relay a message, say, 10,000 Recipients, and you inserted a five-second delay for each Recipient after the fiftieth, the spammer would be 'tarpitted', and would most likely assume that the connection had stalled and give up."
Two additional control files can be employed:
control/tarpitcount
is the number of "Rcpt To:"s to accept before starting tarpitting and defaults to 0 (no tarpitting).
control/tarpitdelay
is the number of seconds of delay to introduce after each subsequent "Rcpt To:". Default is 5 seconds.
Instead, the environment variables TARPITCOUNT and TARPITDELAY can be used.
Note: In combination with the Recipients extension, the TARPITCOUNT is used to terminate the SMTP session if the number of invalid Recipients ("Rcpt to:") exceeds the TARPITCOUNT. Unlike the typical tarpitting mechanism, this is a hard limit (Smart Rejection).
--------------------------------------------------------------------------------
3. Recipients Extension
3.1 Scope
qmail-smtpd accepts messages if the SMTP domain part of Recipient address ("Rcpt to:
For normal SMTP mail traffic that's fine as long as the rate of undeliverable messages don't exceed 10% and the Sender is 'legitmate'; ie. exists. Today's situation is different: Spam and Virus attacks with forged/faked Sender addresses to a bunch of random Recipient addresses yield an undeliverable rate up to 90%. Worse, the generated bounces will never reach the Sender and a double-bounce is eventually send to the postmaster.
3.2 qmail-smtpd Recipients
The Recipients extension makes qmail-smtpd aware of acceptable Recipients and is employed in a none-RELAYCLIENT case only. The Recipients are kept in 'fastforward' compatible cdbs for a case-insensitive quick lookup during the SMTP session.
The Recipients mechanism supports Qmail's address extensions. If a Recipient address like 'foo@mydomain.com' is included in a cdb, all VERP addresses like 'foo-bar@mydomain.com' are accepted for SMTP reception.
Within the Recipients mechanism you can define domain-wide wildlisting. Simply include '!@domain' into one of the cdbs to allow all addresses for domain "domain" to be accepted. For performance reasons it is advisable to put those "wilddomains" at the beginning of the first cdb.
Unqualified Recipient addresses are always translated to full qualified, appending the domain part '@localhost' (eg. 'foo' -> 'foo@localhost').
3.3 Usage
The qmail-smtpd Recipients extension is available by means of the control file
control/recipients
Here, you include the path to fastforward compatible cdbs in which you keep the Recipient addresses (in lower case).
Build a list of local Recipients (with full qualified address).
Use qmail-pwd2recipients to build this list for local system users.
Use qmail-alias2recipients to build this list for qmail alias users (ie. postmaster, root).
Use qmail-users2recipients to build this list for qmail users (as per users/assign).
Verify that list to be found under users/recipients. If you have a different Qmail home directory, modify the above scripts.You may need to change "localhost" in the above scripts to the real hostname.
Run qmail-recipients to transform that list into a cdb: users/recipients.cdb.
Edit control/recipients and include users/recipients.cdb therein.
If you have fastforward cdbs (those which are generated by setforward) you have to place the output somewhere in a subdirectory under Qmail's home directory and include those into control/recipients.
At that time, your control/recipients file may look like:
control/wilddomains.cdb
users/recipients.cdb
etc/fastforward.cdb
You can add an arbitrary number of cdbs to control/recipients. Any change regarding control/recipients and/or the content of the cdbs is effective on the fly. If you use qmail-recipients to build up several cdbs, simply rename them afterwards.
If you run EZMLM, you have simply to set up a list of Recipient addresses for all your mailing lists.
3.4 Customization
The Recipients extension needs no customization except for the following circumstances:
Compile-time options:
Your Qmail address extension string is not '-'. This can be modified in recipients.c (#define AUTO_BREAK).
You dont want the Recipients extension to support Qmail's VERP mechanism. This is controlled via "verp=yes" in conf-spamcontrol instead the default 'no'.
Run-time options:
Since some MTAs (namely Exchange) interprete a SMTP 450 reply to try to deliver the email even more frequently, the default behavior of qmail-smtpd is now to issue a 550 SMTP reply resulting in a "550, sorry no mailbox by that name (#5.7.1)" message. The advantage is, that a potential Sender receives a quick bounce with the propper reason, instead a deferred one. The disadvantage is, that this might be used for address harvesting
However, setting the environment variable RECIEPITNS450 in case the Recipients check is negative, qmail-smtpd issues the error: "450 sorry, mailbox currently unavailable (#4.2.1)".
Optional scripts:
You may need to adjust the provided scripts qmail-pwd2recipients, qmail-users2recipients, and qmail-alias2recipient to your need; these are samples.
3.5 Results
With the Recipients extension qmail-smtpd will act for none-RELAYCLIENTs like follows
a) The domain part of the Recipient address is checked against rcpthosts/morercpthosts.cdb and accepted if either it matches or the domain is not provided.
b) the Recipient address is checked against all readable cdbs listed in control/recipients and accepted if
either the entire address matches,
a VERP-part of the address matches,
the domain part of address is wildcarded or,
for the unqualified address an entry exist in the form 'user@localhost'.
In any other case, a SMTP temporary failure protocol error is issued to the client saying:
"450 sorry, mailbox currently unavailable (#4.2.1)" unless
the environment variable RECIPIENTS550 is specified and qmails-smtpd issues the rejection message "550, sorry no mailbox by that name (#5.7.1)"
--------------------------------------------------------------------------------
4. Filtering E-Mails on behalf of the Message Content
Based on the "qmail-smtp-viruscan-1.1.patch" by Russell Nelson (and Charles Cazabon), SPAMCONTROL includes my WARLORD extension, which is a much robuster and efficient filter for BASE64 encoded MIME attachments and bundled with the Qmail High Performance Scanner Interface (QHPSI):
BASE64 encoded MIME attachments are detected and can be filtered accordingly to an easily extendable badmimetypes file.
Within the BASE64 encoded MIME attachments so-called specific loader assignments (ie. for Windows OS) can be detected and messages - containing these suspicious badloadertypes patterns - are rejected.
Additional and optional on-the-fly scanning of emails through the QHPSI.
Initial badmimetype or badloadertype flagged messages are not subject of the QHPSI.
Employing the environment variable BASE64, QHPSI is advised to by-pass virus scanning if no BASE64 encoded attachment is found.
In case a badmimetype or badloadertype filter condition is met or a virus is detected, qmail-smtpd sends a SMTP 554 reply to the sender "554 sorry, invalid message content (#5.3.2)". Populating the REPLY554 environment variable, allows to include additional information (typically an URL), which can be used to deal with potential false-positives.
4.1 The BADMIMETYPE-Filter
The badmimetype filter becomes active if
the environment variable BADMIMETYPE="" is set
and the control file badmimetypes is populated and readable by qmail-smtpd.
4.2 Control file badmimetypes and badmimetypes.cdb
The control file control/badmimetypes.cdb is populated by the additional program qmail-badmimetypes which takes the input of control/badmimetypes. New MIME signatures can be added/removed on-the-fly. Bad MIME Type signatures have to have the length of at least 9 significant characters.
The currently included MIME signatures are:
TVqQAAMAA
TVpQAAIAA
TVpAALQAc
TVpyAXkAX
TVrmAU4AA
TVrhARwAk
TVoFAQUAA
TVoAAAQAA
TVoIARMAA
TVouARsAA
TVrQAT8AA
# *.zip
# UEsDBAkAA
# *.z (gnu-zip)
# H4sIADWWb
# double Base 64 Windows Executable
VFZxUUFBT
# triple Base 64 Windows Executable
VkZaeFVVR
# Pif File
TVoAAAEAA
# Bagle Virus
ZGltIGZpb
Adding new badmimetypes is simple:
Send an E-Mail to a Unix account with corresponding attachment (i.e. *.zip).
Use an editor to view the E-Mail and spot the corresponding BASE64 encoded content-type.
Take the first nine significant characters (for type "zip" its "UEsDBAkAA") and include them into control/badmimetypes.
Run qmail-badmimetypes.
Comments (starting with "#") are allowed in badmimetypes; the length of the signature will be truncated to nine characters.
4.3 The BADLOADERTYPE-Filter
The badloadertype filter becomes active if
the environment variable BADLOADERTYPE="M" is set (see below)
and the control file badloadertypes is populated and readable by qmail-smtpd.
The BADLOADERTYPE mechanism deals in particular with "transport stealth" worms, ie. UPX encoded Windows executables.
4.4 Control file badloadertypes and badloadertypes.cdb
badloadertypes.cdb is populated by the additional program qmail-badloadertypes which takes the input of control/badloadertypes The badloadertype mechanism looks for five significant strings in the BASE64 encoded data-stream which is matched against an entry in control/badloadertypes.cdb. badloadertype signatures can be added/removed on-the-fly.
The currently included Windows OS badloadertype signatures are:
Mi5kb
MzIuZ
MyLmR
MyLkR
Comments (starting with "#") are allowed in badloadertypes; the length of the signature will be truncated to five characters.
Caution: Unlike the badmimetype, the badloadertype signatures are placed anywhere in the BASE64 encoded datastream and are difficult to find out. In order to make the search efficient, a common character has to be providen in the environment variable BADLOADERTYPE. The provided pattern look basically for a string like "32.dll" as a subpart of "Kernel32.dll" which is an indication for an executable for the Windows OS. However, there is a small chance for false positives. Some - lets say - Word document attached as BASE64 MIME part in the message containing the buzz words "kernel32.dll" might become flagged and finally rejected as well.
4.5 Employing an AV Scanner with QHPSI
Unlike all other AV Scanners currently in use for Qmail, with Qmail High Performance Scanner Interface (QHPSI) there is no need for any other umbrella program, neither qmail-scanner, AMAViS, qscanq or whatsoever. Further, no additional MIME analyzing program like reformime, metamail, or ripmime is required. Even better, no "staging" area for temporary files are needed, except the one, the AV Scanners requires for itself.
Today's AV Scanners - and in particular Clam AV - are able to read the BASE64 encoded message and eventually dig out the files in archives, ie. in zip format. In order to use an AV Scanner with QHPSI, the AV Scanner has to have the following qualifications:
Correct interpretation of the BASE64 and perhaps the uudecoded data in order to detect the virii/worms therein.
Results have to be made available on STDERR/STDOUT.
(And perhaps) Suppression of 'negative' scan results.
The QHPSI allows to use the following environment variables:
QHPSI="av_scanner" - this is the path to the involved AV Scanner.
QHPSIARG1, QHAPSIARG2, QHPSIARG3 - to call the AV Scanner with three arbitrary arguments.
QHPSIRC="RC" - if the AV Scanner does not return with RC=1 in case of a detected infection, you can specify one here.
QHPSIMINSIZE="size-in-bytes" - the minimum size of the message to scan.
QHPSIMAXSIZE="size-in-bytes" - the maximum size of the message to scan.
The AV Scanner is directly called in the start scripts of Qmail (i.e. the run script for qmail-smtpd) or by means of tcpserver's capabilities. Here is a typical example, how to customize QHPSI together with Clam AV (clamd/clamdscan) for a tcpserver tcp.smtpd file:
:allow,QHPSI='clamdscan',QHPSIARG1='--disable-summary'
Comments:
The path of clamdscan can be omitted, because it is in the standard path (/usr/local/bin).
The argument QHPSIARG1='--disable-summary' tells Clam AV to provide a single line output of the scan results.
In clamd's clamav.conf configuration file the "Mail support" has to be enabled, clamd has to run as root.
Note:
qmail-start does not pass environment variables to qmail-queue.
If you need to scan in addition outgoing emails, you have to write a wrapper for e.g. qmail-inject:
#!/bin/sh
export QHPSI='clamdscan'
export QHPSIARG1='--disable-summary'
exec /var/qmail/bin/qmail-queue
Results:
In case a client sends an infected email by means of SMTP, it receives the SMTP return code and error message: "554 mail server permanently rejected message (#5.3.0)"
In case the virus scanner can not be called properly or has an internal problem, the following SMTP message is given: "454 mail server temporarily rejected message (#4.3.0)"
Eventually, the scan results show up in the logs.
Here is a sample of Clam AV without and with the argument "--disable-summary":
@... tcpserver: pid 49943 from 192.168.192.11
@... tcpserver: ok 49943 qmailer.fehnet.de:192.168.192.2:25 arkon.fehnet.de:192.168.192.11::1074
@... /var/qmail/queue/mess/4/89439: Worm.Klez.H FOUND
@...
@... ----------- SCAN SUMMARY -----------
@... Infected files: 1
@... Time: 0.099 sec (0 m 0 s)
@ Reject::DATA::Virus_Infected: S:192.168.192.11:arkon.fehnet.de H:mail.fehnet.de F:me T:erwin 'clamdscan'
Note: Even in case no virus is detected, the "SCAN SUMMARY" is provided.
@... tcpserver: pid 49989 from 192.168.192.11
@... tcpserver: ok 49989 qmailer.fehnet.de:192.168.192.2:25 arkon.fehnet.de:192.168.192.11::1077
@... /var/qmail/queue/mess/4/89543: Worm.Klez.H FOUND
@ Reject::DATA::Virus_Infected: S:192.168.192.11:arkon.fehnet.de H:mail.fehnet.de F:me T:erwin 'clamdscan'
@... tcpserver: end 49989 status 256
Attention:
The virus scanner has to be able to read the messages in the Qmail queue:
# ls -ld /var/qmail/queue/mess/
drwxr-x--- 25 qmailq qmail 4096 Jun 13 00:19 /var/qmail/queue/mess/
If the virus scanner drops privileges during execution, it might be necessary to make the executable sticky (ie. clamscan).
Note: As with this writing, clamav 0.8x is broken, since it writes all logs to STDOUT instead of STDERR; thus no scanning messages will apear in the qmail-smtpd log.
4.6 QHPSI performance improvements for Virus Scanning
The badmimtypes and badloadertypes mechanism provides a wire-speed filtering of incoming emails. However, typically all not-filtered emails are subject of the AV Scannner as defined via the QHPSI. Almost all worms and virii are transported as BASE64 encoded attachments (except some trojans, encapsulated as HTML files). By means of the environment variable
BASE64=""
one can advice QHPSI to scan only those emails which contain a BASE64 encoded attachment.
4.7 Particular SMTP 554 Replies
qmail-smtpd will send a SMTP 554 Error Reply under the following conditions:
a badmimetype was encountered,
a badloadertype was encountered,
a Virus was found via an external AV Scanner,
the email exceeds the hop count.
The SMTP Reply code for the first three conditions is always "554 sorry, invalid message content (#5.3.2)". The rejection of email because of the message content is due to some internal policy. For those users, which a subject of this policy innocently (and did not send ie. a virus mail on purpose), it might be advisable to explain the company's email policy.
The environment variable
REPLY554
allows to include a particular SMTP 554 Reply. Typically, an URL is referenced: REPLY554="[ see: http://www.fehcom.de/emailpolicy.html ]" which allows to detail possible circumventions.
4.8 Qmail QUEUE_EXTRA
Bruce Guenter's Qmail QUEUE_EXTRA patch has almost the rank of a recommended patch, because it's used by many Qmail extensions like the Qmail-Scanner and qmail-qfilter.
The actual use is controlled via the content of environment variable "QMAILQUEUE", which usually set in a tcpserver's cdb ie. tcp.smtp or globally defined in the qmail-smtpd's run script. A typical use is:
12.34.56:allow,RELAYCLIENT=""
:allow,QMAILQUEUE="bin/qmail-qfilter"
which advices qmail-smtpd to use the executable qmail-qfilter as first stage queueing program instead of qmail-queue itself.
Note: The QUEUE_EXTRA patch is not applied against qmail-smtpd but rather against the module qmail.c itself, since it is just an extension to the general queue call-mechanism.
.
--------------------------------------------------------------------------------
5. qmail-remote Extensions
SPAMCONTROL modifies and extends the behavior of qmail-remote in the following ways:
Adding ESMTP support and in particular SMTP Authentication of types PLAIN and LOGIN.
Connection attempts to all MX-listed MTAs in case of a rejection and not just the primary.
Fast delivery by means of an increased reading buffer for the message.
5.1 qmail-remote Authentication
The qmail-remote (qmail-smtp-auth-send) authentication from Bjoern Kalkbrenner has been included in a modular and RFC-complient version. qmail-remote sessions can be SMTP authenticated with the types PLAIN and LOGIN on a per-sender (Reverse-Path) base. Thus for each sender you can advice qmail-remote to use SMTP authentication with a particular username and password connecting to relay at port.
The qmail-remote authentication follows in this respect the smtproutes mechanism. Authentication for outgoing SMTP sessions is faciliated, if the control file
control/authsenders
is populated accordingly. Sample:
mail@example.com|test|testpass
info@example.com:smtp.example.com:26|other|otherpw
:mailrelay.example.com|e=mc2|testpass
5.2 qmail-remote MX connectivity
Typically sites/domains on the Internet are reachable over serveral MTA listed and deployed in the DNS MX records (o.k. qmail.org is an exception). By theory, the MX with the smallest weight is the primary MX for that domain;though often sites have redundant MTA with equal weights:
10 mailc.microsoft.com
10 maila.microsoft.com
10 mailb.microsoft.com
In order to deliver emails, qmail-remote follows two strategies:
qmail-remote will deliver only one email per connection.
qmail-remote always connects to the MTA with the lowest weight coming first in the list.
In case this MTA is exhausted and rejects the connection during the EHELO/HELO greeting, qmail-remote exists and retries the very same MTA again with it's quadratic queue schedule mechanism.
Running EZMLM with many messages to the vary same domain but different Recipients, email delivery may become throttled, which particularly happens for t-online.de sites which don't allow too many connections from the same client MTA (a policy which is actually not covered by any SMTP RFC).
Back 10 years ago, when Dan was designing qmail he already was aware of that problem:
"If I successfully connect to an MX host but it temporarily refuses to accept the message, I give up and put the message back into the queue.
But several documents seem to suggest that I should try further MX records. What are they thinking? My approach deals properly with downed
hosts, hosts that are unreachable through a firewall, and load balancing; what else do people use multiple MX records for? " (THOUGHTS)
and included already the code base into qmail-remote which I simply activated. Thus, in case qmail-remote receives a rejection during the EHLO/HELO greeting it will simply try the next MTA for the DNS MX list.
5.2 Fast delivery for qmail-remote
qmail-remote incorporates two performance critical steps for the delivery:
"Currently qmail-remote sends data in 1024-byte buffers. Perhaps it should try to take account of the MTU. " (THOUGHTS)
qmail-remote processes the input data byte after byte which is costly in terms of CPU cylces.
Bruce Guenter recognized the last fact and patched qmail-remote accordingly ("fastremote"), thus qmail-remotes processes the input data in chunks of 4 Kbyte. This patch has been included into SPAMCONTROL.
--------------------------------------------------------------------------------
6. qmail-pop3d
6.1 STLS support
The STLS (Start TLS support) for qmail-popup follows the same scheme as for qmail-smtpd
Requirements:
Put the SSL variables into the environment (see 1.3.3).
Define UCSPITLS="" in qmail-pop3d's run script.
Actually, you use the same "env" file as for qmail-smtpd. In this case qmail-popup announces in the capability list "STLS" and the following POP3 dialogue is encrypted as is the transmission of the received emails as well.
6.2 qmail-pop3d run Script
In order to use SSL encryption for a POP3 connection, the following run script for qmail-pop3d is appropriate:
#!/bin/sh
HOSTNAME=`hostname`
export UCSPITLS=""
. /var/qmail/ssl/env
exec /usr/local/bin/sslserver -sevn -R -c100 0 pop3 \
/var/qmail/bin/qmail-popup $HOSTNAME /usr/local/bin/checkvpw \
/var/qmail/bin/qmail-pop3d Maildir 2>&1
Note: In this run script I use Bruce Guenter's checkvpw as PAM (for vmailmgr), which requires the additional presence of the "Maildir" argument after the call of qmail-pop3d.
The profile /var/qmail/ssl/env is the same as for qmail-smtpd. Defining the environment variable UCSPITLS directly in the run script instead of the profiles, allows a flexible use of the STARTTLS/STLS option for qmail-pop3d and qmail-smtpd without modifying the common profile.
--------------------------------------------------------------------------------
7. qmail-send. qmail-queue, and the qmail's sendmail
7.1 qmail-send: Bounces
Bounces have generally a Null-Sender address (Mail From: <>) and are out-of-band error-messages to indicate a failure in the delivery process. In fact, RFC 2821/821 requires that all notification E-Mails have to have a Null-Sender address!
For every undeliverable message, qmail-send generates a bounce to the Sender.While this is legitimate and necessary for normal operation, in case of SPAM attacks the bounces are meaningless:
The SPAMMER uses a lexical approach to generate random addresses, thus most of the E-Mails are undeliverable per se.
The SPAMMER's Sender address is forged or is only limited available (during the SPAM attack), thus the bounce fail; and eventually generate a double-bounce. One undeliverable SPAM message may result in two additional bounces.
Unless you use a 'whitelisting' of Recipient E-Mail addresses, there is not much to do about. However, SPAMCONTORL helps you in three cases:
7.1.1 Limiting the Size of Bounces
By definition, a bounce is a SMTP notification for a failure situation. It is common practice, to include the original message in the bounce. Qmail uses a specific format, introduced by Dan Bernstein and called "QSBMF" (qmail-send Bounce Message Format); other MTA encapsulate the original message as MIME attachment in rfc822/message format.
Anyway, for a legitimate bounce reaching the Sender the original message is usually of no interest, except for identification purposes. In order to save bandwidth, you can limit the size of bounces using the control file
control/bouncemaxbytes
Unlike the original patch (from Frank DENIS aka Jedi/Sector One
The original message included in the bounce will be limited to the defined bouncemaxbytes and truncated, which is displayed in the bounce with "--- Rest of message truncated." at the end of the bounce.
7.2.1 Dumping Double Bounces
Double bounces are generated, if the bounce can not be delivered to the Sender.
Double bounces are usually delivered to the 'Postmaster' account. It is convenient that this account is local and eventual double bounces are stored in a mbox/Maildir for later inspection. However, Qmail allows you to forward double bounces to some other account defined in
control/doublebounceto
However, due to the forged Sender address in SPAM E-Mails, practically all bounces become double bounces eventually. In this case any storage and inspection is fruitless. Taken from Russell Nelson and Charles Cazabon, you can optionally dump all double bounces immediately. This is facilitated if doublebounceto contains a '@' in the first line.
Those dumped double bounces show up in the qmail-send log as: "double bounce: discarding".
7.3 qmail-send: IPME- and MOREIPME-Patch
Scott Gifford's "ipme.c" patch (or qmail-0.0.0.0-patch) is included by default. According to RFC 1122, Sec. 3.2.1.3 the IP address "0.0.0.0" is a special address which always refers to "this host, this network". You may wish to tell Qmail about arbitrary IP addresses employing the moreipme patch and include the following control files:
control/moreipme
control/notipme
See the enhanced man page qmail-smtpd and or consult the README.moreipme.
7.4 qmail-send Gadgets
As a further gadget, the qmail-send control files
control/concurrencylocal
control/concurrencyremote
are re-read by means of a HUP signal (eg. svc -h /service/qmail-send).
7.5 qmail-queue: BigToDo enhancement
The queue directories ./intd and ./todo may be splitted (as per conf-split) into subdirectories to allow a more efficient treatment of many incoming messages.
Caution: Make sure, that the directories ./queue/todo and ./queue/intd are empty before applying the patch; otherwise qmail-send will not be able to process those messages anymore!
Note: The shell script qmail-qstat and in addition some qmail-mrtg analyses are affected by this patch.
Hint: Consider raising the value in conf-split in the first place !
7.6 sendmail Fixes
The following fixes for Qmail's sendmail wrapper have been included for compatibility reasons:
The 'sendmail -f' patch by David Philips, thus sendmail sets the default for the username.
The 'sendmail -N dsn' patch by Matthias Andree, thus sendmail ignores the -N option.
--------------------------------------------------------------------------------
8. SMTP Protocol Return Codes
SMTP allows to reject Sessions based on some technical and/or political criteria, which are not well expressed in the RFCs (2821, 2554, 2505, 1122).
The SMTP protocol mechanism between the client and the server are defined as Commands and Replies. SMTP uses a three-letter Reply Code. The first digit tells whether a command was accepted and completed (2), transaction begin (3), or whether there was as transient (4) or permanent failure (5). In addition, an explanatory description may be given.
RFC 1893 introduces a concept of "Enhanced Mail System Status Codes" (EMSSC) which should provide easily parseable SMTP server conditions and transaction statuses, usually at the end of the SMTP reply and included in parenthesis, eg. (#5.5.1).
The SMTP Reply Codes and the EMSSC are detailed in the corresponding RFCs, but don't fit well to each other, thus either providing redundant information or almost no additional information at all. In short, the EMSSC is nowadays almost meaningless.
Here's a breakdown of SPAMCONTROL's SMTP Reply Codes, informational texts, and the used EMSSC.
Reply Informational text EMSSC
450 sorry, mailbox currently unavailable (#4.2.1)
451 DNS temporary failure (#4.3.0)
454 TLS not available due to temporary reason (#5.7.3)
501 auth exchange canceled (#5.0.0)
501 malformed auth input (#5.5.4)
503 you're already authenticated (#5.5.0)
503 no auth during mail transaction (#5.5.0)
503 sorry, SMTP Authentication not available (#5.7.3)
504 auth type unimplemented (#5.5.1)
535 authorization failed (#5.7.1)
535 authentication required (#5.7.1)
535 STARTTLS required (#5.7.1)
550 sorry, invalid HELO/EHLO greeting (#5.7.1)
550 sorry, your envelope recipient is in my badrcptto list (#5.7.1)
550 sorry, invalid sender address specified (#5.7.1)
550 sorry, too many recipients (#5.5.3)
550 sorry, bounce messages should have a single envelope recipient (#5.7.1)
552 sorry, that message size exceeds my databytes limit (#5.3.4)
553 sorry, your envelope sender is in my badmailfrom list (#5.7.1)
550 sorry, that domain isn't in my list of allowed rcpthosts (#5.7.1)
553 sorry, your envelope sender domain must exist (#5.7.1)
554 too many hops, this message is looping (#5.4.6)
554 sorry, invalid message content (optional text) (#5.3.2)
--------------------------------------------------------------------------------
9. qmail-smtpd Logging
Normally, qmail-smtpd doesn't log anything.With SPAMCONTROL qmail-smtpd logs accepted and some (important) rejected SMTP session attempts. The logging is done at the end of the "Rcpt To:" and eventually at the end of the "Data" phase.
Format: "Action::Type::Condition: Information"
9.1 Extensible logging scheme
Action Type Condition Explanation
Reject SMTP Toomany_Hops Message Hop count exceeded
Reject SMTP Syntax_Error Malformed SMTP address (e.g. missing brackets)
Reject DATA Invalid_Size DATA exceeds sizelimit
Reject DATA Bad_MIME DATA includes BASE 64 MIME type listed in badmimetypes
Reject DATA Bad_Loader DATA includes BASE64 loader type listed in badmimetypes
Reject DATA Virus_Infected DATA includes virus infected message (QHPSI)
Reject SNDR Bad_Helo SNDR's HELO is in the badhelo
Reject SNDR DNS_HELO SNDR's HELO has no DNS A/MX RR
Reject SNDR Invalid_Relay SNDR's tries relaying; but not allowd
Reject SNDR Missing_TLS STARTTLS was required but not granted by client
Accept Accept Relay_Client SNDR was identfied as relay client
Accept Accept Start_TLS SNDR succesfully started TLS
Reject ORIG Bad_Mailfrom ORIG is in badmailfrom
Reject ORIG DNS_MF Domain part of ORIG has no DNS MX RR
Reject ORIG Failed_Auth ORIG tried SMTP Authentication; but failed
Reject ORIG Invalid_Sender ORIG not allowed to send
Reject ORIG Missing_Auth SMTP Authentication required, but not granted
Accept ORIG Valid_Auth ORIG was successful authenticated
Accept ORIG Local_Sender ORIG was identified as local sender address
Accept ORIG Relay_Mailfrom ORIG was accepted als Relaymailfrom
Reject RCPT Bad_Rcptto RCPT is in badrcptto
Reject RCPT Toomany_Rcptto Too many RCPTs
Reject RCPT Failed_Rcptto RCPT could not acceptd as per recipients/cdb.
Accept RCPT Recipients_Rcptto RCPT was accepted as per recipients/cdb.
Accept RCPT Recpients_VERP RCPT was accepted per VERP address in recipients/cdb.
Accept RCPT Recpients_Domain RCPT was accepted per Domain wildlisting in recipients/cdb.
Accept RCPT Rcpthosts_Rcptto RCPT was accepted as per rcpthosts/morercpthosts
SNDR corresponds to the sending MTA.
ORIG is the "MAIL From:
RCPT is the "RCPT To:
DATA is the Message.
The Information includes typically the following
PROTOCOL: SMTP/ESMTP/EMSTPA/ESMTPS/EMSTPM
SNDR: IP/FQDN (from tcpserver/sslserver)
HELO/EHLO: The provided greeting string
ORIG: The content of the Mail From: (if applicable).
RCPT. The intended Rcpt To:
This scheme is easy extensible to other successful/deferred SMTP sessions. Sample:
Accept::SNDR::Relay_Client: P:orion.fehnet.de S:81.173.229.48:xdsl-81-173-229-48.netcologne.de H:mail.fehcom.de F:feh@fehcom.de T:erwin@example.com
9.2 qmail-smtpd Start-Up Script Using tcpserver And splogger
A typical tcpserver start script applying standard splogger:
#!/bin/sh
QMAILDUID=`id -u qmaild`
QMAILDGID=`id -g qmaild`
/usr/local/bin/tcpserver -v -R -H -u $QMAILDUID -g $QMAILDGID QMAIL_IP_ADDRESS smtp \
/var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1 | \
/var/qmail/bin/splogger smtpd &
Since splogger is now facilitated, ACCUSTAMP time information is included.
9.3 qmail-smtpd Start-Up Script Using Daemontools, tcpserver, And multilog
A better choice would be multilog. multilog allows you to write separate filtered logs; to individual directories, and/or files, STDERR respectively.A typical Daemontools qmail-smtpd run script would look like:
#!/bin/sh
QMAILDUID=`id -u qmaild`
QMAILDGID=`id -g qmaild`
/usr/local/bin/tcpserver -R -u $QMAILDUID -g $QMAILDGID QMAIL_IP_ADDRESS smtp \
/var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1
Note: tcpserver's logging via the '-v' flag can be omitted to get mostly a full comprehensive and terse one-line logging of the SMTP session.
The corresponding multilog run script allows not only to filter the log information and write them to the file "current" in a specific directory but in addition to feed a file with specific information; here's a sample:
#!/bin/sh
exec setuidgid qmaill multilog t \
'-*tcpserver: status:*' /var/log/qmail-smtpd \
'-*' '+*Reject::*' =/var/log/qmail-smtpd/rejected \
'-*' '+*DNS*' =/var/log/qmail-smtpd/nodnsmx
In this case, multilog adds at first a TAI64 time stamp.
In a second step, the log information - except those line including the string "tcpserver: status" - are written into the log file /var/log/qmail-smtpd/current.
On the third line the filter "-*" removes any unwanted lines and "+*Reject::*" picks up from qmail-smtpd's log all rejected sessions and placing the last rejected condition in the file 'rejected'!
On the fourth line the last failing DNS MX lookups for Mail From: is logged in a file named /var/log/qmail-smtpd/nodnsmx.
9. SPAMCONTROL compile Options
In this version of SPAMCONTROL I have substantially reduced the number of compile-time options:
In order to consistently change all relevant binaries, use the file conf-spamcontrol which is evaluated by the installation routine install_spamcontrol.sh and passes the changes to the Qmail c-files:
# Configuration for SPAMCONTROL (no tabs allowed)
#
# Additional RELAYING
#
relaymailfrom=no # might be dangerous - use SMTP Auth
#
# Additional CONTROLLING
#
reqbrackets=yes # qmail-smtpd requires brackets "" in SMTP addresses
verp=yes # allow VERP addresses for RECIPIENTS
#
# LOADSHARING enhancements
#
moreipme=no # Scott Gifford's additional control files moreipme and notipme
#
# PERFORMCANCE enhancements
#
bigtodo=no # Bruce Guenter's BigToDo patch
10. Howto
In case your E-Mail environment complies to the assumption in PURPOSE do the following:
Stop your Qmail system (receive and send).
Remove SMTP/POP3 connection to be serviced by INETD/XINETD.
Employ tcpserver instead.
Follow the INSTALL.spamcontrol instructions.
Edit the files
./control/badhelo
./control/badmailfrom
./control/badrcptto
./control/tarpitcount
./control/tarpitdelay
./control/moreipme
./control/notipme
./control/bouncemaxbytes
./control/recipients
to your needs.
See above samples and check the included samples for ./badmailfrom and ./badrcptto.
Restart Qmail.
Check your configuration by means of qmail-showctl. It is a good idea to pipe the output of that command with timestamp information to a file.
Test your filters locally (see TESTING.spamcontrol).
If you are already blacklisted, inform those sites that you don't act as an OPEN RELAY anymore.
Watch the Qmail behavior by means of the log file information.
*) Not useful, if tcpserver in employed.
Good luck!
10.1 Tested Environment
LINUX KERNEL 2.4
FREEBSD 3.x, 4.x, 5.x, 6.x
SOLARIS 2.7, 2.8
HP-UX 11
Attention: For 64 bit OS'es you may need in addition the qmail-isoc patch from James Craig Burley (http://www.jcb-sc.com/qmail/patches/qmail-isoc.patch).
10.2 Incompatibilities
The SPAMCONTROL patch is incompatible with the Qmail LDAP patch. It should be applied against qmail-1.03 and not against netqmail-1.0x.
Actually, the Qmail LDAP enhancement is a super set of SPAMCONTROL. If you need to incorporate parts of SPAMCONTROL into the LDAP patch, look at the different pieces and pick 'em up from the sources.
THANKS !!!!!!!!
ANAND Shah