How does ssl work

Internet communication typically runs through multiple program layers on a server before getting to the requested data such as a web page or cgi scripts.

The outer layer is the first to be hit by the request. This is the high level protocols such as HTTP (web server), IMAP (mail server), and FTP (file transfer).

Determining which outer layer protocol will handle the request depends on the type of request made by the client. This high level protocol then processes the request through the Secure Sockets Layer. If the request is for a non-secure connection it passes through to the TCP/IP layer and the server application or data.

If the client requested a secure connection the ssl layer initiates a handshake to begin the secure communication process. Depending on the SSL setup on the server, it may require that a secure connection be made before allowing communication to pass through to the TCP/IP layer in which case a non-secure request will send back an error asking for them to retry securely (or simply deny the non-secure connection).

This is a good time to answer a question we periodically get: "what does ssl encrypt?"

This question is usually geared toward whether or not the path and query string is encrypted in an HTTPS "get" request (this is where form field responses or program variables are tagged on to the end of the url). These fields are stripped off of the URL when creating the routing information in the https packaging process by the browser and are included in the encrypted data block.

The page data (form, text, and query string) are passed in the encrypted block after the encryption methods are determined and the handshake completes.

A related issue that frequently comes up is whether or not form data is transmited with encryption if the blank form is displayed without https. If the form "action" is set to use https then the ssl handshake will take place before the data is sent. Whether or not the original form is displayed using https has little to do with the form submission unless the form action uses a relative path, in which case the default will be to use the protocol that was used to display the form.

PHP SESSION HANDLING EXAMPLE

The below code is a very simple example of Session.
Please note that the "session_start" statement *must* be the first line of your code.
To see the example at work, please review:

http://www.xn--ovg.com/session

tedd


--- copy below and save as session1.php -----
session_start();
?>

"http://www.w3.org/TR/html4/strict.dtd">

Anand's session stuff (page 1)

---

PHPSESSID =

---

Click the next page and see numbr of visits during this visit.

Next page




--- copy below and save as session2.php -----

session_start();
($_SESSION['count']) ? $_SESSION['count']++ : $_SESSION['count'] = 1;
?>

"http://www.w3.org/TR/html4/strict.dtd">

Anand's session stuff (page 2)

---

PHPSESSID =

---

You have been here times in this session.

Previous page

Test the SMTP Service on WIndows

Test the SMTP Service

To test the SMTP service, follow these steps:
1. On a computer running Windows Server 2003, type Telnet at a command prompt, and then press ENTER.
2. At the telnet prompt, type set LocalEcho, press ENTER, and then type open 25, and then press ENTER.

The output resembles the following:

220 computername.microsoft.com ESMTP Server (Microsoft Exchange Internet Mail Service 5.5.2651.58) ready


3. Type helo me, and then press ENTER.

The output resembles the following:

250 OK


4. Type mail from:email@domain.com, and then press ENTER.

The output resembles the following:

250 OK - mail from


5. Type rcpt to:youremail@yourdomain.com, and then press ENTER.

The output resembles the following:

250 OK - Recipient


6. Type Data, and then press ENTER.

The output resembles the following:

354 Send data. End with CRLF.CRLF


7. Type Subject:This is a test, and then press ENTER two times.
8. Type Testing, and then press ENTER.
9. Press ENTER, type a period (.), and then press ENTER.

The output resembles the following:

250 OK


10. Type quit, and then press ENTER.
The output resembles the following:

221 Closing Port / Mail queued for delivery

Disallow download .MDB file

Disallow download of certain file type

If what you want to do is prevent a download of your .mdb file,
just create an App_Data directory and place the Access database in it.

Files stored in the App_Data folder are not returned in response to direct HTTP requests,
which makes the App_Data folder the recommended location for data stored with your
application, including .mdf (SQL Server Express Edition), .mdb (Microsoft Access), or XML files.

Command Line Examples linux

Linux command line reference for common operations.

This is a linux command line reference for common operations.
Examples marked with • are valid/safe to paste without modification into a terminal, so
you may want to keep a terminal window open while reading this so you can cut & paste.
All these commands have been tested both on Fedora and Ubuntu.

Command Description
• apropos whatis Show commands pertinent to string. See also threadsafe
• man -t man | ps2pdf - > man.pdf make a pdf of a manual page
which command Show full path name of command
time command See how long a command takes
• time cat Start stopwatch. Ctrl-d to stop. See also sw
• nice info Run a low priority command (The "info" reader in this case)
• renice 19 -p $$ Make shell (script) low priority. Use for non interactive tasks
dir navigation
• cd - Go to previous directory
• cd Go to $HOME directory
(cd dir && command) Go to dir, execute command and return to current dir
• pushd . Put current dir on stack so you can popd back to it
file searching
• alias l='ls -l --color=auto' quick dir listing
• ls -lrt List files by date. See also newest and find_mm_yyyy
• ls /usr/bin | pr -T9 -W$COLUMNS Print in 9 columns to width of terminal
find -name '*.[ch]' | xargs grep -E 'expr' Search 'expr' in this dir and below. See also findrepo
find -type f -print0 | xargs -r0 grep -F 'example' Search all regular files for 'example' in this dir and below
find -maxdepth 1 -type f | xargs grep -F 'example' Search all regular files for 'example' in this dir
find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; done Process each item with multiple commands (in while loop)
• find -type f ! -perm -444 Find files not readable by all (useful for web site)
• find -type d ! -perm -111 Find dirs not accessible by all (useful for web site)
• locate -r 'file[^/]*\.txt' Search cached index for names. This re is like glob *file*.txt
• look reference Quickly search (sorted) dictionary for prefix
• grep --color reference /usr/share/dict/words Highlight occurances of regular expression in dictionary
archives and compression
gpg -c file Encrypt file
gpg file.gpg Decrypt file
tar -c dir/ | bzip2 > dir.tar.bz2 Make compressed archive of dir/
bzip2 -dc dir.tar.bz2 | tar -x Extract archive (use gzip instead of bzip2 for tar.gz files)
tar -c dir/ | gzip | gpg -c | ssh user@remote 'dd of=dir.tar.gz.gpg' Make encrypted archive of dir/ on remote machine
find dir/ -name '*.txt' | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2 Make archive of subset of dir/ and below
find dir/ -name '*.txt' | xargs cp -a --target-directory=dir_txt/ --parents Make copy of subset of dir/ and below
( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p ) Copy (with permissions) copy/ dir to /where/to/ dir
( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p ) Copy (with permissions) contents of copy/ dir to /where/to/
( tar -c /dir/to/copy ) | ssh -C user@remote 'cd /where/to/ && tar -x -p' Copy (with permissions) copy/ dir to remote:/where/to/ dir
dd bs=1M if=/dev/sda | gzip | ssh user@remote 'dd of=sda.gz' Backup harddisk to remote machine
rsync (Use the --dry-run option for testing)
rsync -P rsync://rsync.server.com/path/to/file file Only get diffs. Do multiple times for troublesome downloads
rsync --bwlimit=1000 fromfile tofile Locally copy with rate limit. It's like nice for I/O
rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html' Mirror web site (using compression and encryption)
rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/ Synchronize current directory with remote one
ssh (Secure SHell)
ssh $USER@$HOST command Run command on $HOST as $USER (default command=shell)
• ssh -f -Y $USER@$HOSTNAME xeyes Run GUI command on $HOSTNAME as $USER
scp -p -r $USER@$HOST: file dir/ Copy with permissions to $USER's home directory on $HOST
ssh -g -L 8080:localhost:80 root@$HOST Forward connections to $HOSTNAME:8080 out to $HOST:80
ssh -R 1434:imap:143 root@$HOST Forward connections from $HOST:1434 in to imap:143
wget (multi purpose download tool)
• (cd cli && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html) Store local browsable version of a page to the current dir
wget -c http://www.example.com/large.file Continue downloading a partially downloaded file
wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/ Download a set of files to the current directory
wget ftp://remote/file[1-9].iso/ FTP supports globbing directly
• wget -q -O- http://www.pixelbeat.org/timeline.html | grep 'a href' | head Process output directly
echo 'wget url' | at 01:00 Download url at 1AM to current dir
wget --limit-rate=20k url Do a low priority download (limit to 20KB/s in this case)
wget -nv --spider --force-html -i bookmarks.html Check links in a file
wget --mirror http://www.example.com/ Efficiently update a local copy of a site (handy from cron)
networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete)
ethtool eth0 Show status of ethernet interface eth0
ethtool --change eth0 autoneg off speed 100 duplex full Manually set ethernet interface speed
iwconfig eth1 Show status of wireless interface eth1
iwconfig eth1 rate 1Mb/s fixed Manually set wireless interface speed
• iwlist scan List wireless networks in range
• ip link show List network interfaces
ip link set dev eth0 name wan Rename interface eth0 to wan
ip link set dev eth0 up Bring interface eth0 up (or down)
• ip addr show List addresses for interfaces
ip addr add 1.2.3.4/24 brd + dev eth0 Add (or del) ip and mask (255.255.255.0)
• ip route show List routing table
ip route add default via 1.2.3.254 Set default gateway to 1.2.3.254
• tc qdisc add dev lo root handle 1:0 netem delay 20msec Add 20ms latency to loopback device (for testing)
• tc qdisc del dev lo root Remove latency added above
• host pixelbeat.org Lookup DNS ip address for name or vice versa
• hostname -i Lookup local ip address (equivalent to host `hostname`)
• whois pixelbeat.org Lookup whois info for hostname or ip address
• netstat -tupl List internet services on a system
• netstat -tup List active connections to/from system
windows networking (Note samba is the package that provides all this windows specific networking support)
• smbtree Find windows machines. See also findsmb
nmblookup -A 1.2.3.4 Find the windows (netbios) name associated with ip address
smbclient -L windows_box List shares on windows machine or samba server
mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/share Mount a windows share
echo 'message' | smbclient -M windows_box Send popup to windows machine (off by default in XP sp2)
text manipulation (Note sed uses stdin and stdout, so if you want to edit files, append newfile)
sed 's/string1/string2/g' Replace string1 with string2
sed 's/\(.*\)1/\12/g' Modify anystring1 to anystring2
sed '/ *#/d; /^ *$/d' Remove comments and blank lines
sed ':a; /\\$/N; s/\\\n//; ta' Concatenate lines with trailing \
sed 's/[ \t]*$//' Remove trailing spaces from lines
sed 's/\([\\`\\"$\\\\]\)/\\\1/g' Escape shell metacharacters active within double quotes
• seq 10 | sed "s/^/ /; s/ *\(.\{7,\}\)/\1/" Right align numbers
sed -n '1000p;1000q' Print 1000th line
sed -n '10,20p;20q' Print lines 10 to 20
sed -n 's/.*(title)\(.*\)<\/title>.*/\1/ip;T;q' Extract title from HTML web page
sort -t. -k1,1n -k2,2n -k3,3n -k4,4n Sort IPV4 ip addresses
• echo 'Test' | tr '[:lower:]' '[:upper:]' Case conversion
• tr -dc '[:print:]' < /dev/urandom Filter non printable characters
• history | wc -l Count lines
set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file)
sort file1 file2 | uniq Union of unsorted files
sort file1 file2 | uniq -d Intersection of unsorted files
sort file1 file1 file2 | uniq -u Difference of unsorted files
sort file1 file2 | uniq -u Symmetric Difference of unsorted files
join -a1 -a2 file1 file2 Union of sorted files
join file1 file2 Intersection of sorted files
join -v2 file1 file2 Difference of sorted files
join -v1 -v2 file1 file2 Symmetric Difference of sorted files
math
• echo '(1 + sqrt(5))/2' | bc -l Quick math (Calculate φ). See also bc
• echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bc More complex (int) e.g. This shows max FastE packet rate
• echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' | python Python handles scientific notation
• echo 'pad=20; plot [64:1518] (100*10**6)/((pad+x)*8)' | gnuplot -persist Plot FastE packet rate vs packet size
• echo 'obase=16; ibase=10; 64206' | bc Base conversion (decimal to hexadecimal)
• echo $((0x2dec)) Base conversion (hex to dec) ((shell arithmetic expansion))
• units -t '100m/9.74s' 'miles/hour' Unit conversion (metric to imperial)
• units -t '500GB' 'GiB' Unit conversion (SI to IEC prefixes)
• units -t '1 googol' Definition lookup
• seq 100 | (tr '\n' +; echo 0) | bc Add a column of numbers. See also add and funcpy
calendar
• cal -3 Display a calendar
• cal 9 1752 Display a calendar for a particular month year
• date -d fri What date is it this friday. See also day
• date --date='25 Dec' +%A What day does xmas fall on, this year
• date --date '1970-01-01 UTC 2147483647 seconds' Convert number of seconds since the epoch to a date
• TZ=':America/Los_Angeles' date What time is it on West coast of US (use tzselect to find TZ)
echo "mail -s 'get the train' P@draigBrady.com < /dev/null" | at 17:45 Email reminder
• echo "DISPLAY=$DISPLAY xmessage cooker" | at "NOW + 30 minutes" Popup reminder
locales
• printf "%'d\n" 1234 Print number with thousands grouping appropriate to locale
• BLOCK_SIZE=\'1 ls -l get ls to do thousands grouping appropriate to locale
• echo "I live in `locale territory`" Extract info from locale database
• LANG=en_IE.utf8 locale int_prefix Lookup locale info for specific country. See also ccodes
• locale | cut -d= -f1 | xargs locale -kc | less List fields available in locale database
recode (Obsoletes iconv, dos2unix, unix2dos)
• recode -l | less Show available conversions (aliases on each line)
recode windows-1252.. file_to_change.txt Windows "ansi" to local charset (auto does CRLF conversion)
recode utf-8/CRLF.. file_to_change.txt Windows utf8 to local charset
recode iso-8859-15..utf8 file_to_change.txt Latin9 (western europe) to utf8
recode ../b64 < file.txt > file.b64 Base64 encode
recode /qp.. < file.txt > file.qp Quoted printable decode
recode ..HTML < file.txt > file.html Text to HTML
• recode -lf windows-1252 | grep euro Lookup table of characters
• echo -n 0x80 | recode latin-9/x1..dump Show what a code represents in latin-9 charmap
• echo -n 0x20AC | recode ucs-2/x2..latin-9/x Show latin-9 encoding
• echo -n 0x20AC | recode ucs-2/x2..utf-8/x Show utf-8 encoding
CDs
gzip < /dev/cdrom > cdrom.iso.gz Save copy of data cdrom
mkisofs -V LABEL -r dir | gzip > cdrom.iso.gz Create cdrom image from contents of dir
mount -o loop cdrom.iso /mnt/dir Mount the cdrom image at /mnt/dir (read only)
cdrecord -v dev=/dev/cdrom blank=fast Clear a CDRW
gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom - Burn cdrom image (use dev=ATAPI -scanbus to confirm dev)
cdparanoia -B Rip audio tracks from CD to wav files in current dir
cdrecord -v dev=/dev/cdrom -audio *.wav Make audio CD from all wavs in current dir (see also cdrdao)
oggenc --tracknum='track' track.cdda.wav -o 'track.ogg' Make ogg file from wav file
disk space (See also FSlint)
• ls -lSr Show files by size, biggest last
• du -s * | sort -k1,1rn | head Show top disk users in current dir. See also dutop
• df -h Show free space on mounted filesystems
• df -i Show free inodes on mounted filesystems
• fdisk -l Show disks partitions sizes and types (run as root)
• rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' | sort -k1,1n List all packages by installed size (Bytes) on rpm distros
• dpkg-query -W -f='${Installed-Size;10}\t${Package}\n' | sort -k1,1n List all packages by installed size (KBytes) on deb distros
• dd bs=1 seek=2TB if=/dev/null of=ext3.test Create a large test file (taking no space). See also truncate
monitoring/debugging
• tail -f /var/log/messages Monitor messages in a log file
• strace -c ls >/dev/null Summarise/profile system calls made by command
• strace -f -e open ls >/dev/null List system calls made by command
• ltrace -f -e getenv ls >/dev/null List library calls made by command
• lsof -p $$ List paths that process id has open
• lsof ~ List processes that have specified path open
• tcpdump not port 22 Show network traffic except ssh. See also tcpdump_not_me
• ps -e -o pid,args --forest List processes in a hierarchy
• ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d' List processes by % cpu usage
• ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS List processes by mem usage. See also ps_mem.py
• ps -C firefox-bin -L -o pid,tid,pcpu,state List all threads for a particular process
• ps -p 1,2 List info for particular process IDs
• last reboot Show system reboot history
• free -m Show amount of (remaining) RAM (-m displays in MB)
• watch -n1 'cat /proc/interrupts' Watch changeable data continuously
system information (see also sysinfo) ('#' means root access is required)
• uname -a Show kernel version and system architecture
• head -n1 /etc/issue Show name and version of distribution
• cat /proc/partitions Show all partitions registered on the system
• grep MemTotal /proc/meminfo Show RAM total seen by the system
• grep "model name" /proc/cpuinfo Show CPU(s) info
• lspci -tv Show PCI info
• lsusb -tv Show USB info
• mount | column -t List mounted filesystems on the system (and align output)
# dmidecode -q | less Display SMBIOS/DMI information
# smartctl -A /dev/sda | grep Power_On_Hours How long has this disk (system) been powered on in total
# hdparm -i /dev/sda Show info about disk sda
# hdparm -tT /dev/sda Do a read speed test on disk sda
# badblocks -s /dev/sda Test for unreadable blocks on disk sda
interactive (see also linux keyboard shortcuts)
• readline Line editor used by bash, python, bc, gnuplot, ...
• screen Virtual terminals with detach capability, ...
• mc Powerful file manager that can browse rpm, tar, ftp, ssh, ...
• gnuplot Interactive/scriptable graphing
• links Web browser
miscellaneous
• alias hd='od -Ax -tx1z -v' Handy hexdump. (usage e.g.: • hd /proc/self/cmdline | less)
• alias realpath='readlink -f' Canonicalize path. (usage e.g.: • realpath ~/../$USER)
• set | grep $USER Search current environment
touch -c -t 0304050607 file Set file timestamp (YYMMDDhhmm)


####################################################################################

World of FIND Commands in LINUX

FIND COMMANDS

sudo find / -type f -name *.jpg -exec cp {} . \;

find . -type f -size +10000 -exec ls -al {} \;
find . -atime +1 -type f -exec mv {} TMP \; # mv files older then 1 day to dir TMP
find . -name "-F" -exec rm {} \; # a script error created a file called -F
find . -exec grep -i "vds admin" {} \;
find . \! -name "*.Z" -exec compress -f {} \;
find . -type f \! -name "*.Z" \! -name ".comment" -print | tee -a /tmp/list
find . -name *.ini
find . -exec chmod 775 {} \;
find . -user xuser1 -exec chown -R user2 {} \;
find . -name ebtcom*
find . -name mkbook
find . -exec grep PW0 {} \;
find . -exec grep -i "pw0" {} \;
find . -atime +6
find . -atime +6 -exec ll | more
find . -atime +6 -exec ll | more \;
find . -atime +6 -exec ll \;
find . -atime +6 -exec ls \;
find . -atime +30 -exec ls \;
find . -atime +30 -exec ls \; | wc -l
find . -name auth*
find . -exec grep -i plotme10 {};
find . -exec grep -i plotme10 {} \;
find . -ls -exec grep 'PLOT_FORMAT 22' {} \;
find . -print -exec grep 'PLOT_FORMAT 22' {} \;
find . -print -exec grep 'PLOT_FORMAT' {} \;
find . -print -exec grep 'PLOT_FORMAT' {} \;
find ./machbook -exec chown 184 {} \;
find . \! -name '*.Z' -exec compress {} \;
find . \! -name "*.Z" -exec compress -f {} \;
find /raid/03c/ecn -xdev -type f -print
find /raid/03c/ecn -xdev -path -type f -print
find / -name .ssh* -print | tee -a ssh-stuff
find . -name "*font*"
find . -name hpmcad*
find . -name *fnt*
find . -name hp_mcad* -print
find . -grep Pld {} \;
find . -exec grep Pld {} \;
find . -exec grep Pld {} \;
find . -exec grep PENWIDTH {} \; | more
find . -name config.pro
find . -name config.pro
find /raid -type d ".local_sd_customize" -print
find /raid -type d -name ".local_sd_customize" -print
find /raid -type d -name ".local_sd_customize" -ok cp /raid/04d/MCAD-apps/I_Custom/SD_custom/site_sd_customize/user_filer_project_dirs {} \;
find /raid -type d -name ".local_sd_customize" -exec cp /raid/04d/MCAD-apps/I_Custom/SD_custom/site_sd_customize/user_filer_project_dirs {} \;
find . -name xeroxrelease
find . -exec grep xeroxrelease {} \;
find . -name xeroxrelease
find . -name xeroxrelease* -print 2>/dev/null
find . -name "*release*" 2>/dev/null
find / -name "*xerox*" 2>/dev/null
find . -exec grep -i xeroxrelease {} \;
find . -print -exec grep -i xeroxrelease {} \;
find . -print -exec grep -i xeroxrelease {} \; > xeroxrel.lis
find . -exec grep -i xeroxrel {} \;
find . -print -exec grep -i xeroxrel {} \;
find . -print -exec grep -i xeroxrel {} \; | more
find /raid/03c/inwork -xdev -type f -print >> /raid/04d/user_scripts/prt_list.tmp
find . -exec grep '31.53' {} \;
find . -ls -exec grep "31/.53" {} \; > this.lis
find . -print -exec grep "31/.53" {} \; > this.lis
find . -print -exec grep 31.53 {} \; > this.lis
find . -exec grep -i pen {} /;
find . -exec grep -i pen {} \;
find . -print -exec grep -i pen {} \; | more
find . -exec grep -i pen {} \;
find . -atime +6 -exec ll | more \;
find . -atime +6 -exec ll \;
find . -atime +6 -exec ls \;
find . -atime +30 -exec ls \;
find . -atime +30 -exec ls \; | wc -l
find . \! -name '*.Z' -exec compress -f {} \;
find . -name 'cache*' -depth -exec rm {} \;
find . -name 'cache*' -depth -print | tee -a /tmp/cachefiles
find . -name 'cache[0-9][0-9]*' -depth -print | tee -a /tmp/cachefiles
find . -name 'hp_catfile' 'hp_catlock' -depth -print | tee -a /tmp/hp.cats
find . -name 'hp_catfile' -name 'hp_catlock' -depth -print | tee -a /tmp/hp.cats
find . -name 'hp_cat*' -depth -print | tee -a /tmp/hp.cats
find . -name 'hp_cat[fl]*' -depth -print | tee -a /tmp/hp.cats
find /raid -name 'hp_cat[fl]*' -depth -print
find . \! -name '*.Z' -exec compress -f {} \;
find . -name '*' -exec compress -f {} \;
find . -xdev -name "wshp1*" -print
find . -xdev -name "wagoneer*" -print
find . -name "xcmd" -depth -print
find /usr/contrib/src -name "xcmd" -depth -print
find /raid -type d -name ".local_sd_customize" -exec ls {} \;
find /raid -type d -name ".local_sd_customize" \
-exec cp /raid/04d/MCAD-apps/I_Custom/SD_custom/site_sd_customize/user_filer_project_dirs {} \;

Qmail Tip sand Tricks 2

Mail to Valid Users Is Bouncing or Disappearing

If you use users/assign as described in Chapter 15, a common mistake is to add a user to the system without updating the users file. Fortunately, this oversight is easily remedied:

# cd /var/qmail/users; make

Delivering Mail on Intermittent Connections

If your qmail system is a hub host for remote systems that connect intermittently by dialup, it is straightforward but messy to deliver the mail while the remote systems are connected.

One approach is to create a flag file in a known directory when a host connects and delete the file when the host disconnects. Then run a script periodically from cron that loops over all of the flag files to push out mail to currently connected hosts.

To flesh out this example, assume there are three dialup hosts called red.example.com, blue.example.com, and green.example.com. Create virtualdomains that give them different virtual domain prefixes:

red.example.com:alias-dial-red
blue.example.com:alias-dial-blue
green.example.com:alias-dial-green

You can put all of the alias-dial mail into one Maildir since the Delivered-To: prefixes keep them separate. To put all the mail for the three hosts into ~alias/dialmail/, create ~alias/.qmail-dial-default containing the line ./dialmail/.

To track the currently connected hosts, put the flag files into ~alias/dialflags and have the dialup connection script create a file with the host's simple name (red, blue, or green) in that directory containing the host's current IP address. Then run this script from cron to push out the mail to whichever hosts are currently connected:

#!/bin/sh
# run this every 15 minutes from cron to push out the mail

cd /var/qmail/alias/dialflags

for hn in *
do
ip=$(cat $hn) # IP address in the flag file

setlock ../$hn.lock \ # lock deliveries to this host
maildirsmtp /var/qmail/alias/dialmail \
alias-dial-$hn- $ip my.example.com 2>&1 |
splogger serial
done

If you also want to push out any waiting mail as soon as a host connects, also put a call to maildirsmtp into the host's connection script. Be sure to use the same lock file to avoid confusion if the cron job happens to run at the same time. If you add another host called purple, you only need to add another line to virtualdomains:

purple.example.com:alias-dial-purple

The remote hosts can use a similar setup to forward their mail to the main host, using a single smarthost entry in virtualdomains.

Code in extra.h to copy everything to log

#define QUEUE_EXTRA "Tlog\0"
#define QUEUE_EXTRALEN 5

Now every message will be copied to the address log, so you can create ~alias/.qmail-log to save the mail:

./logmaildir/

The .qmail file must save the mail but cannot forward it. Why not? Because forwarding mail invokes qmail-queue again, which will redeliver the mail to log, creating a nasty mail loop.

Deleting Stale Mail

cd /home
{
# unread mail over a month old
find /home/*/Maildir/new -type f -mtime +30 -p
# read mail over three months
find /home/*/Maildir/cur -type f -mtime +90 -p
# any mail marked deleted
find */Maildir -type f -name "*:2,*T*" -print
# any mail in Trash/new or cur
find */Maildir/.Trash/??? -type f -print
} | xargs -t rm

Web Sites

There are several excellent sources of qmail information online.


http://cr.yp.to

Dan Bernstein's web site, the official source for qmail and all of his ad-on packages.

http://www.qmail.org

Russ Nelson's qmail resource site, intended to have links to all of the other resources on the Web.

http://qmail.gurus.com

The author's companion site for this book, containing scripts, updates and corrections, links to other resources, and ordering info for more copies.

http://www.lifewithqmail.org

Dave Sill's Life with qmail, an online guide to setting up and using qmail. It offers specific advice about where to install qmail, and where to put all of the files and directories that qmail needs. This is by far the most widely used setup and the one that qmail experts are the most familiar with, so it's the one you should use. The file and directory locations used in this book are consistent with these.

http://www.lifewithqmail.org/ldap/

Henning Brauer's Life with qmail-ldap, a guide to setting up qmail-ldap. Indispensable for qmail-ldap users.

http://www.ezmlm.org

The home page for the ezmlm-idx mailing list manager, with software and documentation.

http://tinydns.org

Russ Nelson's site for Dan Bernstein's djbdns, a DNS package that relates to BIND roughly as qmail relates to sendmail. Not required for qmail, but if you're setting up a DNS server along with your mail server, it's probably the software you want to use.

Qmail Tips and Tricks

Qmail Won't Compile

You have unpacked the qmail sources and typed make, but it won't compile. If you're receiving error messages about errno, you've run into a compatibility problem between qmail and recent versions of the GNU C library. The fix is very simple. See Building with Recent GLIBC and Fixing the errno Problem in Chapter 3.

(This is the number one question on the qmail mailing list, so frequent that there's an autoresponder that mails back the answer to any message that contains the word "errno".)

Why Qmail Is Delivering Mail Very Slowly

If qmail seems to wait about half a minute to do anything when you inject mail, the problem is almost certainly that the lock/trigger file used to communicate between qmail-queue and qmail-send is messed up. That file should be a named pipe:

# ls -l /var/qmail/queue/lock/trigger
prw--w--w- 1 qmails qmail 0 Nov 7 03:02 /var/qmail/queue/lock/trigger

If it's a regular file or anything other than a pipe, you have a problem. Fortunately, it's a problem that's easy to fix:

# svc -td /service/qmail-send # shut qmail down for a minute
# tail -f /service/qmail-send/log/main/current
# # wait until the log says that it's exited
# rm /var/qmail/queue/lock/trigger # remove bogus trigger
# cd wherever you built qmail from source
# make setup check # recreates all the crucial files including trigger
# svc -u /service/qmail-send # restart qmail

This is the second most frequently asked question on the qmail mailing list, and tends to get aggrieved responses pointing out that the answer is in the archives about a hundred times. So don't ask it, because now you know the answer.

Daemons Won't Start, or They Start and Crash Every Few Seconds

Starting a daemon under svscan and supervise is simple in concept, although the details can bite you. The super-daemon is started at system boot time by running /command/svscanboot. It runs svscan to control daemons and the useful but obscure readproctitle, which takes any error messages from svscan and puts them into its command area so that ps will show it.[1]

[1] This odd way of displaying error messages is intended to work even in the presence of serious configuration screwups like disks that should be mounted but aren't and directories that are supposed to be writable but aren't.

Every five seconds svscan looks at all of the subdirectories of /service and starts up a supervise process on any that don't have one running. In the usual case that the subdirectory in turn has a subdirectory called log, it starts a second supervise process in the subdirectory and pipes the output from the first process to the second.

When supervise starts up a daemon, it runs the file run in the daemon's directory. That file has to be a runnable program that either is or, more commonly, exec's the daemon itself. That means that run has to have its execute bits set and, if it's a shell script, start with #!/bin/sh so that it's runnable. If either of those isn't the case, there is a failed attempt to start the daemon every five seconds. A ps l that shows readproctitle should reveal the error messages and give hints about what needs to be fixed.

The run script generally sets up the program environment and then exec's the actual daemon. If you become super-user and type ./run, the daemon should start. If that works, the daemon still doesn't start, and you don't use full program paths in the run file, the problem is most likely that the search path that supervise uses isn't the same as the one you're using. Look at /command/svscanboot to see the search patch that it uses. Most notably, it does not include /var/qmail/bin unless you edit the file yourself to include it.

Nothing Gets Logged

Sometimes the daemon runs but nothing's going into the log files. This generally is due to either file protection problems or an incorrect set of multilog options. The usual way to run multilog is to create a subdirectory called main in which it rotates log files. It's safer to run daemons as a user other than root, so when possible, use qmaill, the qmail log user. A common error is to forget to change the ownership of the log file directory to qmaill (or whatever the log user is). When multilog starts successfully, it creates a current log file in the directory, so if there's no main/current, the most likely problem is directory ownership or protection.

If multilog is running but there's nothing logged, the most likely problems are that the daemon isn't sending anything to log, or that multilog's options are telling it to discard everything. Because the daemon and the logger are connected with a regular Unix pipe, only messages sent to the daemon's standard output go to the logger. In particular, anything sent to standard error shows up in readproctitle, not the log. If, as is usually the case, you want to log the errors a daemon reports, just redirect the error output to the standard output in the run script with the standard shell redirect 2>&1. (That redirect is at the end of just about every run script example in this book.)

If the daemon is a program originally intended to run as a standalone daemon rather than under daemontools, it probably sends its reports to syslog, not to standard output or standard error. In most cases, there is an option to send messages to stdout or stderr.

If you are using multilog options to select what to log, be sure that you're selecting what you think you are. In particular, its pattern language resembles shell wildcards but is in fact considerably weaker because it doesn't move ahead or back up on a failed match. (Patterns do resemble shell wildcards closely enough that they should always be quoted to keep the shell from messing with them.) The pattern must match the whole line, and stars stop matching the moment they see the following character in the pattern. If a pattern is, say, +'+*: status: *', it will match one: status: two, but it will not match one: two: status: three, because the star will stop at the first colon and won't look for the second one. If the pattern didn't have the star at the end, it wouldn't match anything useful because it wouldn't match any lines with anything after the status:. In practice, most log file messages have a pretty simple syntax, and it's not hard to come up with adequate patterns if you keep in mind the limitations of the pattern-matching language. For debugging, start with no patterns to be sure that the stream of messages going into the log files contains what you expect, then add one or two patterns at a time and restart multilog with svc -t and see what's going into main/current each time until it looks right.

Daemons Are Running but Making No Progress

One of the most baffling problems occurs when the daemon seems OK, the logger seems OK, but the daemon's not doing anything. What's wrong? Usually the problem is that the disk to which the log files are written has filled up or is mounted read-only. Because multilog is designed not to lose any log data, if it can't write to the disk, it just waits and retries until it can. This means that the pipe between the daemon and multilog fills up and the daemon stalls waiting to be able to write to the pipe. The solution is to delete some files and fix whatever it was that filled up the disk so it doesn't happen again. If the disk is full of files written by various multilog loggers, adding or adjusting s and n options to set the maximum size and number of log files can help.

Mail Rejected with Stray Newline Reports

The SMTP spec says that the way that each line of text in an SMTP session ends is with a carriage return/line feed pair (0d 0a in hex or \r\n in C.) Some buggy MUAs and MTAs only try to send mail that contains linefeeds with no preceding carriage return. Qmail's SMTP daemon normally rejects such mail with a log message like Stray newline from 10.2.3.4 because there's no way to tell whether the bare linefeed is just missing a carriage return or it's some kind of malformed binary data.

If you're seeing stray newline entries in your logs and you're reasonably sure that they're being sent by MTAs or MUAs that intend them to be handled as an end-of-line, use the fixcrio program from the ucspi-tcp package to placate the SMTP daemon. Modify the run script for qmail-smtpd so that it pipes mail through fixcrio, as shown in Example 18-1:

Example SMTP daemon that forgives stray newlines

1. #!/bin/sh
2. limit datasize 3m
3. exec tcpserver \
4. -u000 -g000 -v -p -R \
5. 0 25 \
6. /usr/local/bin/fixcrio | /var/qmail/bin/qmail-smtpd" 2>&1

Line 6 is the modified one, starting up fixcrio and qmail-smtpd. When fixcrio runs, it passes the input and output of qmail-smtpd through pipes so it can add missing carriage returns in front of newlines as needed. In the longer run, see if you can persuade your correspondents to upgrade their SMTP clients to newer, less buggy versions.