It's a well-known fact that I absolutely adore lighttpd. Why, you ask?

Continued

Note: This post is from many years ago, and I no longer operate an IRC network. No support is offered for this module.

Continued

Yes I know I still haven't done part two of that mail server post. I'll get it done soon, I promise.

While chatting on IRC, someone mentioned that they were having a problem with a process going mental and creating a bunch of file descriptors in linux, eventually hitting the "max FD limit" linux has. They couldn't figure out which process it was and they couldn't find a program that would list a count of how many FDs a process has open. A few minutes later I'd thrown together this bash one-liner for him. I'm posting it here just in case someone else might find it useful.

echo "$(for pid in $(ls -a /proc|egrep '^([0-9])*$'|sort -n 2>/dev/null); do if [ -e /proc/$pid/fd ]; then FHC=$(ls -l /proc/$pid/fd|wc -l); if [ $FHC -gt 0 ]; then PNAME="$(cat /proc/$pid/comm)"; echo "$FHC files opened by $pid ($PNAME)"; fi; fi; done)"|sort -r -n|head -n4

To explain: It loops through every file/folder in /proc that is a process ID, then checks that there's a file descriptor folder. Then it gets a count of all the FDs that process currently holds, gets the process name and outputs how many file descriptors that process has open, as well as the process name. This is then reverse-sorted and cut down to only the four processes with the most FDs open.

So a few months ago, I moved my primary mail hosting to my own VPS. Over the months since then, I've been tweaking and adding to my mail system, and I figure it'd help both myself and others if I documented what I've done, so I'll start with a list of all the software I use.

Main Software

• Gentoo - My VPS runs Gentoo. I personally prefer it over other distros, as it's both lighter and less screwed up.
• postfix - I was originally going to use Exim, but found it strangely difficult to configure, plus postfix seems to universally have lower transaction times.
• dovecot 2 - I'm switching away from Gmail, so obviously the things I would've missed most would be things like push email and mail filters. Dovecot supports IMAP IDLE, and has sieve/managesieve, so it was easy to port my Gmail filters over.
• saslauthd - While dovecot has its own SASL authentication, I prefer to use this when authenticating over SMTP. EDIT: I have since switched from saslauthd to Dovecot 2 for SASL authentication. Dovecot works well enough for it that I questioned why I actually needed saslauthd.
• Mutt - Mutt is my primary MUA. I'll also be discussing configuration changes I made to Mutt, and ways I made it work more like Gmail.

Extra Software

• SpamAssassin - This should be obvious. Does a great deal to cut down on spam. Plus, with a filter set up with dovecot's sieve, I have a spam folder like before. EDIT: When I first published this post, I had only just set up Postgrey and had no idea what kind of impact it would have on incoming spam/junk mail. It had a massive impact – I haven't received a single spam email since. Spamassassin and Amavisd may actually be unnecessary when using Postgrey (Unless of course you plan to host mail for others, or if you receive a *lot* of spam).
• Postgrey - This does a fantastic job of cutting down on spam.
• Amavisd - Somewhat necessary for making postfix work with SpamAssassin, but also makes it easy to offload the antispam part of the mail system to another server. Amavisd can also be used for integrating antivirus systems into your mail scanning process, but I don't need that.
• OpenDKIM - Used for signing outgoing mail with my DKIM key, and for validating incoming signed mail. This does a good job of ensuring that mail sent from my domain is actually coming from one of my servers.
• policyd-spf - Originally, I used pypolicyd-spf, but it quite literally breaks every time there's an update to python, it's since been replaced with this perl equivalent which has never had any issues. This uses SPF to validate incoming mail, and ensure that the sending server is actually authorised to send mail for the given domain.
• fail2ban - This isn't strictly part of the process I go through when setting mail up, but fail2ban helps cut down load a lot when bots are trying (and failing) to use a server as an unauthenticated relay.

In the near future, I'll write a second post detailing how I linked all this together, including config excerpts, but in this post I'm just discussing the software I used, as well as why I use each package. I'll also leave you with a list of extremely good tips.

• Get an SSL certificate. This makes setting secure mail up a lot easier, especially if you plan to send or receive mail remotely with stuff like IMAP.
• If you do get an SSL certificate, disable or firewall unencrypted mail ports. Obviously leave port 25 in place, but if you're sending or receiving mail remotely, disable the unencrypted IMAP/POP3 ports (143 for IMAP and 110 for POP3), and set your MTA up to only accept submission mail through 465.
• Set up SPF records for your domain appropriately. SPF does a good job of telling other mail systems who is or is not allowed to send mail for your domain.
• Generate a DKIM key, and add it to your domain's DNS. As with SPF, DKIM (DomainKeys Identified Mail) does a fantastic job of indicating to other mail systems whether an email is actually legitimate or not.
• Use blacklists. There's a large number of DNS-based blacklists which indicate whether a given IP address is known for sending spam or for attempting to compromise servers. This can go a long way in preventing spam.
• Report any spam you receive. Reporting received spam to places like SpamCop not only reduces the chance of you receiving similar spam in the future, but it helps others too. It helps identify servers that send spam (Contributing to blacklists), helps identify possible domains used for spam (again, contributing to blacklists), and can contribute to the accuracy of antispam systems like SpamAssassin.
• Monitor your services extensively. This is definitely a big one. It's not easy to monitor your server by looking at logs, and often unless you've got systems set up to email you when anything out of the ordinary happens, you just plain don't know what's happening with your server. Packages like Monitorix (disclaimer: I'm the package maintainer for monitorix on Gentoo), do a fantastic job of showing you at-a-glance whether anything abnormal is happening, so it's easy to see if and when your mail server is rejecting mail. This can also be great for indicating when you've misconfigured something.
• Use external monitoring services. Services like MXToolbox have free accounts, and you can use them to set up checks so that you get an email if your server's IP is on any IP blacklists. Services like Pingdom are also great for monitoring both uptime and external availability.
• Make sure your forward and reverse DNS match and that your reverse DNS is your primary mail domain (or what your server actually identifies itself as). This is definitely a good way of ensuring your mail isn't identified as spam.

One final thing to note, the guide and so on will discuss my current mail setup. This means it assumes you'll be using sockets for things like Postgrey. Please read everything carefully before making configuration changes to your own mail setup, as what works for me may not work for you.

That's all for now, but I'll be adding to this list, and writing a second post documenting how I actually set my mail system up, very soon. EDIT: Disregard that. Second part of this post will arrive eventually but I am tremendously lazy.

Since I figure it's bad form to have a blog and /only/ use it for ranting, here's a somewhat useful post. I've been with [a][1] [number][2] [of][3] [different][4] [hosts][5] over the past year or two, and I figure it'd be useful for others to know why I like or don't like them.

[SimplexWebs][6]

SimplexWebs has been in the hosting business for quite a while now, and do enterprise webhosting, online radio hosting and domains, as well as VPS (Xen, powered by OnApp) hosting. I was lucky enough to grab one of their limited birthday sale servers, which gave you 256MB RAM, decent CPU speed, 20GB disk and 100GB monthly bandwidth, for £25 a year.

This server's basically been my primary server/main workhorse - despite initially purchasing it to run a VPN server, and has held up ridiculously well. Uptime has been fantastic - my server's only been down a few times over the last half a year or so, including migrations between their old SolusVM platform to their new OnApp platform, and including the recent downtime when they moved datacentres.

Server and network speed are really good. I think this is one of the fastest servers I've ever used, and that includes the brief time I was with Linode. Support is fantastic, their team is always quick to respond and very helpful, and you definitely get the feeling that they actually care about their customers. Overall, they're one of the best hosts I've ever been with, and I don't feel I'll ever need to move to another host.

Here's some benchmarks:

Network speed:

~ wget cachefly.cachefly.net/100mb.test -O /dev/null
--2012-04-28 02:34:32--  http://cachefly.cachefly.net/100mb.test
Resolving cachefly.cachefly.net... 140.99.94.175
Connecting to cachefly.cachefly.net|140.99.94.175|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `/dev/null'

100%[======>] 104,857,600 11.8M/s   in 8.7s

2012-04-28 02:34:41 (11.5 MB/s) - `/dev/null' saved [104857600/104857600]

Disk I/O:

~ dd if=/dev/zero of=/tmp/disktest bs=64k count=16k
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 4.52632 s, 237 MB/s

[Bhost][7]

Bhost are a UK-centric VPS provider, primarily dealing with OpenVZ servers although they've recently launched Xen PV plans. I should mention their servers are some of the cheapest you can get, the cheapest OpenVZ server being £4.70 (before VAT) and including 512MB RAM and 512MB burst RAM.

Their support is pretty great, considering they're a budget host, and the servers themselves are quite speedy. I do have some issues with the OS templates they have available - their Arch images are broken due to the host kernel being too old, although I'd say that's an Arch issue, and their Gentoo image is too old to actually use. Debian works fine though, so that's what I'm using. Overall, network and server speed is really good. Uptime is fantastic, in the entire time I've been with them (a number of months now), my server's almost never been down. Definitely one of the most reliable hosts I've been with.

Here's some benchmarks, since I still have a server with them: Network speed:

# wget cachefly.cachefly.net/100mb.test -O /dev/null
--2012-04-28 02:30:40--  http://cachefly.cachefly.net/100mb.test
Resolving cachefly.cachefly.net... 205.234.175.175
Connecting to cachefly.cachefly.net|205.234.175.175|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `/dev/null'

100%[======>] 104,857,600 10.2M/s   in 9.8s

2012-04-28 02:30:49 (10.2 MB/s) - `/dev/null' saved [104857600/104857600]