Patrick Koetter, Ralf Hildebrandt, The Book of Postfix

Postfix is fast becoming a popular alternative to sendmail. Although it can be complex to configure, it’s easier to use Postfix with additional filtering applications, for example Spam and virus filters, than with some other mail transfer agents. I spoke to Patrick Koetter and Ralk Hildebrandt about The Book of Postfix, the complexities of configuring Postfix, Spam, and email security.

The Book of PostfixHow does Postfix compare to sendmail and qmail?

Ralf Hildebrandt (RH): As opposed to sendmail, Postfix was built with security in mind.

As opposed to qmail, Postfix was built for real-life systems in mind that have to adapt to the hardships of the Internet today. qmail is effectively unmaintained.

Patrick Koetter (PK): That’s a tough question because I am not one of those postmasters who spent half their life working with Eric Allman’s Sendmail nor did I spent too much time enlarging my knowledge on qmail, so I can’t give you an in detail answer that will really tackle specific features or functionalities.

Let me give it a different spin and try if that answers it:

When I took out to run my first own mailserver I looked at Sendmail, qmail and Postfix.

Sendmail to me was too complicated to configure and since my knowledge of the M4 macro language was very little, but my fear of loosing e-mail or even configuring my server to be an open relay was large I dropped it. The ongoing rally of CERT Advisories about this or that Sendmail exploit by then didn’t make it a hard choice.

Then I took a look at qmail, but wasn’t really sure I wanted it because it is more or less a series of patches if you want to use it with nowadays feature range. But I gave it a try anyway and ended up asking some questions on the mailing list because the documentation would not answer what I was looking for.

To cut it short: I was under the impression you had to enter the “Church of qmail” before anyone would take the time to answer a question to a qmail novice. It might have changed since then, but back then I left and I never looked back because all I wanted was to run a MTA.

Finally I took a look at Postfix and was very surprised by the amount of documentation that was available. I also immediately fell in love with the configuration syntax, which seemed to simple and clear to me. For a while I thought this must be a very feature limited MTA, but the more I read the more I understood that it did almost the same things, but was simply easier to configure.

I finally decided to stick with Postfix after I had joined the Postfix mailing list and found out that people really cared for my questions, pointed me to documentation to read again or give me advice on how to do this or that more efficient.

Of course, as the Postfix community grew larger, one or the other character turned up who would rather lecture someone seeking help, but the overall impression still remains the same.

Postfix is well maintained, its security record is unbeaten up to now and the community is how I wished every community supporting a software should be. The modular software architecture Wietse Venema has chosen makes it easy to expand Postfix’ capabilities. Its a system that can grow very well. I haven’t seen another piece of software that does the complex job of being a MTA that well.

Postfix seems a little complex to install - there are quite a few configuration files, some of which seem to contain arcane magic to get things working. Is this a downside to the application?

PK: That’s the provoking question, isn’t it? ;)

To me Postfix is as simple or complex as the process of mail transport itself is. That’s why we added so many theory chapters to the book that explain the e-mail handling process before we took out to explain how Postfix does it in the follow-up chapter. If you understand the process its pretty straightforward to configure Postfix to deal with it.

But basically all you need is three files, main.cf, master.cf and the aliases file. Wait! You could even remove the main.cf file and Postfix would work with reasonable defaults on this specific server.

The main.cf file carries all parameters that are applied globally. If you need options that are specific to a special daemon and should override global options from main.cf, you add them in master.cf in the context of that special daemon. That’s the basic idea of configuring Postfix.

Then there is a lot of tables in the /etc/postfix directory, which you usually don’t need unless you take out or configure a specific feature that isn’t part of basic functionality.

Sure, the amount of tables might frighten a novice, but then they are there for the sole purpose of supporting a novice and even advanced users because they hold the documentation about what the specific table is about and how you would add entries to the table if you wanted to use it.

The rest is complexity added by additional software, for example Cyrus SASL which is a royal pain for beginners.

Of course your mileage will vary when you take out to configure a full blown MTA that incorporates Anti-Spam measures, Anti-Virus checking, SMTP Authentication and Transport Layer Security, where Postfix looks up recipient names and other information from an LDAP server that also drives an IMAP MTA.

But when you begin it boils down to the two configuration files and an aliases file.

As for the “arcane magic” I don’t exactly understand what you relate to so I do some speculation based on my own experiences.

I struggled with smtpd_*_restrictions for quite a while until I realized: “Its the mail transport process that makes it so hard to understand.” Once you’ve understood how a SMTP dialog should be processed it suddenly seems very simple. This is at least what happened to me. I recall hours sitting in front of these restrictions, Ralf ripping hair out of his head and looking at me as if I was from another planet.

The quote we used in the restrictions chapter alludes to that day and it also contains the answer I came up with: “To know what to restrict you need to know what what is.” I looked the “what” parts up in the RFCs, understood what smtpd_*_restrictions were all about and saved Ralf from going mad ;)

But that’s specific to smtpd_*_restrictions. For all other parameters and options it pays to read RFCs as well, but you can get very far by reading the excellent documentation Wietse has written _and_ by looking at the mere names he used for the parameters. Most of the time they speak for themselves and tell you what they will do. I think Wietse has done a great job at thinking of catchy self-explanatory parameter names.

RH: Postfix works with the default main.cf and master.cf. If you have advanced requirements, the configuration can get elaborate. But configuration files like I created them and also offer them on http://www.stahl.bau.tu-bs.de/~hildeb/postfix/ have evolved over several years of use (and abuse of the Internet by Spammers) - I never thought “That’s the way to do it”, but it was rather “trial and error”.

Postfix seems to work exceptionally well as a mail transport agent - i.e. one that operates as an intermediate relay or relayhost (I’ve just set up a Postfix relay that filters spam and viruses, but ultimately delivers to a sendmail host, for example). Is this because of the flexible external interface Postfix uses?

RH: It also works excellent as a mailbox host :) Over the years, Wietse added features for content filtering and the ability to specify maps that tell the system which recipient addresses should be accepted and send on further inwards.

That makes it easy to say “Instead of throwing away our old Product-X server, we simply wedge Postfix in between”

But there’s no special preference as “intermediate relay”. It’s an universal MTA. We use it everywhere. Also for the server handling the mailboxes. Or our list exploder.

Do you have a preferred deployment platform for postfix?

PK: Basically I go for any platform that suits the needs. As for Linux I prefer distributions that don’t patch Postfix, but that’s only because I support many people on SMTP AUTH issues on the Postfix mailing list and some maintainers have taken to do this or that different, which makes configuring SMTP AUTH even harder.

Personally I’d go for RedHat Linux because I know it the best and produce good results faster as on other platforms. But then I wouldn’t hesitate a second to go for something else if it suits the scenario better. That’s another side of Postfix I like very much: It runs on many many systems.

RH: Debian GNU/Linux with Kernel 2.6.x. Patrick begs to differ on the Debian thing. Anyway, it works on any Unixoid OS. I ran it on Solaris and HP-UX back in the old days.

You cover the performance aspects of Postfix. Is it particularly taxing on hardware?

PK: That’s a question that turns up regularly on the Postfix mailing list. Read the archives… ;)

But seriously, you can run Postfix for a single domain on almost any old hardware that flies around. If your OS works with the hardware Postfix will probably go along with it as well.

The more domains you add the more mail you put through the likelier of course that you will get to the limits. But those limits usually aren’t limits imposed by Postfix, but by the I/O performance of your hardware.

Think of it this way: Mail Transport is about writing, moving and copying little files in the filesystem of your computer. The MTA receives a mail from a client and writes it to a mail queue where it waits for further processing. A scheduler determines the next job for the file and the message is moved to another queue. There it might wait another while until it gets picked up again to be delivered to another, maybe remote destination. If the remote server is unreachable at the moment it will be written back to the filesystem again to another queue and so an and so on until it finally can be removed after successful delivery.

The calculation to decide what to do with the mail doesn’t take a lot of time, but writing, moving and copying the file takes a lot longer. That’s due to the limitations of hardware. Hard discs nowadays really can store a lot of e-mail away, but the access speed didn’t grow at the same time. Still you need to stick to them because storing the message in a temporary device would lose the mail if the system was turned off suddenly.

So the basic rule is to get fast discs, arrays and controllers when you need to handle _a lot_ email. Regular hardware does it for private users quite well.

Another slowdown you should be prepared to expect is when you integrate Anti-Spam and Anti-Virus measures. They do not only require to read and write the files they also examine the content which often requires to unpack attached archives. This will temporary eat some of your CPU. But that’s something current hardware can deal with as well.

For hard facts you will need to find somebody who is willing to come up with a real world and well documented test scenario. So far one or the other has posted “measurement data”, but none of them would really tell about their setup and how they tested. Also I don’t know about a sophisticated comparison of Sendmail, qmail and Postfix.

Most of the “comparisons” I’ve heard weren’t able get rid of the odor of “because you wanted it to be better”.

Such tests are not what Postfix is and, as far as I can say without asking him, isn’t Wietse Venema about. I vividly recall him posting “Stop speculating, start measuring!” to someone who came up with a performance problem. I like that attitude a lot, because comparisons should be about facts and not believe.

I enjoyed the in-depth coverage on using certificate based security for authenticating communication between clients and servers. Do you see this as a vital step in the deployment process?

PK: Vital or not depends on your requirements and your in-house policy. Personally I do like certificate based relaying a lot and I think it should be used more widely, because you could really track spam a lot better down and would gain a more secure mail transport at the same time, but then certificate based relaying simply lacks the critical mass of servers and clients supporting it.

As long as you don’t have the critical mass of servers and clients using it there will always be a relay that does it without and that can be tricked to relay spam one or the other way and you loose track of the sender.

It also takes more work to configure, but especially maintain certificate based relaying because you need to maintain the list of certificates. You need to remove the ones that are expired, add others, hand out new ones, this and that…

I think its a “good thing to do [TM]” if you use it in your company, have many mobile users, but most of all (!) have all clients and serves under your control. Then you can automatize some of the work that needs to be done and all that together can pay up for the security and simplicity you get on your network.

But I doubt any private user would be willing to pay the additional fee for maintenance not to mention the certificate infrastructure to maintain the certificates themselves.

Was it Yahoo who had some certificate based Anti-spam measure on their mind? So many attempts to fix the effects of Spam… I think what we really need is a redesign of SMTP to cope with the current challenges. But that’s another topic and I am certainly not the one to be asked how it should be done. ;)

Is it better to use files or MySQL for the control tables in Postfix?

RH: “He said Jehova!”

Performance-wise mysql just sucks. The latency for queries is way higher than when asking a file based map. But then with mysql maps, any changes to the map become effective immediately, without the daemons that use the map having to exit and restart again. If your maps change often AND you get a lot of mail: mysql In all other cases: file based maps.

And: Keep it simple! If you don’t NEED mysql, why use it?

PK: I don’t think there’s a better or worse, because either way you loose or you gain something, but what you loose and gain aren’t the same things:

From a performance point of view you loose a lot of time when you use SQL or LDAP databases because of their higher lookup latency so you might want to stick with the files.

But then, if you host many domains, you win a lot when you maintain the data in a database. You can delegate many administrational tasks to the end user who accesses such a database through some web frontend. So there’s the pro for databases.

If you need both, performance and maintainability, you can build a chain from databases and files. The editing is done in the database and job on your computer checks the database on a regular base and builds (new) files from it when the data has changed. This way you get the best of both worlds for the price of a little delay after changes had been done in the database.

IMAP or POP?

PK: An old couple sits in the kitchen at home.

She: “Let’s go to the movies.”
He: “But we have been to the movies just recently…”
She: “Yes, but they show movies in colour AND with sound now!”

Definitely IMAP ;)

RH: Depends on your needs. Let the user decide: go for courier-imap (which also does pop), so the user can choose.

Is there a simple solution to the spam problem?

RH: Mind control? Orbital lasers? No, but Postfix’s restrictions and the possibility of delegating policy decisions to external programs can help.

PK: No, unfortunately not. There are too many reasons why Spam works and a working solution would have to be technical, political and business oriented at the same time.

First of all it works because the SMTP protocol as designed has little to no means to prove that a message was really sent by the sender given in the e-mail. Anybody can claim to be anybody. As long as this design problem persists it will cost a fortune to track spammers down.

Even if you know where the spam came from the spammer might have redrawn to a country that don’t mind spammers and will protect them from being pursued by foreign law.

The world simply lacks anti-spam laws all countries agree on. You typically are forced to end your chase for a spammer the moment you pass another countries borders because you are not entitled to chase the suspect.

Still, if you where entitled to do so, if costs a fortune to track a spammer down and even then it might take ages to get some money for the damage they have done. Is your company willing to pay that much just to nail one spammer down when another two emerge the moment the one goes behind bars?

And then Spam works, because it is so cheap. You buy a hundred thousand addresses for 250 bucks or even less and IIRC Yahoo found out that 1/3 of their mail users read spam and VISIT the pages they promote.

If one wants to make it go away one must make it expensive for those that send or endorse spam. If you ruin the business concept no one will send spam. That’s business… ;)

To sum my position up: The problem is global and we don’t have the right tools to hinder the cause. Currently all we can do is diminish the effect, by using as many anti-spam features as we can think of.

Do either of you have a favourite comic book hero?

PK: The “Tasmanian Devil” is my all time favourite. I even have a little plastic figure sitting in front of me under my monitor, which has become some kind of talisman. It reminds me to smile about myself on days where I’d rather go out and kill somebody else for not being the way I would want them to be ;)

RH: Calvin (of Calvin and Hobbes)
or
Too much Coffee Man!

Author Bios
Ralf Hildebrandt and Patrick Koetter are active and well-known figures in the Postfix community. Hildebrandt is a systems engineer for T-NetPro, a German telecommunications company, and Koetter runs his own company consulting and developing corporate communication for customers in Europe and Africa. Both have spoken about Postfix at industry conferences and contribute regularly to a number of open source mailing lists.