It is the administration task we love to hate: securing a website. Apache forms the backbone of most websites so it makes sense to start there. In Hardening Apache, Tony Mobily does just that, starting with the basics of creating of a secure Apache installation and moving on to more in depth techniques for securing Apache installations from attack. Let’s see what Tony has to say when I talk to him about his new book and how to approach security, Apache and otherwise.
One of the key elements I get from your book is the back to basics approach. For example, I know a lot of companies with extensive login systems that leave their server room doors wide open. Do you it’s best to work from the inside out or the outside in when setting up security?
I believe that you always need to get the right person for the job. For example, if you need to re-tile your bathroom, you don’t call a wood worker. It’s the same with computer security; "physical" security (e.g. preventing people from breaking in) and "logical" security (preventing crackers and script kiddies from using your servers and resources) are very different things which require very different skills and training.
In this field - in fact, in any field - improvisation is just not an option.
If a company asked me to secure their physical network, I would redirect them to Steve, a friend of mine who does just that. Steve tells me amazing stories of sniffing packets by just placing a device next to the cable, for example, or other stories which I would see nicely in a James Bond movie rather than real life.
Even "logical" security branches out! I wouldn’t be able to audit the source code of a complex program, for example.
The problem is that even though improvisation shouldn’t be an option, it still happens. When a manager installs updates on a Unix system, or (worse) a service pack on a Windows machine, he is improvising and putting his systems at risk - full stop.
To go back to the question, security is a problem that has to be faced as a whole. To connect to the example I made earlier, a good physical design will prevent problems such as random people getting to close to a network cable and sniffing packets, or people accessing the servers’ consoles. On the other hand, a good logical design will mean that any piece of information will be encrypted, and if intruders did manage to access the cable, they won’t be able to do anything with the collected information.
Apache 1.3.x or Apache 2.x?
For me, there is no doubt: Apache 2.x.
It’s not just a matter of wanting to use the latest piece of software at any cost.
The problem with security is that often you are tied to the Apache version you are using. For example, if you use Apache 1.3.x for long enough on a complex web site, eventually you will be using a number of modules which are only available on Apache 1.3.x. In this common situation, upgrading to Apache 2.x can be really hard and might even require redesigning some parts of your web sites in order to use different technologies. The longer you leave it, the harder it will be to actually upgrade.
The problem is that eventually, you will have to upgrade because the 1.3.x branch of Apache will no longer be supported and patched anymore. It might not be soon, but it will happen. A lazy system administrator, at that point, will find himself (or herself) with an unpatchable system and, what’s worse, he or she won’t be able to upgrade without majorly disrupting the hosted web sites.
You make good use of the warnings and notifications made by sites like CVE and ApacheWeek. Are these sites that Apache administrators should be checking regularly?
Yes, absolutely.
Checking sites like ApacheWeek is both necessary and boring. I think there is also fear - sometimes you are just about to go on holiday or go home, and you discover that your production server has a security hole as big as a crater, and you urgently need to recompile the whole thing!
These sites are crucial to make sure that system administrators don’t live in their own "little world", and can realize that software is not just something they install on their computers and it works; software changes, evolves, improves, stumble across problems, and so on.
You use a lot of sample exploits to demonstrate weaknesses. Is it worth creating a tool-kit for checking these exploits against your site?
Writing such a tool-kit is a good idea in theory. In practice, however, there isn’t really much point because you know that if you upgrade your Apache server when you need to, then the security problems will be fixed.
What would you say was the weakest part, security wise, of most websites ?
That’s a hard question! It took me a while to work out what the most sensible answer is: the weakest part is the lack of maintenance and upgrade.
The problem is that keeping a system updated is hard work. If you manage 40, 50, or 150 Unix systems, then keeping up with all of them does require a whole lot of skills, because at that point the shell is just not good enough. You need to use something like CFEngine to configure them, and other automated tools to keep an eye on their security.
Here is an example: I have my own server, where I host my personal web site, my friends’ email, their small sites and so on.
I receive my email from LogWatch every day.
Today, it read:
**Unmatched Entries**
Illegal user patrick from 161.53.202.3
Illegal user patrick from 161.53.202.3
Illegal user rolo from 161.53.202.3
[…]
Illegal user john from 161.53.202.3
Illegal user test from 161.53.202.3
Illegal user merc from 151.31.36.81
Normally, I would run whois, find out who manages those networks, and report these attempts. Well, today I simply didn’t have the time. I am writing this answer on a train to London. Tomorrow I will be in Brunei, and in three days I will be back to Perth. My Internet connection is expensive and erratic. So here I am, Mr. 161.53.202.3 tried to attack me and he won’t be reported. And that’s only one person (me) with only one server!
Do you advocate the use of ‘cracker’ tools for testing?
That’s another tough one.
Well, I don’t advocate the /use/ of such tools. However, I am strongly against making these tools illegal.
Crackers and script kiddies, at the end of the day, are our friends (!). If you compared the Internet to an living organism, they are like those nasty (biological) viruses which occasionally knock you down and give you a tremendous sore throat, but are necessary to keep your body alert and your antibodies "trained". Also, if you catch a cold you can’t blame it completely on the virus - you’ve got to wonder if your body is healthy.
I believe it’s the same with the Internet: crackers will randomly try and get into your system (literally!). You have to make sure your defences are strong enough and well organized, so that when that happens you are prepared.
Some big companies won’t accept that. They will try to make tools such as Nexus illegal. Why? Maybe because they think that if such tools are not available anymore, then crackers will simply disappear. Or who knows, maybe they would like to sell testing tools to certified companies for a lot of money…
You cover quite a few security modules. Which would pick, and why, as the best modules?
The best and most useful module in my opinion is mod_dosevasive written by Jonathan A. Zdziarski. I believe Jonathan deserves a monument dedicated to him, also because he wrote DSPAM (which saves my life on a daily basis).
I believe that it should be part of the default Apache installation - in fact, I wonder if the Apache group would.
Tell us what your ‘Apache in Jail’ chapter is all about.
Well, jailing can be extremely complicated, but at the same time it is a very powerful tool against crackers.
Thanks to the system call "chroot()", you can tell a program what the root directory is when it runs. For example, you could run Apache making it believe that the root directory ("/") is "/cage/apache". This means that Apache will not be able to see anything outside "/cage/apache" - which is while you say that it’s "jailed". If a cracker does manage to use a buffer overflow exploit against your server, and get Apache to execute arbitrary commands, there will be nothing in /bin or /sbin to be executed, because /cage/apache/bin and /cage/apache/sbin will be nearly empty!
In my book, I tried to explain how to "jail" Apache step by step, by trying to make the readers aware of why and how everything was done. This deep understanding is necessary, because it is really quite tricky to use more complex software and third party modules on a jailed Apache.
You have some unusual outside interests. How did you end up sharing your life between Apache security with Jazz and Ballet?
At the moment it looks like I have a broken knee and I haven’t danced in ages (2 months), which is very sad. Classical Ballet (I am hopeless at jazz) has become part of me. As you can imagine, I spend a lot of time sitting down in front of a computer. Dancing is my escape: I love classical music, and I love feeling fit. You see, when you are training at ballet, you are a sort of a super-human: you never get tired, you are very flexible, and you generally feel good (and as this is a serious interview, I won’t mention the fun of making six pirouettes suddenly in the middle of the footpath while having a stroll with friends).
It’s funny, because I never considered the two things (computers and dancing) to be in contrast.
Anything in the pipeline?
Well, right now I am working on "Free Software Magazine" (www.freesoftwaremagazine.com), a magazine which concentrates entirely on free software.
It has been amazingly challenging. The first issue (January 2005) required a huge effort from many people, but the result is really rewarding.
Tony Mobily Bio
Tony Mobily is the project coordinator of Free Software Magazine.
When he is not talking about himself in the third person, Tony Mobily, BSc, is an ordinary human being, enjoying his life in the best city in the world: Perth (Western Australia). He is a senior system administrator and security expert, and is knowledgeable in several internet technologies. He loves Linux, Apache, Perl, C, and Bash.
Tony has been in the publishing industry his whole life, starting from the Italian magazine Dev. (he is lucky enough to be bilingual) in 1996.
He is also trained in Classical Ballet (ISTD), and fighting his way through learning hip hop and jazz. He also writes short and long stories, and keeps a blog at http://www.mobily.com.