Nginx virtual host configuration

This entry is part 2 of 2 in the series Self-made webserver

In the first part of the “Self-made webserver” series I’ve described how you can compile Nginx on Ubuntu. Now let’s have a look how we can optimize our virtual host configuration by adding PHP-FPM Support and some other settings.

First of all the configuration file:

server {
    # Redirect from example.com to www.example.com
    listen *:80;
    server_name example.com;
    rewrite ^/(.*) http://www.example.com/$1 permanent;
}

server
{
    server_name www.example.com;
    listen *:80;
    root /var/www/www.example.com/public;

    # logging
    access_log /var/www/www.example.com/log/access.log;
    error_log /var/www/www.example.com/log/error.log;

    # deny access to hidden files
    location ~ /\. {
        access_log off;
        log_not_found off;
        deny all;
    }

    # Maximum expiration for static files
    location ~* \.(?:ico|css|js|gif|jpe?g|png)$ {
        expires max;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    }

    if (-f $request_filename) {
        break;
    }

    # removes trailing slashes (prevents duplicate content issues)
    if (!-d $request_filename) {
        rewrite ^/(.*)/$ /$1 permanent;
    }

    # Index files
    location / {
        index index.html index.php;
        autoindex off;
    }

    # If file doesn't exists rewrite to index.php,
    # necessary for most php based frameworks/applications
    if (!-e $request_filename) {
        rewrite ^.*$ /index.php last;
    }

    # Pass PHP scripts to PHP-FPM
    location ~* \.php$ {
        try_files $uri /index.php;
        fastcgi_index index.php;
        fastcgi_pass php5-fpm;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param SCRIPT_NAME $fastcgi_script_name;
    }
}

Configuration in detail

As you can see I have two server sections for the same domain. The purpose is to redirect addresses without www prefix. There are groups of people that say it’s outdated, but I prefer to do so because of the cookie sending behavior of the browser. If you use only example.com without prefix, and you set a cookie there, the browser will send the cookie information to all request of *.example.com. So static.example.com/my.jpg would also receive cookie headers, which generates more overhead traffic.

Another point is that most application recognize www.example.com as link, whereas example.com isn’t. Ok, I must confess this is a small issue, but anyway it’s up to you how you configure it.

Deny and redirect

Hidden files are supposed to be hidden also from the user, therefore I’ve placed a rule to deny access to hidden files:

location ~ /\. {
    access_log off;
    log_not_found off;
    deny all;
}

For static files I’ve added an expiration setting which is set to maximum. Also Cache-Controll headers are added automatically:

location ~* \.(?:ico|css|js|gif|jpe?g|png)$ {
    expires max;
    add_header Cache-Control "public, must-revalidate, proxy-revalidate";
}

Files that don’t exist are going to be redirected to index.php. This setting is required by most applications to realize pretty URIs.

if (!-e $request_filename) {
    rewrite ^.*$ /index.php last;
}

The last section worth mentioning is the PHP-FPM configuration. PHP files are passed to fast-cgi, in this case to PHP-FPM. If you followed the series, you will notice that we didn’t installed PHP-FPM yet. This is the next step I will describe in the next part.

Apache Tomcat 7 on Windows – Running out of disk space

In a current at my work I’ve deployed an Apache Tomcat 7 on a Windows 2008 server machine as development environment. I’ve realized that from time to time Tomcat crashes, before doing so the memory usage grow incredibly fast and then Tomcat dies.

I’ve traced down this issue to bad thread implementation in one of the application components. However, there was an issue with the server running out of disk space pretty fast. After analyzing the folders I’ve found out that in the Tomcat bin folder there is a bunch of dump files which are over 2.5 GB in size. I after deleting them the server had a lot of free disk space again.
Just posted this in case you have a similar issue, I hope this may help someone.

Windows Server Out of disk space 300x125 Apache Tomcat 7 on Windows   Running out of disk space

Installing Nginx on Ubuntu 11.10 (Oneiric Ocelot)

This entry is part 1 of 2 in the series Self-made webserver

Nginx got gained a lot of popularity in the past times. The reason is pretty simple: It’s lightweight and very fast, in the same time it uses a minimum of resources. Although I don’t have a high-traffic site on my own, I see that Nginx uses by far less resources than Apache. This makes it perfect to run on a small sized server like an AWS EC2 T1.Micro instance. In combination with PHP-FPM and proper caching you can handle quiet a lot of traffic with an minimum amount of resources. Even on my development machine (MacBook Pro early 2011) I’m running Nginx with PHP-FPM.

Nginx logo 300x91 Installing Nginx on Ubuntu 11.10 (Oneiric Ocelot)

In this article series I’m using Ubuntu 11.10 server edition, as it’s running on all my server machines.

Getting ready

I assume that you do all the steps on a fresh installed Ubuntu server, so let’s start with updating the system.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential

After apt finished updating the system we need the source files of Nginx, zLib and PCRE. For SSL support the libssl-dev package must be installed.

sudo apt-get install libssl-dev
cd ~/
wget http://nginx.org/download/nginx-1.0.14.tar.gz
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.30.tar.gz
wget http://zlib.net/zlib-1.2.6.tar.gz
tar -xzf nginx-1.0.14.tar.gz
tar -xzf pcre-8.30.tar.gz
tar -xzf zlib-1.2.6.tar.gz

On a plain Ubuntu 11.10 server installation the /var/www and /var/log/nginx directory don’t exists. Create them first before configuring and installing Nginx.

mkdir /var/www
mkdir /var/log/nginx
chown www-data:www-data /var/www

./configure –conf-path=/etc/nginx/nginx.conf \
–pid-path=/var/run/nginx.pid \
–error-log-path=/var/log/nginx/erroor.log \
–http-log-path=/var/log/nginx/http.log \
–user=www-data \
–group=www-data \
–with-pcre=../pcre-8.30 \
–with-zlib=../zlib-1.2.6 \
–with-http_ssl_module \
–sbin-path=/usr/sbin/nginx

make && sudo make install

sudo mkdir /etc/nginx/sites-available
sudo mkdir /etc/nginx/sites-enabled
sudo chown -R root:root /etc/nginx/sites-available
sudo chown -R root:root /etc/nginx/sites-enabled

The ‘make install’ command installs Nginx into the configured folders. Before having Nginx up and running we need to do a couple of things. First you should download the Nginx init script I’ve extracted from the ubuntu package and put it to /etc/init.d/nginx. Don’t forget to set the executable flag (‘sudo chmod +x /etc/init.d/nginx’).

Customizing the configuration

The next step is to modify the default Nginx configuration:

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;

events {
worker_connections 768;
}

http {
client_max_body_size 1024m;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_tokens off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
include /etc/nginx/sites-enabled/*;
}

Note that I’ve deleted all commented lines and the default server. More important is the include line: It will include our virtual host configurations folder. Feel free to customize the settings to fit your needs. Me for example I prefer to set gzip settings per virtual host. You can now run nginx using the init scirpt:


sudo /etc/init.d/nginx start

Creating a virtual host

Before you create the first virtual host you should create the web root folder.

mkdir /var/www/www.example.com
mkdir /var/www/www.example.com/public
mkdir /var/www/www.example.com/log
sudo chown www-data:www-data /var/www/example.com

And now let’s have a look into the configuration file for a virtual host:

#/etc/sites-available/example.com.conf
server {
listen 80;
server_name example.com www.example.com;
root /var/www/www.example.com/public;
access_log /var/www/www.example.com/log/access.log;
error_log /var/www/www.example.com/log/error.log;
index index.php index.html;
charset utf-8;

location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ {
expires max;
}

gzip on;
gzip_disable “msie6″;

gzip_vary on;
gzip_proxied any;
gzip_comp_level 4;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
}

By the way, you don’t need to add html type to the list as it’s compressed by default.
Create a symbolic link to activate the configuration and restart the server:

sudo ln -s /etc/nginx/sites-available/example.com.conf \
/etc/nginx/sites-enabled/example.com.conf
sudo /etc/init.d/nginx restart

That’s it. If you have a host entry for www.example.com you should now see the site working. If you have troubles Nginx getting working don’t be shy to drop a comment icon smile Installing Nginx on Ubuntu 11.10 (Oneiric Ocelot)

Mac OS X Lion – Upgrade to PHP 5.4

One of the reasons I like about Mac OS X is that it ships PHP out-of-the box. Until now there is no update by Apple for the slightly old PHP 5.3.8 version. Yesterday I’ve read the news about the PHP 5.4 release and I decided to take this opportunity to upgrade the PHP installation on my MacBook Pro.

PHP 5.4 on Lion 300x175 Mac OS X Lion   Upgrade to PHP 5.4

I’ve google’d a little bit, but I couldn’t find an article that describes in a short form. The PHP documentation has a ‘Installation on Mac OS X’ section also, but guess what: It redirects you to the ‘Installation on Unix systems’ section. Not many things in this section are relevant for your OS X box.

Until now I’ve used pre-made packages for every system I use, rarely I really needed to compile something. Because of this I was unsure about the configure parameters I need for PHP compilation. When you look at out of the php_info() command you can see the compile time options. I’ve figured out that you need to install a few libraries using brew and you need to remove some parameters because some functionality are now built-in by default or moved to PECL.

I recommend to all Mac OS X users to install Homebrew. It’s like a packet manager for Mac, you can type brew install <package> in a console window to get applications, tools and libraries installed with an one-liner. Grab it from the Homebrew Github Site.

Now let’s upgrade PHP in a few steps:

  1. Download and unpack the PHP source archive
  2. Install following packages using brew: libjpeg and pcre
  3. Change directory to the source archive of PHP
  4. Use the configure command to prepare the compilation process:
./configure \
--prefix=/usr \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--sysconfdir=/private/etc \
--with-apxs2=/usr/sbin/apxs \
--enable-cli \
--with-config-file-path=/etc \
--with-libxml-dir=/usr \
--with-openssl=/usr \
--with-kerberos=/usr \
--with-zlib=/usr \
--enable-bcmath \
--with-bz2=/usr \
--enable-calendar \
--with-curl=/usr \
--enable-dba \
--enable-exif \
--enable-ftp \
--with-gd \
--enable-gd-native-ttf \
--with-icu-dir=/usr \
--with-iodbc=/usr \
--with-ldap=/usr \
--with-ldap-sasl=/usr \
--with-libedit=/usr \
--enable-mbstring \
--enable-mbregex \
--with-mysql=mysqlnd \
--with-mysqli=mysqlnd \
--without-pear \
--with-pdo-mysql=mysqlnd \
--with-mysql-sock=/var/mysql/mysql.sock \
--with-readline=/usr \
--enable-shmop \
--with-snmp=/usr \
--enable-soap \
--enable-sockets \
--enable-sysvmsg \
--enable-sysvsem \
--enable-sysvshm \
--with-tidy \
--enable-wddx \
--with-xmlrpc \
--with-iconv-dir=/usr \
--with-xsl=/usr \
--enable-zip \
--with-pcre-regex \
--with-pgsql=/usr \
--with-pdo-pgsql=/usr \
--with-freetype-dir=/usr/X11 \
--with-jpeg-dir=/usr \
--with-png-dir=/usr/X11
view raw configure.sh This Gist brought to you by GitHub.

That’s it. After successful configuration use ‘make test’ to check your compilation and ‘sudo make install’ to actually install the new version.

MySQL – Versioning records of tables using triggers

Versioning of records is often realized in the application logic. For example WordPress implements versioning by insterting a new post version into the posts table whenever a new revision is added. I don’t know the PHP interna of WordPress good enough to judge the performance / complexity this feature adds to the product. In this article I’m going to show you how you can let versioning handle by MySQL.

Creating the tables

For demonstration purposes I’m using a simple table with two columns (id and data) and I name it mydata. Not hard until now, huh?

The next step is to create a version table. In my projects I use the convention that a leading underscore indicates a versioning table; so I’m going to use _mydata as version table name.

CREATE TABLE `mydata` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `data` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `id_UNIQUE` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
view raw mydata.sql This Gist brought to you by GitHub.
CREATE TABLE `_mydata` (
  `id` int(10) unsigned NOT NULL,
  `version` int(10) unsigned NOT NULL,
  `creation` datetime DEFAULT NULL,
  `data` varchar(100) DEFAULT NULL,
  `action` varchar(45) DEFAULT NULL,
  PRIMARY KEY (`id`,`version`),
  UNIQUE KEY `id_version` (`id`,`version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
view raw _mydata.sql This Gist brought to you by GitHub.

You might have noticed the additional fields in the versioning table. The version field points to the version the record represents. The second one stores the creation time of a version. Additionally I’ve added an action column. The action column contains the operation on the main table which triggered the creation of a version entry. A record with the action INSERT always has version number one – as it’s the creation of the row in the main table. The action UPDATE is in the action column when a UPDATE statement was executed on the main table. Finally, what happens when you delete a record on the main table? Very simple: If you delete a record then our trigger will insert a new entry with empty data column and DELETE in the action field.

Creating a trigger

Triggers are similar to event handlers in common programming languages. A good example is the onClick-handler in JavaScript which is invoked by a user click on a element in the HTML document. Triggers in MySQL can handle a limited amount of events:

  • INSERT
  • UPDATE
  • DELETE

There is also a modifier BEFORE and AFTER for the event declaration – one of them must be used. The syntax of the CREATE TRIGGER statement is pretty simple:

CREATE TRIGGER <trigger_name> <BEFORE|AFTER> <INSERT|UPDATE|DELETE>
ON <table_name> FOR EACH ROW <trigger_body>

I think the parameters don’t need to be explained – except <trigger_body>. Basically you can put there – more or less – any SQL statements including variables and function calls. If you want to execute multiple statements, then the <trigger_body> must be enclosed by a BEGIN and END statement.

The delimiter thing

The default delimiter of MySQL is the semicolon. It marks the end of a statement, the server knows that it can process the statement. While creating a trigger you enclose your statements in the trigger body between a BEGIN and END. In between both of these keywords you usually have other statements which are delimited by a semicolon. MySQL will not process your trigger definition properly, as you delimited in the middle of the CREATE TRIGGER statement. To avoid such problems you can redefine the delimiter in your query. Check how delimiters are used in the following snippets to create the triggers:

Creating the INSERT trigger

DELIMITER $$

CREATE
TRIGGER `mydata_after_insert`
AFTER INSERT ON `mydata`
FOR EACH ROW
BEGIN
    INSERT INTO `_mydata` VALUES(NEW.id, 1, NOW(), NEW.data, 'INSERT');
END;
$$

Creating the UPDATE trigger

DELIMITER $$
CREATE
TRIGGER `mydata_after_update`
AFTER UPDATE ON `mydata`
FOR EACH ROW
BEGIN
    DECLARE nextversion int(10) unsigned;
    SELECT (MAX(`version`) + 1) FROM `_mydata` WHERE id = NEW.id into nextversion ;
    INSERT INTO `_mydata` VALUES(NEW.id, nextversion, NOW(), NEW.data, 'UPDATE');
END;
$$

Creating the DELETE trigger

DELIMITER $$
CREATE
TRIGGER `mydata_after_delete`
AFTER DELETE ON `mydata`
FOR EACH ROW
BEGIN
    DECLARE nextversion int(10) unsigned;
    SELECT (MAX(`version`) + 1) FROM `_mydata` WHERE id = OLD.id into nextversion ;
    INSERT INTO `_mydata` VALUES(OLD.id, nextversion, NOW(), '', 'DELETE');
END
$$

Testing the triggers

Now we have created the main table, the version table and the triggers. Let’s see if it works:

INSERT INTO `mydata` (`data`) VALUES('data-insert');
UPDATE `mydata` SET `data`='data-update' WHERE `id`='1';
DELETE FROM `mydata` WHERE `id`=1;
view raw cud-ops.sql This Gist brought to you by GitHub.

If the data in _mydata looks like this it’s a time for a hurray!

MySQL History Table1 300x128 MySQL   Versioning records of tables using triggers

The first steps – Android App Development

This entry is part 1 of 1 in the series Starting Android development

It’s difficult to find a task that isn’t already covered by an app in the official App Market or from other sources. But some of these tools aren’t written well; they crash, slow down your phone or drain your phone’s battery (hi-five Skype). There are many reasons to start developing on Android. My personal favorite is: because it’s fun!

You can develop apps in Java or C/C++. The NDK enables you to write low-level application that are performance-critical, but I won’t cover this in this tutorial. In this article series I’m going to focus on the Java part. Let’s start with the setup of the development environment. First of all we need to download a few things:

The Android SDK contains libraries and tools that helps you while developing apps. After finishing the download unzip the SDK and start the application named android located in the tools folder:

Unzipped Android SDK folder 300x138 The first steps   Android App Development

A terminal (or console) window pops up and the Android SDK Manager starts. Select the Android version you would like to develop for and additionally check the tools node.

Android SDK Manager 300x133 The first steps   Android App Development

In this tutorial I’m using Android 2.2. For learning purposes 2.2 is just fine, although you should consulate the platform version popularity overview when writing your first real-world app. For example if you choose 4.0  for version specific features (at the time of writing this article) your app would run only on about 1% of the devices on the market.

The installation of Eclipse is pretty easy. Just download and unzip it somewhere you like and launch it. At the Eclipse main window click Help > Install additional software. Click the Add button and enter Eclipse ADT Plugin in the name field of the window that pops up. Paste https://dl-ssl.google.com/android/eclipse/ in the location field and click OK. The repository manager refreshes the view and you see a new package in the list. Select the root node, press continue and install all packages.

Add Eclipse Plugin Repository 300x104 The first steps   Android App DevelopmentInstall Eclipse ADT Plugins 300x93 The first steps   Android App Development

When the download and installation of the plugins are finished Eclipse asks for a restart. Confirm it. After the relaunch you will see two new buttons with Android logo in your toolbar. The next step is to create a virtual device.

Creating a virtual device

If you want to test your apps without deploying it on a phone then the Android device emulator is your only option. Eclipse uses it for debugging and running your application. Open the device manager using the menu Window > AVD manager.

Android Virtual Device 286x300 The first steps   Android App Development

Target is the the API level the device should use (basically Android version). Set a SD card size and save the virtual device.

Creating the first project

If you followed the guide you should have now fully configured Android development environment. But that’s not all. Let’s finish this tutorial by letting a HelloWorld app run in the virtual device.

Open Eclipse if it’s not already and create a new project and follow this steps:

Eclipse Create New Project 300x212 The first steps   Android App Development

Select File > New > Other to open the list of all project types available in Eclipse. At the following screen select Android Project.

Eclipse Project Wizard Android 300x284 The first steps   Android App Development

Select Android 2.2 and hit next.

Eclipse Select Build Target 285x300 The first steps   Android App Development

The next form is pretty clear: Set an app name and a Java namespace as package name. Ensure that Create Activity is checked.

Eclipse Application Info 286x300 The first steps   Android App Development

Click finish to complete the project creation process. Use the arrow on the debug icon to open the Debug configurations window. On this screen select Android Application and right-click to select New on the context menu.

Eclipse Debug Configurations Menu The first steps   Android App DevelopmentEclipse Android Application Debug New 300x113 The first steps   Android App Development

When you have created you debug configuration, on the right sight a couple of options become available. Hit the Browse button and set your project to run with this configuration:

Eclipse Android Debug Project Selection 300x294 The first steps   Android App Development

Finally hit the Debug button. The Android virtual device will start and boot Android 2.2. This can take a while. When finished with the initialization you will see the Android screen lock. Unlock the screen to see your app running:

Android Device Locked Screen 300x277 The first steps   Android App DevelopmentHello World App Running 180x300 The first steps   Android App Development

Congratulations! If you’ve done everything right you should see now your HelloWorld app running in the virtual device. Feel free to drop a comment if you have any questions or suggestions!

Setting up a MySQL Master-Slave replication

In a current project I needed to set up a MySQL Master-Slave replication on different VPCs. The servers are located in different data centers.I set up everything in VMWare virtual machines to test the configuration:

  1. Create a Linux virtual machine
  2. Duplicate it
  3. Install MySQL on the servers
  4. Configure MySQL servers
  5. Create a replication user
  6. Create a Snapshot of master
  7. Start the replication

Configuring MySQL

In this example I’m using Ubuntu 11.10. Use the apt command to install mysql-server package on both servers:

sudo apt-get install mysql-server

To get replication work it’s necessary to assign a server-id to the MySQL instances. Also binary logging must be enabled as it’s used for synchronization:

Master


bind-address = 10.137.56.141
log-bin=mysql-bin
server-id=1

Slave


bind-address = 10.137.56.91
server-id=2

Don’t forget to restart the servers. The next step is to create replication user on the master server:

CREATE USER 'replication'@'10.137.56.91' IDENTIFIED BY 'pass';
GRANT REPLICATION SLAVE ON *.* TO 'replication'@'10.137.56.91';
view raw file1.sql This Gist brought to you by GitHub.

Creating a snapshot of the master

For a first synchronization of the data you need to first LOCK all tables and export the data using mysqldump.

FLUSH TABLES WITH READ LOCK;
view raw gistfile1.sql This Gist brought to you by GitHub.

Open a second session (using a second terminal or screen) to the master and dump all databases:

mysqldump -ppass --all-databases --master-data > master.db

The –master-data option causes the dump output to include a CHANGE MASTER TO statement that indicates the binary log coordinates (filename and position) of the dumped server. Alltought it doesn’t include the credentials, so we still need to set up the credentials after the import into the slave. Copy the master.db dump file to the slave and import it:

mysql -ppass < master.db

At this point your slave server has a snapshot of the master data. To complete the configuration the slave needs now to know how to connect to the master.

Configuring the slave

Execute following SQL to finish the slave configuration:

CHANGE MASTER TO MASTER_HOST='10.137.56.91', MASTER_USER='replication', MASTER_PASSWORD='pass';
start slave;
view raw gistfile1.sql This Gist brought to you by GitHub.

To check how both servers are doing you can use the SHOW MASTER STATUS or SHOW SLAVE STATUS SQL command.

MySQL Replication Show Master Status 300x108 Setting up a MySQL Master Slave replication

MySQL Replication Show Slave Status 300x95 Setting up a MySQL Master Slave replication