I was wondering how I should manage the transfer of several huge websites that are regulary updated from one server to another one so the downtime is minimal and the transfer is transparant for users.

Finaly, it came the idea to use the power of Apache mod_proxy and rsync. The downtime of the page could then be minimal and I could ask the customer to change their DNS config before the website is entirely transfered.

Requirements:

• A website (in this example: www.example.com)
• Two servers
  • Source should work with rsync
  • Target should run with Apache2 and mod_proxy enabled
• Good knowledges/understanding of Apache/DNS/Linux

Steps:

1. Create a vhost on the target server
2. Copy the website on the target server
3. Configure and test the website on the target server
4. Configure mod_proxy on the target server
5. Make the changes on the DNS
6. Sync the website on the target server
7. Disable the mod_proxy on the target server
8. Sync again the website on the target server
9. Remove the waiting page
10. Make a full backup on the old server
11. Enjoy your migration drinking a Mojito

Conventions:

• We are migrating the website www.example.com
• The source server have the IP: 80.0.0.1
• The target server have the IP: 80.0.0.2

Step 1: Create a vhost on the target server

Step 2: Copy the website on the target server

Now that your target server is ready to be the hosting place of your website, it’s time to make your first copy of your website. This will reduce the unavailable time of your website during the real switch.

To do that properly, it should be a good idea to think about a synchronisation system to allow us to make a first copy and then to synchronise it later. When the synchronisation occures, only the modified/added files are downloaded from the source server.

To copy the files from the source to the target, you have many possibilities.

One of them is using rsync (http://www.samba.org/rsync). The advantage of rsync is to allow you to make a real syncronisation of the files, allowing you to just download the changes during the final syncronisation.

Step 3: Configure and test the website on the target server

After the syncronisation, you could have to changes the parameters of the website on the target for it to work.

• Change the files permissions
• Change the databases settings
• You may have to configure a hard coded local path in your configuration files
• Configure your local “hosts” file to be able to check the website on the new server

All theses changes should only have an impact on your side because you didn’t change the public DNS settings yet.

Step 4: Configure mod_proxy on the target server

So, basicaly, we have to tell Apache to proxy all the requests to the target server with the host www.example.com onto the source server. The source server have to “think” that we have asked it the website www.example.com.

First, we need to set some rules into the Apache httpd.conf file

# Essential modules
LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
# These must be set else the host isn't passed correctly
UseCanonicalName Off
ProxyVia On
ProxyRequests Off
# This must also be set, though it's only an option in Apache2
ProxyPreserveHost On

Then we create a vhost for www.example.com:

<VirtualHost *:80>
  ServerName   www.example.com # We are treating all the request to www.example.com
  ServerAlias example.com
  ProxyPass / http://80.0.0.1:80/
  ProxyPassReverse / http://80.0.0.1:80/
  ProxyPassReverse / http://www.example.com:80/
  ProxyPassReverse / http://example.com:80/
</VirtualHost>
Now, reload Apache2 on the target.
To test the configuration, you can temporary modify your host file (Google it or check here) setting www.example.com to IP 80.0.0.2 and see if www.example.com gives the good result into your browser.
Step 5: Make the changes on the DNS
When you are sure that your apache mod_proxy works, you can change your DNS. You can now set the domain www.example.com to point to IP 80.0.0.2.
When that done, check during 48h if everything went fine and if your website is still working correctly. If you have tested your proxy correctly, it should go fine. If not, just pray…
Step 6: Sync the website on the target server
When you think the DNS changes has been globally impacted, you can synchronise your files again.
Regarding the technique you have used in step 2, do it again to have a fresh synchro of the files.
Check your website configuration again regarding what you did in step 3.
Step 7: Disable the mod_proxy on the target server
It is now time to take the real risks 🙂
What you should do is first to create a waiting page (that will stay in place for a few minutes if everything goes fine).
Then, when that is ready, just disable the mod_proxy settings. You should now see the waiting page instead of the website.
Step 8: Sync again the website on the target server
Because it could have take some time to do the step 7 or some changes could have been made, just synchronise one last time.
Do not forget to synchronise the databases if needed.
Step 9: Remove the waiting page
Now, you have to make the latest big step: Remove your waiting page. This is like a rebirth for your website so take it slow and be carefull.
If you have the opportunity, make a last check of your website before removing the waiting page.
When the waiting page is removed, check one last time the entire website to be sure everything went right.
Step 7: Make a full backup on the old server
It is not finished yet! Your website is running on your new server, but never forget to do an extensive backup of your old hosting. This should also contain databases and logs.
Step 8: Enjoy your migration drinking a Mojito
Your migration is now done. You can take a break and enjoy your Mojito ! 🙂
Sources:

irc.feenode.net: #httpd – Covener and Fajita
http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypreservehost
http://www.ibm.com/developerworks/library/wa-lampsec/
http://www.samba.org/rsync/