LoadBalanced Web Master Recovery

Last modified: 27 Dec 2007

To restore crashed web master server:

1. Install H-Sphere 2.5 and install web service on this box.
2. Move /etc/passwd, /etc/shadow, /etc/group from slave to master server.
3. Log to this server under root and create hsphere and hsphere2 directories:
   

   mkdir /hsphere
   mkdir /hsphere2

4. Put the following lines into /etc/fstab:
   

   <NetApp_IP>:/vol/hsphere/home/linuxweb1-001 /hsphere nfs defaults,nfsvers=3 0 0
   <NetApp_IP>:/vol/hsphere/home/linuxweb2-001 /hsphere2 nfs defaults,nfsvers=3 0 0
   <NetApp_IP>:/vol/hsphere/home/linuxweb1-001/frontpage /usr/local/frontpage nfs defaults,nfsvers=3 0 0
   <NetApp_IP>:/vol/hsphere/home/linuxweb1-001/ssl.shared /hsphere/local/config/httpd/ssl.shared nfs defaults,nfsvers=3 0 0
   

   Where:

   • <NetApp_IP> is the IP of your Filer;
   • /vol/hsphere/home/ is the path to the directory with H-Sphere content on your Filer;
   • linuxweb1-001 is the name of your master web server.
   • linuxweb2-001 is the name of your slave web server.
5. Mount all the mount points from /etc/fstab:

   mount -a

6. Execute:

   perl -pi -e's:/hsphere/local/var/httpd/logs/httpd.pid:/hsphere/local/var/httpd/logs/logs1/httpd.pid:g'/etc/rc.d/init.d/httpd

7. Kill all httpd processes on the box:
   

   killall -9 httpd

8. Start httpd on the box:
   

   /etc/rc.d/init.d/httpd start

9. Generate SSH keys on the master web box by executing the following commands:
   

   # ssh-keygen -t rsa1
   # ssh-keygen -d

   * The passphrases must be empty.
10. Place content of the generated identity.pub and id_dsa.pub files into the authorized_keys and authorized_keys2 files correspondingly in /root/.ssh directory on your slave web box.
11. Check that you can SSH connect from your master web box to the slave without a password prompt. On the master web server execute:
    

    #ssh IP

    Where IP is the IP of your slave web server.
12. Put the following lines to the root's cron:
    

    1 */4 * * * /usr/sbin/ntpdate ntps1-{0,1,2}.uni-erlangen.de # time syncronization script
    0 1 * * * nice -15 /hsphere/shared/scripts/cron/cron_analyze.pl
    0 2 * * * nice -15 /hsphere/shared/scripts/cron/cron_rotate.pl
    0 3 * * * nice -15 /hsphere/shared/scripts/cron/ftp_anlz.pl
    0 4 * * * nice -15 /hsphere/shared/scripts/cron/ftp_anlz_user.pl
    */1 * * * * nice -15 /hsphere/shared/scripts/cron/apache-confsynch.pl
    */1 * * * * nice -15 /hsphere/shared/scripts/cron/apache-restart.pl > /dev/null 2>&1
    */1 * * * * nice -15 /hsphere/shared/scripts/cron/ftp-confsynch.pl > /dev/null 2>&1
    */2 * * * * nice -15 /hsphere/shared/scripts/cron/ftp-restart.pl > /dev/null 2>&1
    0 5 * * * nice -15 /hsphere/shared/scripts/cron/db_usage.pl
    */1 * * * * rsync -e ssh /etc/passwd IP:/etc/passwd
    */1 * * * * rsync -e ssh /etc/shadow IP:/etc/shadow
    */1 * * * * rsync -e ssh /etc/group IP:/etc/group
    */4 * * * * /hsphere/shared/scripts/cron/master-ipsynch.pl

    where IP is the IP of your slave web server.