Parallels H-Sphere Documentation System Administrator Guide

 

Implementation of Load Balanced Cluster in H-Sphere

(H-Sphere 3.0 RC 1 +)
 
 

Related Docs:   Understanding Load Balancing in H-Sphere Installation of Load Balanced Web/Mail Clusters in H-Sphere Load Balanced Clusters (Admin Guide)

Last modified: 27 Dec 2007

 

WARNING: This documentation covers Parallels H-Sphere versions up to 3.1. For the latest up-to-date Parallels H-Sphere documentation, please proceed to the official Parallels site.

This document describes load balanced Web/mail cluster scheme for generic Linux NFS NAS.

 

Load balanced cluster in CP

 

Distribution of requests across load balanced cluster

H-Sphere regards Web or mail load balanced cluster as a single server. Requests are passed to external IPs routed by the load balancer. Load balancer distributes requests evenly across the master and slave servers (internal IPs corresponding to external IP).

 

Shared content

Master and slave servers share the same /hsphere directory mounted by NFS to the /filer/<cluster_type>_<cluster_id>/ directory on the NAS where load balanced cluster content is actually stored. Here, <cluster_type> is mail or web, and <cluster_id> is cluster id - there may be multiple load balanced clusters mounted to the NAS: 01, 02, ... (See the illustration for 2 Web and 2 mail clusters.)

All user content, scripts, and the majority of H-Sphere related binaries are installed into the /hsphere directory and shared by master and all slaves.

Follow the Adding Load Balanced Clusters on Shared Linux NFS instructions to learn how to correctly mount the shared storage on the NAS to the master and slave servers.

 

Specific master/slave content

Along with the common shared storage, master and slave servers have their own H-Sphere specific (Apache, FTP) and IP-dependent (network) logs and configuration:

  • Both master and slave servers have unique Apache logs stored locally in the /var/log/hsphere/httpd directory instead of the default /hsphere/local/var/httpd/logs directory.
  • On the master server, Apache, FTP and network configuration is located in the /hsphere directory (which is a mountpoint to the NAS and is common for the master and the slave servers):
    /hsphere/local/config/httpd/
    /hsphere/local/config/ftpd/
    /hsphere/local/network/
    On slave servers, however, this data is unique and is stored locally in the /etc/hsphere directory:
    /etc/hsphere/httpd/
    /etc/hsphere/ftpd/
    /etc/hsphere/network/
    H-Sphere updater running with the hspackages slaves=web|mail|all option creates the /etc/hsphere directory data on slave servers.

 

Synchronization between master and slave servers

The special cron script /hsphere/shared/scripts/cron/lb_sync.sh runs each minute on each slave server to synchronize data on master and slave servers. It parses and synchronizes:

  • User Apache/ProFTPd config files /etc/hsphere/[httpd|ftpd]/sites/*.conf, and /etc/hsphere/network/ips;
  • the /etc/passwd, /etc/shadow, /etc/group files.

 

Traffic calculation

User logs are located in the /hsphere/local/home/<user>/logs/domain.com/ directory. On master server, log filenames look like:

{domain.name}
referrer_log
access_log
error_log

Slave servers write the following logs:

{domain.name}_SRV-N
referrer_log_SRV-N
access_log_SRV-N
error_log_SRV-N

where N is slave server id: 1, 2, ...

The /hsphere/shared/scripts/cron/cron-rotate.pl script has been adapted to calculate statistics on load balanced cluster. It runs on master and slave servers by the following scheme:

  • on slave servers (e.g., 1:30am):
    • synchronizes logrotate conf files on master and slaves: It parses log rotate configs in the /hsphere/local/config/httpd/logrotate_confs/ directory on the master server. They point to the user logs located in the /hsphere/local/home/<user>/logs/domain.com/ directory. On master server, log filenames look like:

      {domain.name}
      referrer_log
      access_log
      error_log

      Slave servers write the following logs:

      {domain.name}-SRV-N
      referrer_log-SRV-N
      access_log-SRV-N
      error_log_SRV-N
      where N is slave server id: 1, 2, ... cron-rotate.pl merges data and synchronizes respective master and slave logs.
    • rotates logs on slave servers
    • restarts Apache
  • on master server (e.g., 2:00am):
    • rotates logs on the master
    • restarts Apache
    • merges master and slave logs
    • launches log analyzers (Webalizer, AWStats, ModLogAn)

 

Load balanced cluster map

The following two files construct load balanced cluster map. At the moment, they need to be created manually on master and slave servers:

  • /hsphere/local/config/lb.map - created on the master server and has the following format:

    <Master_IP>|<Slave1_IP>|...|<SlaveN_IP>

    The lines of the same format should be also added for each dedicated IP bound on the cluster:

    <Master_Dedicated_IP>|<Slave1_Dedicated_IP>|...|<SlaveN_Dedicated_IP>

  • /etc/hsphere/lb.id - created on both the master and slave servers and contains the following line:

    <CLUSTER_TYPE>|<SERVER_ID>

    where <CLUSTER_TYPE> is mail or web; <SERVER_ID> is LB server id: 0 for master, 1 for the first slave, 2 for the second slave, etc.

    For example, for slave server with <Slave2_IP> in LB Web cluster the lb.id file will look like:

    web|2

 

NAT configuration for load balanced clusters

To configure load balanced Web/mail cluster with NAT, you must have NAT turned on in H-Sphere and put external Web/mail server IP routed by the Load Balancer into correspondence with the master server's internal IP.

For example, for a load balanced Web cluster with one master and 4 slave servers, where the master Web server's internal IP 192.168.0.100 corresponds to the external IP 12.34.56.100 bound to the Load Balancer.

  • In the ~cpanel/shiva/psoft_config/ips-map.xml file on the CP server there should be the following record:

    <ips>
    . . .
       <ip ext="12.34.56.100" int="192.168.0.100"/>
    . . .
    </ips>

  • All dedicated IPs on the master server must be also associated with corresponding IPs on the Load Balancer and similar records must be added to the ip-map.xml file:

       <ip ext="LB_Dedicated_IP" int="Master_Dedicated_IP"/>

  • Also, you should have external IP in the E.Manager -> DNS Manager -> Service Zone menu in admin CP. For example:

    www.test.com 3600 IN A 12.34.56.100
    mail.test.com 3600 IN MX 12.34.56.111


Related Docs:   Understanding Load Balancing in H-Sphere Installation of Load Balanced Web/Mail Clusters in H-Sphere Load Balanced Clusters (Admin Guide)



© Copyright 2017. Parallels Holdings. All rights reserved.