H-Sphere Traffic Calculation

This document dwells specifically on the issues of traffic logs and traffic calculation.

1. Check traffic with H-Sphere Control Panel

   Log into H-Sphere as admin and view the Transfer Traffic Report in the Reports menu.

2. Check traffic on physical servers

   Web, FTP and mail logs are located in the /hsphere/local/var/statistic directory of the corresponding physical server.

   Log files have the following names:

   • dd.mm.YYYY.txt - web logs,
   • dd.mm.YYYY.gst.txt - ftp logs,
   • dd.mm.YYYY.ftp.txt - virtual ftp logs.
   • dd.mm.YYYY.qml - mail logs,

   where dd.mm.YYYY is the timestamp of log file creation date.

   Here, mail logs are generated by the qmail server, and ftp logs by proftpd utility.

   Log files contain specially-formatted information tabulated as follows:

   |name|xFer(kB)|Hits_All|Hits_HTML|

   Here, name is the domain name, xFer is total traffic in kilobytes.

   Processed traffic files are moved to the /hsphere/local/var/statistic/loaded directory as .gz archives.

3. Traffic processing scripts in cron
   
   
   • HTTP traffic:
     
     

     Cron runs the /hsphere/shared/scripts/cron/cron_analyze.pl script on everyday basis to collect Web traffic. The script, in turn, calls the /hsphere/shared/sbin/analyze script to parse the contents of the /hsphere/local/var/httpd/logs/access_log.{timestamp} Apache log files and writes traffic statistics into the specially formatted dd.mm.YYYY.txt files in the /hsphere/local/var/statistic directory.

   • User FTP traffic:
     
     

     Cron runs the /hsphere/shared/scripts/cron/ftp_anlz_user.pl script on everyday basis for collecting user FTP traffic.

     ftp_anlz_user.pl parses the /hsphere/local/var/proftpd/xferlog FTP log file and writes FTP traffic statistics into the timestamp-named /hsphere/local/var/statistic/dd.mm.YYYY.gst.txt statistics files.

   • Virtual FTP traffic:
     
     

     Cron runs the /hsphere/shared/scripts/cron/ftp_anlz.pl script on everyday basis for collecting virtual FTP traffic.

     ftp_anlz.pl parses the /hsphere/local/var/proftpd/logs/{vhost_id}.ftp.log logs files for each virtual FTP account and writes traffic statistics into the timestamp-named /hsphere/local/var/statistic/dd.mm.YYYY.ftp.txt statistics files.

   • Mail traffic:
     
     

     Cron runs the /hsphere/scripts/cron/mail_anlz.pl script on everyday basis to collect mail traffic. The script analyzes the /var/log/maillog Qmail log file and collects mail statistics into the specially formatted dd.mm.YYYY.qml.txt files in the H-Sphere statistics directory (/hsphere/local/var/statistic).

4. TrafficLoader

   TrafficLoader H-Sphere Java class is in charge of parsing the server traffic. That's how it is launched by cron:
   
   30 5 * * * su -l cpanel -c 'java psoft.hsphere.TrafficLoader'

   TrafficLoader processes Web, mail, FTP and virtual FTP traffic in the formatted statistics files located in the /hsphere/local/var/statistic directory and inserts these lines into the translog table of the H-Sphere system database.

   TrafficLoader also calls the /hsphere/shared/scripts/xfer_cat.pl script to move the already loaded statistics files to the /hsphere/local/var/statistic/loaded directory as .txt.gz archives.