Configure varnish logs (varnishnsca), logrotate and awstats

Postado por em
autor: Guillermo Garron.

Introduction


Varnish


Varnish is a reverse cache server:


Varnish is an HTTP accelerator designed for content-heavy dynamic web sites. In contrast to other HTTP accelerators, such as Squid, which began life as a client-side cache, or Apache, which is primarily an origin server, Varnish was designed from the ground up as an HTTP accelerator. – Wikipedia


I will assume you already have varnish installed and configure on your server, this article will only cover the generation and analysis of the varnish logs, via varnishnsca and awstats.


Configure varnishncsa


We’ll use varnishncsa to get the logs that awstats will be able to analyse.


Varnishncsa: Display Varnish logs in Apache / NCSA combined log format


The syntax is:


varnishncsa [?a] [?b] [?C] [?c] [?D] [?d] [?f] [?I regex] [?i tag] [?n varnish_name] [?P file] [?r file] [?V] [?w file] [?X regex] [?x tag]

What I did is to add this line in the /etc/rc.local file:


varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid

That line tells varnishncsa:


  • -a: To append the logs to an already existing file
  • -w: To write the logs to the /var/log/varnish/access.log file
  • -D: To run varnishncsa as a daemon
  • -P: To write the PID file in the /var/run/ folder

We now have varnishncsa up and running, now configure logrotate to rotate the logs, everyday at midnight.


Create the following file /etc/logrotate.d/varnish and put this contents on it:


    /var/log/varnish/*log {
            create 640 http log
            compress
            postrotate
                    /bin/kill -USR1 `cat /var/run/varnishncsa.pid 2>/dev/null` 2> /dev/null || true
            endscript
    }

If you need more info or options run: man logrotate


Done, we now have varnish writing logs to a file, and logrotate will rotate them everyday, we only need now to analyse them.


Configure awstats with varnish


If you are using ubuntu, run:


sudo apt-get install awstats

Then:


sudo cp /etc/awstats/awstats.conf /etc/awstats/awstats.www.domain1.com.conf

Install the GeoIP perl module


And configure these lines to look like this:


LogFile="/var/log/varnish/www.domain1.com.log"
SiteDomain="www.domain1.com"
HostAliases="www.domain1.com localhost 127.0.0.1"
LogFormat=1
DNSLookup=0
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"

Check here for more info about installing awstats and varnishncsa on Ubuntu


If you are not using Ubuntu, you can get the latest version here and install it basically all you need to do is to copy the contents of the .tar.gz file in /usr/local/awstats/ folder, and run the awstats_configure.pl tool, skip (write none) in the webserver config file question, then follow the questions and you will end up with a config file like: /etc/awstats/awstats.www.your.domanin.conf and should look more or less like this:


LogFile="/var/log/varnish/access.log"
LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain="www.go2linux.org"
HostAliases="go2linux.org www.go2linux.org 127.0.0.1 localhost"
# Possible values:
# 0 - No DNS Lookup
# 1 - DNS Lookup is fully enabled
# 2 - DNS Lookup is made only from static DNS cache file (if it exists)
# Default: 2
DNSLookup=1
DirData="/var/lib/awstats"
DirIcons="/icon"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2

Make awstats update hourly


Create the file /etc/cron.hourly/awstats and copy these lines inside the file:


#!/bin/sh
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.your.domain

Note: Change www.your.domain with your real domain name, the same one you entered when you configured awstats before.


That will update your awstats database hourly, let’s now make if visible, better only for you:


Copy these folders to your www root directory:


  • /usr/local/awstats/wwwroot/clases
  • /usr/local/awstats/wwwroot/css
  • /usr/local/awstats/wwwroot/icon

Do it with these commands.


cp -R /usr/local/awstats/wwwroot/clases /your-html-root-directory
cp -R /usr/local/awstats/wwwroot/css /your-html-root-directory
cp -R /usr/local/awstats/wwwroot/icon /your-html-root-directory

Be sure to replace your-html-root-directory with something like /html_public/root/ or anything you may have as your root directory, and be sure to use the -R options to copy recursively.


Now edit the /etc/httpd/conf/httpd.conf file and add this lines:


ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"
AllowOverride None
Options None
 AuthType Basic
 AuthName 'Private scripts'
 AuthUserFile '/public_html/root/.htpasswd'
 Require valid-user

 Order allow,deny
 Allow from all

And now create a user that will be able to read the stats:


httpasswd /public_html/root/.htpasswd user

Replace /public_html/root/.htpasswd with the file and folder where you want the password file to be in, but be sure the file and upwards folders are owned by the user that runs your Apache server.


Final Steps


Now restart Apache, and start varnishnsca.


varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid

And the command to restart Apache (It is not the same in all distributions).













Comentários

Postar um comentário