How to pipe logs from Apache to PHP / How to seperate Apache log files with PHP

I recently needed to do this for a client to customise and merge the logged data from Apache into a presentable format. A by-product of this was the ability to also roll the log file on a daily basis.I found limited information regarding how to do this most of which were methods that were faulty or out dated. So to help out fellow Googlers and spread light on a few PHP functions that aren’t commonly used I thought I’d put up a brief guide on how to do it. My test and live systems are running PHP 5.2.5 and Apache 2.0.61 (which is compiled in a fairly basic configuration), however all of this code is PHP 4 and Apache 1.3 friendly (as far as I’ve checked).

First of all we need to create the script. Unlike PHP being executed in the normal method via Apache we need to execute the script as a shell script. This line of code placed at the top of your PHP script will tell your Unix box what to try and execute this script with when you call it from the shell or another program. The location that this ‘link’ refers to may be different on your system, the link actually refers to the location of your install of PHP.

#!/usr/local/bin/php

The next part of code within this script contains a few functions that help us handle file activities.

function make_file($filename){
$f = fopen($filename, 'w') or die("can't open file");
fclose($f);
}

function append_file($filename,$newdata) {
$f=fopen($filename,"a");
fwrite($f,$newdata);
fclose($f);
}

The next portion of code sets up the files to store the logs. The filename is generated based on the date, therefore allowing daily logging.

$myFile = "/home/myuser/logs/mydomain.co.uk-".date("d-m-Y").".txt";

if(!is_file($myFile)){
make_file($myFile);
}

All of the code above is fairly straight forward and commonly used. Now for the interesting part, picking up the piped data from Apache.

$fd = fopen("php://stdin", "r");

This opens the ‘stream’ of piped data being sent to PHP, note the same method as opening a read-only file. We now have the data that we’re being sent from Apache, time to log it!

// Setup blank var for the data being recieved
$data = "";

//Create a new file handler and open the read only stream being sent from Apache
$fd = fopen("php://stdin", "r");

//Characters we would like to remove from the piped lines
$strip = array(" ","r","n");

// For each loop that will explode the piped data by line and add it to our data var
foreach (explode("n", fread($fd,8192)) as $l) {

    //If the line is not empty add it to our data var
    if (str_replace($strip,"",$l) !== "") {

        // Adding the data to the var
        $data .= trim($l)."r";

    }

}

//Close the file handler
fclose($fd);

The last part of the code appends the data in our var into the file we specified.

append_file($myFile,$data);

All we have left to do is change the scripts permissons so it is executable. Easiest way to do this is by logging into the servers console and running the following command on your script:

chmod +x /home/myuser/myscript.php

Thats it for the PHP script, onto the Apache configuration. We need to add a ‘CustomLog’ directive to the Apache conf file. Below is a copy of an example virtual host taken from a default httpd.conf file with the directive added. (Note: The example below is missing the leading and trailing VirtualHost tags used to define a virtual host within the Apache config file.)

ServerAlias http://www.mydomain.com/
ServerAdmin webmaster@mydomain.com
DocumentRoot /home/myuser/public_html
BytesLog domlogs/mydomain.com-bytes_log
ServerName www.mydomain.com
User myuser
Group myuser
CustomLog "|/home/myuser/processlogs.php" combined
ScriptAlias /cgi-bin/ /home/myuser/public_html/cgi-bin/

The line you are looking at is:

CustomLog "|/home/myuser/processlogs.php" combined

The path to your script will differ. This is the line you will need to add to your Apache conf. file. If your not sure where this is use to help find it.Now everytime Apache serves a request it calls this script and pipes the request data to our script for logging. Our script could be setup for weekly / monthly logging by changing the logic for the file creation.

I have uploaded a zipped copy of the script for you to download to help get started. If anyone needs help, register to post and then leave a comment for this post. If you have views or found this article helpful please spare a minute to leave some feedback.

You can download the example script here.



One thought on “How to pipe logs from Apache to PHP / How to seperate Apache log files with PHP”

  1. neilmcaliece says:

    Hi, I found your script useful.

    Although this script appears to offer me what I am looking for – a way to process the apache logs as they happen I notice that it gets called multiple times.

    I have come across a system which allows a single process to cnotinuously process the apache logfile.

    This is what I am really looking for – any idea how to do this without running multiple processes all the time ?

    Neil

Leave a Reply

Your email address will not be published. Required fields are marked *