How and Why: Enabling Apache’s Piped Logging

[NickTheGreek 160]

Apache by default logs data directly to log files. While this isn’t a bad thing, it is not your only option. Both Apache 1.x and Apache 2.x bring with them the option of enabling something called “Piped Logging”, though cPanel will only allow you to enable it for version 2.x.

Piped logging is extremely powerful when used correctly, and has far more flexibility than what we are using here. The way it is described here, we will be attempting to negate the memory hungry apache processes that creep up when a server is hosting very low traffic websites (less than 1 request per second) with traditional Apache log configurations.
The symptom: Apache processes using a lot of memory.

You will see Apache using a large percent of the memory (MEM) when running a ‘top’, such as what you see below (when sorted by memory use) and you’ll also note that the root Apache process has been running for a long time:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15733 root 15 0 554m 541m 5752 S 0.0 52.9 851:59.84 httpd
17790 www 15 0 556m 544m 5408 S 0.0 53.1 0:02.77 httpd
17616 www 16 0 555m 543m 5440 S 0.0 53.1 0:04.69 httpd
18368 www 15 0 555m 543m 5396 S 0.0 53.1 0:01.05 httpd
29924 www 16 0 555m 543m 5548 S 0.0 53.1 0:08.91 httpd
18363 www 15 0 555m 542m 5352 S 0.0 53.0 0:00.55 httpd
22294 www 15 0 554m 542m 5376 S 3.9 53.0 0:00.27 httpd
22093 www 15 0 555m 541m 4556 S 0.0 52.9 0:00.33 httpd
22232 www 15 0 554m 541m 4552 S 0.0 52.9 0:00.27 httpd

To see if traditional logging is enabled, check your Apache error log for messages that show Apache being restarted around every 2 hours:

[host - root]: grep Graceful /usr/local/apache/logs/error_log

You will see something like this:

[Mon May 31 14:29:55 2010] [notice] Graceful restart requested, doing restart
[Mon May 31 16:43:37 2010] [notice] Graceful restart requested, doing restart
[Mon May 31 18:57:19 2010] [notice] Graceful restart requested, doing restart
[Mon May 31 21:11:02 2010] [notice] Graceful restart requested, doing restart
The Fix!

Enter Piped logging. Enabling piped logging in this way has a few different effects, but the one we are primarily concerned with is preventing Apache from initiating that graceful restart request every two hours.

WARNING: By following these directions your Apache configuration is rebuilt from the existing cPanel templates (the last distilled configuration), so you will lose anything that was not added or configured through cPanel/WHM. The directions below explain how to make a backup of the configuration before rebuilding it.
Pre-implementation:

Note: You will need to have root access to the server in order to implement piped logging.

Software Requirements:
Cpanel Version: 11.25.0-R43471 or later
Apache Version 2 or later

You can check your versions with the following commands:
cPanel:
[host - root]: cat /usr/local/cpanel/version
For Apache:
[host - root]: /usr/local/apache/bin/httpd -v
Implementation

There are two possible ways to implement this fix, one is via the command line, the other is through the WHM.

Method 1: All Command Line

Make a backup of the Apache config:

[host:root]: cp /usr/local/apache/conf/httpd.conf /usr/local/apache/conf/httpd.prepipedlogging

Edit /var/cpanel/cpanel.config:

[host:root]: vi /var/cpanel/cpanel.config

Add the following

enable_piped_logs=1

Make cPanel aware of the change:

[host:root]: /usr/local/cpanel/whostmgr/bin/whostmgr2 --updatetweaksettings

Rebuild the Apache config:

[host:root]: /scripts/rebuildhttpdconf

Stop and Start Apache:

/etc/init.d/httpd stop
/etc/init.d/httpd start

Method 2: Allow cPanel/WHM to do the hard parts

Make a backup of the Apache config:
[host:root]: cp /usr/local/apache/conf/httpd.conf /usr/local/apache/conf/httpd.prepipedlogging

Log into WHM, and follow this sequence to the right place:
Service Configuration >> Apache Configuration >> Piped Log Configuration
Enable piped Apache logging, save it and let it rebuild the configuration.
Finishing up

After making any changes that involve your Apache configuration it is a very good idea to test all your hosted sites to make sure they still work. If it’s not practical to check all of your sites check as many different sites as you can.

You should see a difference in your memory use and your service stability nearly immediately, and it should be long term.