For web servers with large numbers of transactions the logs files can grow and keep growing until become huge, some of them more than 1G so the way we can deal with this (and avoid to run out of disk space) is rotating the log files, zipping the old ones, creating new empty available files and after some time completily delete some logs, we can easily achieve this with (logrotate)
1. Install log rotate:
#>yum install logrotate
2. Create the configuration file (refer below for the complete list of options):
#>vim /etc/logrotate.d/apache
/usr/local/apache2/logs/*log {
size 70M
compress
dateext
notifempty
rotate 7
missingok
sharedscripts
postrotate
/etc/init.d/apachectl restart
endscript
}
so basically here we are specifying the next:
- /usr/local/apache2/logs/*log :
- any of the logs in “/usr/local/apache2/logs/”
- size 70M :
- The files should be rotated when goes bigger than 70M (we can specify based on time, daily, weekly, monthly)
- compress :
- Compress older files with gzip. Opposite: nocompress
- dateext :
- the date will be append in the new filename
- notifempty :
- Don’t do any rotation if the logfile is empty. Opposite: ifempty
- rotate 7 :
- We should keep no more than nn files
- missingok :
- Do not generate an error if the log is missing
- sharedscripts:
- Run any given prerotate or postrotate script for each logfile individually. Opposite: nosharedscripts.
- postrotate:
- Anything between these is executed after the rotation process. Opposite : prerotate
cd /usr/local/apache2/logs/
logrotate -s logstatus /etc/logrotate.d/apache
ls -lah
-rw-r--r-- 1 root root 4.8K Jul 20 2011 access_log
-rw-r--r-- 1 root root 8.2M May 14 22:16 access_log-20120514.gz
-rw-r--r-- 1 root root 0 Jul 20 2011 jsr-access_log
-rw-r--r-- 1 root root 0 Jul 20 2011 jsr-error_log
-rw-r--r-- 1 root root 647 May 14 22:16 logstatus
-rw-r--r-- 1 root root 0 May 14 22:16 members-access_log
-rw-r--r-- 1 root root 4.8M May 14 22:16 members-access_log-20120514.gz
-rw-r--r-- 1 root root 0 May 14 22:16 members-error_log
-rw-r--r-- 1 root root 1.2M May 14 22:16 members-error_log-20120514.gz
Pretty simple… isn’t it? now we’re all set!
List of available options:
compress: Old versions of log files are compressed with gzip by default. See also nocompress.
compresscmd: Specifies which command to use to compress log files. The default is gzip. See also compress.
uncompresscmd: Specifies which command to use to uncompress log files. The default is gunzip.
compressext: Specifies which extension to use on compressed logfiles, if compression is enabled. The default follows that of the configured compression command.
compressoptions: Command line options may be passed to the compression program, if one is in use. The default, for gzip, is “-9″ (maximum compression).
copy: Make a copy of the log file, but don’t change the original at all. This option can be used, for instance, to make a snapshot of the current log file, or when some other utility needs to truncate or pare the file. When this option is used, the createoption will have no effect, as the old log file stays in place.
copytruncate: Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one, It can be used when some program can not be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the createoption will have no effect, as the old log file stays in place.
create: mode owner groupImmediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated). mode specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and group specifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreateoption.
daily: Log files are rotated every day.
delaycompress: Postpone compression of the previous log file to the next rotation cycle. This has only effect when used in combination with compress. It can be used when some program can not be told to close its logfile and thus might continue writing to the previous log file for some time.
extension: extLog files are given the final extension ext after rotation. If compression is used, the compression extension (normally .gz) appears after ext.
ifempty: Rotate the log file even if it is empty, overiding the notifemptyoption (ifempty is the default).
include : file_or_directoryReads the file given as an argument as if it was included inline where the include directive appears. If a directory is given, most of the files in that directory are read in alphabetic order before processing of the including file continues. The only files which are ignored are files which are not regular files (such as directories and named pipes) and files whose names end with one of the taboo extensions, as specified by the tabooext directive. The includedirective may not appear inside of a log file definition.
mail addressWhen a log is rotated out-of-existence, it is mailed to address. If no mail should be generated by a particular log, the nomaildirective may be used.
mailfirstWhen using the mailcommand, mail the just-rotated file, instead of the about-to-expire file.
maillastWhen using the mailcommand, mail the about-to-expire file, instead of the just-rotated file (this is the default).
missingokIf the log file is missing, go on to the next one without issuing an error message. See also nomissingok.
monthlyLog files are rotated the first time logrotateis run in a month (this is normally on the first day of the month).
nocompressOld versions of log files are not compressed with gzip. See also compress.
nocopyDo not copy the original log file and leave it in place. (this overrides the copyoption).
nocopytruncateDo not truncate the original log file in place after creating a copy (this overrides the copytruncateoption).
nocreateNew log files are not created (this overrides the createoption).
nodelaycompressDo not postpone compression of the previous log file to the next rotation cycle (this overrides the delaycompressoption).
nomailDon’t mail old log files to any address.
nomissingokIf a log file does not exist, issue an error. This is the default.
noolddirLogs are rotated in the same directory the log normally resides in (this overrides the olddiroption).
nosharedscriptsRun prerotate and postrotate scripts for every script which is rotated (this is the default, and overrides the sharedscriptsoption).
notifemptyDo not rotate the log if it is empty (this overrides the ifemptyoption).
olddir directoryLogs are moved into directory for rotation. The directory must be on the same physical device as the log file being rotated. When this option is used all old versions of the log end up in directory. This option may be overriden by the noolddiroption.
postrotate/endscriptThe lines between postrotate and endscript (both of which must appear on lines by themselves) are executed after the log file is rotated. These directives may only appear inside of a log file definition. See prerotateas well.
prerotate/endscriptThe lines between prerotate and endscript (both of which must appear on lines by themselves) are executed before the log file is rotated and only if the log will actually be rotated. These directives may only appear inside of a log file definition. See postrotateas well.
firstaction/endscriptThe lines between firstaction and endscript (both of which must appear on lines by themselves) are executed once before all log files that match the wildcarded pattern are rotated, before prerotate script is run and only if at least one log will actually be rotated. These directives may only appear inside of a log file definition. See lastactionas well.
lastaction/endscriptThe lines between lastaction and endscript (both of which must appear on lines by themselves) are executed once after all log files that match the wildcarded pattern are rotated, after postrotate script is run and only if at least one log is rotated. These directives may only appear inside of a log file definition. See lastactionas well.
rotate countLog files are rotated times before being removed or mailed to the address specified in a mail directive. If countis 0, old versions are removed rather then rotated.
size sizeLog files are rotated when they grow bigger then size bytes. If size is followed by M, the size if assumed to be in megabytes. If the k is used, the size is in kilobytes. So size 100, size 100k, and size 100Mare all valid.
sharedscriptsNormally, prescript and postscript scripts are run for each log which is rotated, meaning that a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscriptis specified, the scripts are only run once, no matter how many logs match the wildcarded pattern. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. This option overrides the nosharedscripts option.
start countThis is the number to use as the base for rotation. For example, if you specify 0, the logs will be created with a .0 extension as they are rotated from the original log files. If you specify 9, log files will be created with a .9, skipping 0-8. Files will still be rotated the number of times specified with the countdirective.
tabooext [+] listThe current taboo extension list is changed (see the includedirective for information on the taboo extensions). If a + precedes the list of extensions, the current taboo extension list is augmented, otherwise it is replaced. At startup, the taboo extension list contains .rpmorig, .rpmsave, ,v, .swp, .rpmnew, and ~.
weeklyLog files are rotated if the current weekday is less then the weekday of the last rotation or if more then a week has passed since the last rotation. This is normally the same as rotating logs on the first day of the week, but it works better if logrotate is not run every night.
