July 14, 2015

How to Clean Log Files Before You Enable Log Rotation

Checking disk space usage and removing old log files

cPanel & WHM create and maintain log files. These log files allow you to examine errors and other occurrences on the system.
After an extended period of time, the system's storage devices will begin to fill, as the log files contain more information. You need to delete some old log files or remove some data from the log files to reduce the amount of disk space the log file is using.
You can find the locations of log files using the log file reference graphic or order a free log reference poster from our site!

Checking disk space using the du command

To find out which files and directories consume the most space inside of the /var/log directory, we can use the  du  command. This command will print the estimated disk space usage of each file and directory specified.
1
du -h /var/log/ 

In the example above, the -h argument causes the command to print the information in a human-readable format. When you issue the command in this way, the du utility will print the estimated disk space of each file and directory contained within /var/log.
The output of this command should resemble the following example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
root@host [~]# du -h /var/log/
24K   /var/log/cups
16K   /var/log/mail
36K   /var/log/prelink
19M   /var/log/audit
84K   /var/log/bandwidth/2011/Jun
128K   /var/log/bandwidth/2011/Jan
116K   /var/log/bandwidth/2011/Feb
712K   /var/log/bandwidth/2011
4.5M   /var/log/bandwidth
2.6G   /var/log/munin
8.0K   /var/log/conman.old
8.0K   /var/log/pm
8.0K   /var/log/conman
12K   /var/log/dcpumon/boot.1308161402
5.3M   /var/log/dcpumon
8.0K   /var/log/vbox
3.7G   /var/log/
You should expect more output from the utility than is listed in the example above. Due to the size of the files and directories in /var/log, the du utility can take a long time to finish.
Once we have this information, we can examine the numbers in the far left column. These numbers represent the file size of each file and directory contained within the working directory, in human-readable format.

Removing files using the rm command

To clear some disk space, we will need to remove the offending files. We can use the rm command to do so. This utility removes files from the file system.
1
rm -f $file 

In the example above, the -f argument forces the removal of the file. This means that you will not be prompted to confirm that you wish to remove the file. Use the -f argument to save time. However, you will need to be absolutely certain that you are ready to delete the file.
The $file parameter is the path to the file (or directory) you wish to remove. For our purposes, we will only remove single files, rather than entire directories.
If we examine the output of the du -h /var/log command, we will notice that the largest directory is /var/log/munin. Let's remove one of the munin logs.
1
2
3
4
5
6
7
8
9
10
11
12
13
root@host [~]# cd /var/log/munin/
root@host [/var/log/munin]# du -h
2.7 G.
root@host [/var/log/munin]# du -h *
603M   munin-graph.log
385M   munin-html.log
67M   munin-limits.log
99M   munin-node.log
1.5G   munin-update.log
root@host [/var/log/munin]# rm -f munin-update.log
root@host [/var/log/munin]# du -h
1.2G   .
root@host [/var/log/munin]#  

Let's review what we have done in the example above:
  1. We used the cd command to change directories to /var/log/munin/.
  2. We issued the du command to find out how much disk space /var/log/munin was using. In this case, it was 2.7 gigabytes.
  3. We issued another du command with an asterisk (*). This allowed us to view the disk space usage for each individual file in /var/log/munin. In this case, the largest file was munin-update.log at 1.5 gigabytes.
  4. We then used the rm -f command to remove /var/log/munin/munin-update.log.
  5. Finally, we confirmed that we successfully removed the 1.5-gigabyte log file. To do this, we issued another du command, which confirmed that the directory now uses 1.2 gigabytes of hard disk space.

Freeing up disk space without deleting the log file

If you wish to keep some information from a log file, use the tail command. This command reads and outputs the contents of a file, from the bottom up. This allows you to preserve the most recent information in the log file. You can even specify the number of lines from the file that you wish to keep, and store these lines of data in another file.
1
tail -5000 $file-1 > $file-2 

In the example above, the tail utility takes the last 5000 lines from $file-1 and storesthat information in $file-2.
Once we have created this new file, we can remove the old file ($file-1) from the file system with the rm utility. Then, we can rename the new file.
To rename a file, we can use mv command.
1
mv $old-file $new-file 

Example

In the following example, we will:
  1. Check the disk usage of everything contained within the working directory.
  2. Copy the last 5000 lines from audit.log.2 to audit.log.2.temp
  3. Remove the old audit.log.2 file from the file system.
  4. Rename the newly created audit.log.2.temp to its old file name, audit.log.2
1
2
3
4
5
6
7
8
9
10
11
root@host [/var/log/audit]# du -sh *
1.8M   audit.log
5.1M   audit.log.1
5.1M   audit.log.2
root@host [/var/log/audit]# tail -5000 audit.log.2 > audit.log.2.temp
root@host [/var/log/audit]# rm -f audit.log.2
root@host [/var/log/audit]# mv audit.log.2.temp audit.log.2
root@host [/var/log/audit]# du -h *
1.8M   audit.log
5.1M   audit.log.1
864K   audit.log.2 

We can see from the du utility's output that audit.log.2 has reduced considerably in size, from 5.1 megabytes to 864 kilobytes.

Configuring log rotation

cPanel & WHM's log rotation system compresses and stores old log files in /usr/local/cpanel/logs/archive/. You can enable and configure log rotation in WHM at Home >> Service Configuration >> cPanel Log Rotation Configuration.
The archived log files are stored indefinitely, so you will need to manually remove them. You can use the rm command to remove old log files.

Additional Resources

0 comments:

Post a Comment