Since I’m using Nginx to proxy my AdGuardHome web interface and DNS over HTTPS, the access log files increased rapidly in size. So I’m going to limit these logs with a tool called Logrotate.
Log rotation is a process of removing/storing old logs while not affecting the latest logs. Almost every single application generates meaningful logs, and if not properly handled, logs can eat up all the space on your disk. Logrotate helps to manage logs, compress them, remove unnecessary/old logs or even email them after a certain time period.
Most linux distributions come with logrotate by default. If that’s not the case on your device, just install it with following commands (e.g. Debian/Ubuntu):
sudo apt install logrotate
Logrotate has a configuration file in which we can mention all the files we want to rotate. It needs a time period unit i.e.,
monthly etc., and rotate count i.e.,
5 for each rotation. Log files are rotated count times before being removed. If you set the rotate count to
0, it means old version of logs will be removed rather than rotated. Otherwise, if your logs are being saved in file named myapp.log then after a rotation a new file will be created with name myapp.1.log, and so on.
Logrotate is configured using the main configuration file, or the logrotate configuration folder. the configuration file usually holds the global configuration and the pointer to the configuration folder. In common Linux distributions such as Ubuntu and Debian, the logrotate configuration file can be found:
and the configuration folder can be found:
The configuration folder is usually where you would add new log file configuration which are unique for a particular log file, or set of log files. For example:
- It represents the path where logrotate will monitor the log files with
.log file extension.
- Store logs till 5 rotations, which means it will store maximum 5 files of old logs when rotation hits, and will discard oldest log when there are already 5 files.
- Truncate the original log file to zero size in place after creating a copy, instead of moving the old log file and optionally creating a new one. It can be used when some program cannot be told to close its log file and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost.
missingok indicates if the log file is missing, do not generate an error, and move on the next file.
- Then do not rotate the log if it is empty.
- Old version of logs are compressed.
- Rotate the log file if it exceeds 200Mb, regardless of the rotation time unit.
- Rotation process should happen daily.
Below will explain some commonly used configurations in detail.
Create a new empty template
To create a new logrotate configuration file, you need to create a new file in
/etc/logrotate.d/. You will then need to add a reference to the log files you wish to rotate. This can be directly to a single file or use pattern matching to match a group of log files. The below example matches all log files in
/var/log/myapp/ which have a
You will need to add further commands to this template before it becomes useful.
Rotate based on log file size
size keyword to rotate the log file when it exceeded a given file size. The below example rotates a file when it reaches 10 KB:
Rotate based on time
You can rotate logs using the
daily keyword to create a new log based on duration. The keywords explain them selves, and they can be used in conjunction with the
size keyword to rotate on which ever criteria is met first.
Limit how many log files are kept after rotation by number
rotate keyword allows us to specify how many old, rotated, log files are kept before logrotate deletes them. The
rotate keyword requires an integer to specify the number of old log files.
Limit how many files are kept after rotation by date
You can specify how long to keep rotated files using the
maxage keyword. Any rotated log files which are older then the maxage will be deleted. The below example will keep rotated log files 49 days.
Compress rotated log files
Log files which have been rotated can be compressed to save disk space. Gzip is used by default.
You can change the default gzip compression to another format by specifying the
compresscmd command with a different executable to use. For example, change it to bzip2:
Ignore missing log files
If a log file does not exist when logrotate is running then an error will be thrown. You can use the keyword
missingok to avoid this scenario.
Continue writing to the same file after rotation
Usually when a log file is rotated, the log file is moved to a new location. Some applications may throw an error, and others may continue to write to the relocated file. The
copytruncate keyword copies all the log in the file to a new file and then truncates the original file. This keeps the original log file in place and also allows rotation to continue.
For complete details about logrotate configuration, please refer to
Clean up all log files
If you intend to delete all the log file in
/var/log/, you can use the following script:
# echo empty content to log files
logs=`find /var/log -type f`
for item in $logs
# delete logrotate files
find /var/log -name "*.[0-9]*" -type f -delete