舫摘

知人者智 自知者明 胜人者有力 自胜者强

0%

使用logrotate管理Linxu日志

日志文件包含了关于系统中发生的事件的有用信息,在排障过程中或者系统性能分析时经常被用到。对于忙碌的服务器,日志文件大小会增长极快,服务器会很快消耗磁盘空间,这成了个问题。除此之外,处理一个单个的庞大日志文件也常常是件十分棘手的事。

logrotate是个十分有用的工具,它可以自动对日志进行截断(或轮循)、压缩以及删除旧的日志文件。例如,你可以设置logrotate,让/var/log/foo日志文件每30天轮循,并删除超过6个月的日志。配置完后,logrotate的运作完全自动化,不必进行任何进一步的人为干预。另外,旧日志也可以通过电子邮件发送,不过该选项超出了本教程的讨论范围。

主流Linux发行版上都默认安装有logrotate包,如果出于某种原因,logrotate没有出现在里头,你可以使用apt-get或yum命令来安装。

在Debian或Ubuntu上:

1
# apt-get install logrotate cron

在Fedora,CentOS或RHEL上:

1
# yum install logrotate crontabs

logrotate的配置文件是/etc/logrotate.conf,通常不需要对它进行修改。日志文件的轮循设置在独立的配置文件中,它(们)放在/etc/logrotate.d/目录下。

配置文件参数:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
compress
Old versions of log files are compressed with gzip(1) by
default. See also nocompress.

compresscmd
Specifies which command to use to compress log files. The
default is gzip(1). See also compress.

uncompresscmd
Specifies which command to use to uncompress log files. The
default is gunzip(1).

compressext
Specifies which extension to use on compressed logfiles, if
compression is enabled. The default follows that of the
configured compression command.

compressoptions
Command line options may be passed to the compression program,
if one is in use. The default, for gzip(1), is "-6" (biased
towards high compression at the expense of speed). If you use a
different compression command, you may need to change the
compressoptions to match.

copy
Make a copy of the log file, but don't change the original at
all. This option can be used, for instance, to make a snapshot
of the current log file, or when some other utility needs to
truncate or parse the file. When this option is used, the
create option will have no effect, as the old log file stays in
place.

copytruncate
Truncate the original log file to zero size in place after
creating a copy, instead of moving the old log file and
optionally creating a new one. It can be used when some program
cannot be told to close its logfile and thus might continue
writing (appending) to the previous log file forever. Note that
there is a very small time slice between copying the file and
truncating it, so some logging data might be lost. When this
option is used, the create option will have no effect, as the
old log file stays in place.

create mode owner group, create owner group
Immediately after rotation (before the postrotate script is run)
the log file is created (with the same name as the log file just
rotated). mode specifies the mode for the log file in octal
(the same as chmod(2)), owner specifies the user name who will
own the log file, and group specifies the group the log file
will belong to. Any of the log file attributes may be omitted,
in which case those attributes for the new file will use the
same values as the original log file for the omitted attributes.
This option can be disabled using the nocreate option.

createolddir mode owner group
If the directory specified by olddir directive does not exist,
it is created. mode specifies the mode for the olddir directory
in octal (the same as chmod(2)), owner specifies the user name
who will own the olddir directory, and group specifies the group
the olddir directory will belong to. This option can be disabled
using the nocreateolddir option.

daily
Log files are rotated every day.

dateext
Archive old versions of log files adding a date extension like
YYYYMMDD instead of simply adding a number. The extension may be
configured using the dateformat and dateyesterday options.

dateformat format_string
Specify the extension for dateext using the notation similar to
strftime(3) function. Only %Y %m %d %H %M %S %V and %s
specifiers are allowed. The default value is -%Y%m%d except
hourly, which uses -%Y%m%d%H as default value. Note that also
the character separating log name from the extension is part of
the dateformat string. The system clock must be set past Sep 9th
2001 for %s to work correctly. Note that the datestamps
generated by this format must be lexically sortable (i.e., first
the year, then the month then the day. e.g., 2001/12/01 is ok,
but 01/12/2001 is not, since 01/11/2002 would sort lower while
it is later). This is because when using the rotate option,
logrotate sorts all rotated filenames to find out which logfiles
are older and should be removed.

dateyesterday
Use yesterday's instead of today's date to create the dateext
extension, so that the rotated log file has a date in its name
that is the same as the timestamps within it.

delaycompress
Postpone compression of the previous log file to the next
rotation cycle. This only has effect when used in combination
with compress. It can be used when some program cannot be told
to close its logfile and thus might continue writing to the
previous log file for some time.

extension ext
Log files with ext extension can keep it after the rotation. If
compression is used, the compression extension (normally .gz)
appears after ext. For example you have a logfile named
mylog.foo and want to rotate it to mylog.1.foo.gz instead of
mylog.foo.1.gz.

hourly Log files are rotated every hour. Note that usually logrotate is
configured to be run by cron daily. You have to change this
configuration and run logrotate hourly to be able to really
rotate logs hourly.

addextension ext
Log files are given the final extension ext after rotation. If
the original file already ends with ext, the extension is not
duplicated, but merely moved to the end, i.e. both filename and
filenameext would get rotated to filename.1ext. If compression
is used, the compression extension (normally .gz) appears after
ext.

ifempty
Rotate the log file even if it is empty, overriding the
notifempty option (ifempty is the default).

include file_or_directory
Reads the file given as an argument as if it was included inline
where the include directive appears. If a directory is given,
most of the files in that directory are read in alphabetic order
before processing of the including file continues. The only
files which are ignored are files which are not regular files
(such as directories and named pipes) and files whose names end
with one of the taboo extensions or patterns, as specified by
the tabooext or taboopat directives, respectively.

mail address
When a log is rotated out of existence, it is mailed to address.
If no mail should be generated by a particular log, the nomail
directive may be used.

mailfirst
When using the mail command, mail the just-rotated file, instead
of the about-to-expire file.

maillast
When using the mail command, mail the about-to-expire file,
instead of the just-rotated file (this is the default).

minage count
Do not rotate logs which are less than <count> days old.

maxage count
Remove rotated logs older than <count> days. The age is only
checked if the logfile is to be rotated. The files are mailed to
the configured address if maillast and mail are configured.

maxsize size
Log files are rotated when they grow bigger than size bytes even
before the additionally specified time interval (daily, weekly,
monthly, or yearly). The related size option is similar except
that it is mutually exclusive with the time interval options,
and it causes log files to be rotated without regard for the
last rotation time. When maxsize is used, both the size and
timestamp of a log file are considered.

minsize size
Log files are rotated when they grow bigger than size bytes, but
not before the additionally specified time interval (daily,
weekly, monthly, or yearly). The related size option is similar
except that it is mutually exclusive with the time interval
options, and it causes log files to be rotated without regard
for the last rotation time. When minsize is used, both the size
and timestamp of a log file are considered.

missingok
If the log file is missing, go on to the next one without
issuing an error message. See also nomissingok.

monthly
Log files are rotated the first time logrotate is run in a month
(this is normally on the first day of the month).

nocompress
Old versions of log files are not compressed. See also compress.

nocopy Do not copy the original log file and leave it in place. (this
overrides the copy option).

nocopytruncate
Do not truncate the original log file in place after creating a
copy (this overrides the copytruncate option).

nocreate
New log files are not created (this overrides the create
option).

nocreateolddir
olddir directory is not created by logrotate when it does not
exist.

nodelaycompress
Do not postpone compression of the previous log file to the next
rotation cycle (this overrides the delaycompress option).

nodateext
Do not archive old versions of log files with date extension
(this overrides the dateext option).

nomail
Do not mail old log files to any address.

nomissingok
If a log file does not exist, issue an error. This is the
default.

noolddir
Logs are rotated in the directory they normally reside in (this
overrides the olddir option).

nosharedscripts
Run prerotate and postrotate scripts for every log file which is
rotated (this is the default, and overrides the sharedscripts
option). The absolute path to the log file is passed as first
argument to the script. If the scripts exit with error, the
remaining actions will not be executed for the affected log
only.

noshred
Do not use shred when deleting old log files. See also shred.

notifempty
Do not rotate the log if it is empty (this overrides the ifempty
option).

olddir directory
Logs are moved into directory for rotation. The directory must
be on the same physical device as the log file being rotated,
unless copy, copytruncate or renamecopy option is used. The
directory is assumed to be relative to the directory holding the
log file unless an absolute path name is specified. When this
option is used all old versions of the log end up in directory.
This option may be overridden by the noolddir option.

postrotate/endscript
The lines between postrotate and endscript (both of which must
appear on lines by themselves) are executed (using /bin/sh)
after the log file is rotated. These directives may only appear
inside a log file definition. Normally, the absolute path to the
log file is passed as first argument to the script. If
sharedscripts is specified, whole pattern is passed to the
script. See also prerotate. See sharedscripts and
nosharedscripts for error handling.

prerotate/endscript
The lines between prerotate and endscript (both of which must
appear on lines by themselves) are executed (using /bin/sh)
before the log file is rotated and only if the log will actually
be rotated. These directives may only appear inside a log file
definition. Normally, the absolute path to the log file is
passed as first argument to the script. If sharedscripts is
specified, whole pattern is passed to the script. See also
postrotate. See sharedscripts and nosharedscripts for error
handling.

firstaction/endscript
The lines between firstaction and endscript (both of which must
appear on lines by themselves) are executed (using /bin/sh) once
before all log files that match the wildcarded pattern are
rotated, before prerotate script is run and only if at least one
log will actually be rotated. These directives may only appear
inside a log file definition. Whole pattern is passed to the
script as first argument. If the script exits with error, no
further processing is done. See also lastaction.

lastaction/endscript
The lines between lastaction and endscript (both of which must
appear on lines by themselves) are executed (using /bin/sh) once
after all log files that match the wildcarded pattern are
rotated, after postrotate script is run and only if at least one
log is rotated. These directives may only appear inside a log
file definition. Whole pattern is passed to the script as first
argument. If the script exits with error, just an error message
is shown (as this is the last action). See also firstaction.

preremove/endscript
The lines between preremove and endscript (both of which must
appear on lines by themselves) are executed (using /bin/sh) once
just before removal of a log file. The logrotate will pass the
name of file which is soon to be removed. See also firstaction.

rotate count
Log files are rotated count times before being removed or mailed
to the address specified in a mail directive. If count is 0, old
versions are removed rather than rotated. Default is 0.

renamecopy
Log file is renamed to temporary filename in the same directory
by adding ".tmp" extension to it. After that, postrotate script
is run and log file is copied from temporary filename to final
filename. This allows storing rotated log files on the different
devices using olddir directive. In the end, temporary filename
is removed.

size size
Log files are rotated only if they grow bigger than size bytes.
If size is followed by k, the size is assumed to be in
kilobytes. If the M is used, the size is in megabytes, and if G
is used, the size is in gigabytes. So size 100, size 100k, size
100M and size 100G are all valid.

sharedscripts
Normally, prerotate and postrotate scripts are run for each log
which is rotated and the absolute path to the log file is passed
as first argument to the script. That means a single script may
be run multiple times for log file entries which match multiple
files (such as the /var/log/news/* example). If sharedscripts is
specified, the scripts are only run once, no matter how many
logs match the wildcarded pattern, and whole pattern is passed
to them. However, if none of the logs in the pattern require
rotating, the scripts will not be run at all. If the scripts
exit with error, the remaining actions will not be executed for
any logs. This option overrides the nosharedscripts option and
implies create option.

shred Delete log files using shred -u instead of unlink(). This
should ensure that logs are not readable after their scheduled
deletion; this is off by default. See also noshred.

shredcycles count
Asks GNU shred(1) to overwrite log files count times before
deletion. Without this option, shred's default will be used.

start count
This is the number to use as the base for rotation. For example,
if you specify 0, the logs will be created with a .0 extension
as they are rotated from the original log files. If you specify
9, log files will be created with a .9, skipping 0-8. Files
will still be rotated the number of times specified with the
rotate directive.

su user group
Rotate log files set under this user and group instead of using
default user/group (usually root). user specifies the user name
used for rotation and group specifies the group used for
rotation. If the user/group you specify here does not have
sufficient privilege to make files with the ownership you've
specified in a create instruction, it will cause an error.

tabooext [+] list
The current taboo extension list is changed (see the include
directive for information on the taboo extensions). If a +
precedes the list of extensions, the current taboo extension
list is augmented, otherwise it is replaced. At startup, the
taboo extension list contains .rpmsave, .rpmorig, ~, .disabled,
.dpkg-old, .dpkg-dist, .dpkg-new, .cfsaved, .ucf-old, .ucf-dist,
.ucf-new, .rpmnew, .swp, .cfsaved, .rhn-cfg-tmp-*

taboopat [+] list
The current taboo glob pattern list is changed (see the include
directive for information on the taboo extensions and patterns).
If a + precedes the list of patterns, the current taboo pattern
list is augmented, otherwise it is replaced. At startup, the
taboo pattern list is empty.

weekly Log files are rotated if the current weekday is less than the
weekday of the last rotation or if more than a week has passed
since the last rotation. This is normally the same as rotating
logs on the first day of the week, but it works better if
logrotate is not run every night.

yearly Log files are rotated if the current year is not the same as the
last rotation.

样例一

在第一个样例中,我们将创建一个10MB的日志文件/var/log/log-file。我们将展示怎样使用logrotate来管理该日志文件。

我们从创建一个日志文件开始吧,然后在其中填入一个10MB的随机比特流数据。

1
2
# touch /var/log/log-file
# head -c 10M < /dev/urandom > /var/log/log-file

由于现在日志文件已经准备好,我们将配置logrotate来轮循该日志文件。让我们为该文件创建一个配置文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# vim /etc/logrotate.d/log-file 

/var/log/log-file {
monthly
rotate 5
compress
delaycompress
missingok
notifempty
create 644 root root
postrotate
/usr/bin/killall -HUP rsyslogd
endscript
}

上面的模板是通用的,而配置参数则根据你的需求进行调整,不是所有的参数都是必要的。

样例二

在本例中,我们只想要轮循一个日志文件,然而日志文件大小可以增长到50MB。

1
2
3
4
5
6
7
8
9
10
# vim /etc/logrotate.d/log-file 

/var/log/log-file {
size=50M
rotate 5
create 644 root root
postrotate
/usr/bin/killall -HUP rsyslogd
endscript
}

样例三

我们想要让旧日志文件以创建日期命名,这可以通过添加dateext参数实现。

1
2
3
4
5
6
7
8
9
10
11
# vim /etc/logrotate.d/log-file 

/var/log/log-file {
monthly
rotate 5
dateext
create 644 root root
postrotate
/usr/bin/killall -HUP rsyslogd
endscript
}

这将让归档文件在它们的文件名中包含日期信息。

排障

这里提供了一些logrotate设置的排障提示。

手动运行logrotate

logrotate可以在任何时候从命令行手动调用。

要调用为/etc/lograte.d/下配置的所有日志调用logrotate:

1
# logrotate /etc/logrotate.conf

要为某个特定的配置调用logrotate:

1
# logrotate /etc/logrotate.d/log-file

演练

排障过程中的最佳选择是使用-d选项以预演方式运行logrotate。要进行验证,不用实际轮循任何日志文件,可以模拟演练日志轮循并显示其输出。

1
2
3
4
5
6
7
8
9
10
11
12
13
# logrotate -d /etc/logrotate.d/log-file 

root@ubuntu:-# logrotate -d /etc/logrotate.d/log-file
reading config file /etc/logrotate.d/log-file
reading config info for /var/log/log-flie

Handling 1 logs

rotating pattern: /var/log/log-file monthly (5 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/log-file
log does not need rotating
not running postrotate script, since no logs were rotated

正如我们从上面的输出结果可以看到的,logrotate判断该轮循是不必要的。如果文件的时间小于一天,这就会发生了。

强制轮循

即使轮循条件没有满足,我们也可以通过使用-f选项来强制logrotate轮循日志文件,-v参数提供了详细的输出。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# logrotate -vf /etc/logrotate.d/log-file 

reading config file /etc/logrotate.d/log-file
reading config info for /var/log/log-file

Handling 1 logs

rotating pattern: /var/log/log-file forced from command line (5 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/log-file
log needs rotating
rotating log /var/log/log-file, log->rotateCount is 5
dateext suffix '-20140916'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/log-file.5.gz to /var/log/log-file.6.gz (rotatecount 5, logstart 1, i 5),
old log /var/log/log-file.5.gz does not exist
renaming /var/log/log-file.4.gz to /var/log/log-file.5.gz (rotatecount 5, logstart 1, i 4),
old log /var/log/log-file.4.gz does not exist
. . .
renaming /var/log/log-file.0.gz to /var/log/log-file.1.gz (rotatecount 5, logstart 1, i 0),
old log /var/log/log-file.0.gz does not exist
log /var/log/log-file.6.gz doesn't exist -- won't try to dispose of it
renaming /var/log/log-file to /var/log/log-file.1
creating new /var/log/log-file mode = 0644 uid = 0 gid = 0
running postrotate script
compressing log with: /bin/gzip

Logrotate的记录日志

logrotate自身的日志通常存放于/var/lib/logrotate/status目录。如果处于排障目的,我们想要logrotate记录到任何指定的文件,我们可以指定像下面这样从命令行指定。

1
# logrotate -vf –s /var/log/logrotate-status /etc/logrotate.d/log-file

Logrotate定时任务

logrotate需要的cron任务应该在安装时就自动创建了,我把cron文件的内容贴出来,以供大家参考。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# cat /etc/cron.daily/logrotate 

#!/bin/sh

# Clean non existent log file entries from status file
cd /var/lib/logrotate
test -e status || touch status
head -1 status > status.clean
sed 's/"//g' status | while read logfile date
do
[ -e "$logfile" ] && echo "\"$logfile\" $date"
done >> status.clean
mv status.clean status

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

小结一下,logrotate工具对于防止因庞大的日志文件而耗尽存储空间是十分有用的。配置完毕后,进程是全自动的,可以长时间在不需要人为干预下运行。本教程重点关注几个使用logrotate的几个基本样例,你也可以定制它以满足你的需求。