Historically, newsyslog compressed rotated log files to save disk space.
This was useful in the early days. However, with modern file systems like
ZFS offering native compression, and with the availability of larger hard
drives, the benefits of additional compression have become less significant.
This is particularly true considering the inconvenience of decompressing
log files when searching for specific patterns.
Additionally, the original implementation of compression methods was not
future-proof. As a result, we have redefined the J, X, Y, Z flags to
signify "treat the file as compressible" rather than "compress the file
with that specific method."
A new command-line option, -c, has been introduced to allow overriding
these settings in a more future-proof way. The available choices are:
* none - do not compress, regardless of flag.
* legacy - historical behavior: J=bzip2, X=xz, Y=zstd, Z=gzip.
* bzip2, xz, zstd, gzip - apply the specified compression method.
Currently, the default is set to 'legacy' to preserve historical behavior.
However, our intention is to change this default to 'none' in FreeBSD 15.0.
Additionally, this update changes the default settings for zstd to use
multithreading and long-range options, better aligning with its intended
use.
Inspired by D42961 .
Reviewed by: debdrup (earlier version, mdoc(7))
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43165
Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.
Sponsored by: Netflix
The new default config will only include files from the following
directories which end with '.conf' and do not beginning with a '.'
character:
- /etc/newsyslog.conf.d/
- /usr/local/etc/newsyslog.conf.d/
This matches the syslog.conf(5) functionality, and also prevents '.sample' or
'.pkgnew' files being included. This is important for ports which install files
in /usr/local/etc/newsyslog.conf.d/ and also for pkgbase.
Approved by: eadler
Approved by: bapt
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D17086
This modification adds the capability to newsyslog to write the
rotation message in a format that is compliant with RFC5424. This
capability is enabled on a per-log file basis through a new value
("T") in the flags field in newsyslog.conf. This is useful on systems
that use the RFC5424 format for log files so that the rotation message
format matches that of the other log messages. There has been recent
mention of adding an RFC5424 compliant mode to syslogd and at least
one alternative system log daemon (rsyslogd) that already has the
capability to use that format.
Reviewed by: vangyzen, ngie
Approved by: vangyzen (mentor)
MFC after: 2 months
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10253
This makes newsyslog use zstandard to compress log files.
Given Z is already taken for gzip and zstandard compression level stands in
between gzip and xz (which has the X flag) chosing Y sounds ok :)
default for newsyslog(8).
The /usr/local/etc/newsyslog.conf.d will give packages an opportunity to
install a default configuration to handle their own log files.
MFC after: 2 weeks
Relnotes: yes
instead of the traditional simple counter.
Using the time-stamp based file-names, once a log file is archived, it
will not change name until it is deleted. This means that many backup
systems will only perform one backup of the archived log file, instead
for performing a new backup of the logfile upon each logfile rotation.
This implementation is separate from the patches in the mentioned PR,
as I wasn't aware of the existence of the PR until after I had
implemented the same functionality as the patches in the PR provide.
Unlike the PR, this new code does honor the 'log count' in
newsyslog.conf so old logfiles are deleted. This new code does not
currently support never deleting the archived logfiles.
PR: bin/29363
MFC after: 3 weeks
which stops to proceed further, as it is possible that processes which
fails to create PID file get screwed by rotation.
Requested by: stas
MFC after: 2 weeks
X-MFC with: r200806
cojunction with -C and is used by /etc/rc.d/newsyslog.
I forgot that this was in my perforce tree and not my running system and
thus committed a non-working newsyslog script.
Reported by: des
Pointy hat: brooks
config file. If the -C option is specified once, then newsyslog will create
any entries which specify the 'C' option. If -C is given twice, then
newsyslog will create all missing log files. Some of this code comes
from NetBSD, although this implementation does not exactly match theirs.
Reviewed by: freebsd-arch
MFC after: 10 days
the 'N' flag. These were coded in March as revisions 1.55 and 1.56
of newsyslog.c. I intend to MFC all the matching changes next week.
This also reorganizes the description of the 'flags' field to give
list of the valid flags, instead of a long paragraph explaining
each of the possible values.
Obtained from: NetBSD (in spirit at least, for -s and N)
MFC after: 1 week
`J' flag means that newsyslog should wait for previously started compression
jobs to complete before starting a new one for this entry. When it is used
along with the `G' flag, in the case when several logfiles match the pattern
and should be compressed, the newsyslog will compress logs one by one,
ensuring that only one compression job is running at a time.
This prevents newsyslog(8) from overloading system by starting several
compression jobs on big files simultaneously.
Sponsored by: Porta Software Ltd
MFC after: 2 weeks
the specified filename of the log to be rotated is in fact shell glob
pattern. In this case, all files matching this pattern will be rotated
using the same options. Useful in the case when there is no pre-defined
name for the logfiles (e.g. xtradius, samba etc).
Sponsored by: PortaOne Software Ltd
MFC after: 2 weeks
give an example of how to rotate logs at the beginning of the month.
Although they sound the same, since both of them rotate logs at the
beginning of the day, the former ended up taking place on, e.g., July
31 00:00 instead of the expected July 31 23:59. This is contraty to POLA.
Submitted by: Dan Langille <dan@langille.org>
configurable directory
- implement alternate and more flexible way to specify
logfile rotation time in addition to the ISO 8601
restricted format
- cleanup the source which was a mix of several styles
of persons who maintained it so far, ran through
knfom script got from bde.
Reviewed by: (in part) sheldonh and garyj