Administrators can now specify a global compression method directly
at the beginning of the newsyslog.conf file, for example:
<compress> none
Relnotes: yes
Reviewed by: dvl
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43174
Historically, newsyslog compressed rotated log files to save disk space.
This was useful in the early days. However, with modern file systems like
ZFS offering native compression, and with the availability of larger hard
drives, the benefits of additional compression have become less significant.
This is particularly true considering the inconvenience of decompressing
log files when searching for specific patterns.
Additionally, the original implementation of compression methods was not
future-proof. As a result, we have redefined the J, X, Y, Z flags to
signify "treat the file as compressible" rather than "compress the file
with that specific method."
A new command-line option, -c, has been introduced to allow overriding
these settings in a more future-proof way. The available choices are:
* none - do not compress, regardless of flag.
* legacy - historical behavior: J=bzip2, X=xz, Y=zstd, Z=gzip.
* bzip2, xz, zstd, gzip - apply the specified compression method.
Currently, the default is set to 'legacy' to preserve historical behavior.
However, our intention is to change this default to 'none' in FreeBSD 15.0.
Additionally, this update changes the default settings for zstd to use
multithreading and long-range options, better aligning with its intended
use.
Inspired by D42961 .
Reviewed by: debdrup (earlier version, mdoc(7))
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43165
Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.
Sponsored by: Netflix
In most cases, usage does not return, so mark them as __dead2. For the
cases where they do return, they have not been marked __dead2.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/735
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
parseDWM() can advance the time to the next week. If the next week is
in the next month, then tm_mon is incremented. However, the increment
was failing to handle the wraparound from December to January, so when
parsing a rule during the last week of the December, the month would
advance to month 12. This triggered an out-of-bounds read of the
mtab[] array in days_pmonth() after parseDWM() returned. To fix,
this change resets the month to January and increment the year when
the month increment wraps.
The default rule for /var/log/weekly.log triggers this during the
last week of December each year.
Reported by: CHERI
Obtained from: CheriBSD
Reviewed by: jhb
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: <https://reviews.freebsd.org/D33687>
This allow one to install it without having to install FreeBSD-utilities.
While here put some newsyslog.d file in their own package.
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33456
Also improve temporary file usage in 200.accounting, add an xref to
zstd(1) to newsyslog.conf.5, and clarify in periodic.conf that
"daily accounting" means process accounting and "monthly accounting"
is login accounting.
PR: 253868
Reviewed by: allanjude
Approved by: blackend (mentor)
Differential Revision: https://reviews.freebsd.org/D29267
Based on an idea from dvl's coworker, László DANIELISZ, implement
a new flag, 'E', that prevents newsyslog(8) from rotating the empty
log files. This 'E' flag ist mostly usable in conjunction with 'B'
flag that instructs newsyslog(8) to not insert an informational
message into the log file after rotation, keeping it still empty.
Reviewed by: markj, ian, manpages (rpokala)
Approved by: markj, ian, manpages (rpokala)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28940
Currently newsyslog supports <include> directive that is used
in our default /etc/newsyslog.conf in the following form:
<include> /usr/local/etc/newsyslog.conf.d/*
While this is suitable for ports installing their own rules
for logs rotation, this also makes newsyslog break entire
processing of all files if it encounters single line it cannot parse.
This includes lines referring to nonexistent username/group for log
ownership, so newsyslog stops calling errx() function in the parser.
With this fix, newsyslog uses warnx() instead of errx() in such cases
to print a warning, recover gracefully and continue with execution.
Among other cases, this unbreaks initial creation of log files
having flag "C" at boot time (newsyslog -CN). This is most important
for systems having RAM-based /var file system like nanobsd(8)-based
that rely on newsyslog to bring system log files into existence.
MFC after: 1 month
While here do a bit of cleanup:
- declare local variables as such,
- make tmpdir_create() clean up logfile directories, to handle a
previously interrupt test run more gracefully.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
This happens when compressing a previously uncompressed already-rotated
file, as happens when handling the 'p' flag in newsyslog.conf. The file
name is stored in a flexible array member, so these structures cannot be
stack allocated.
Also make sure that we call change_attrs() and do_zipwork() in dry-run
mode; they handle this properly, contrary to the commit log message for
r327451.
CID: 1008168
Github PR: https://github.com/freebsd/freebsd/pull/427
MFC after: 2 weeks
Submitted by: Radek Brich (original version)
autofs was introduced with FreeBSD 10.1 and is the supported method for
automounting filesystems. As of r296194 the amd man page claimed that it
is deprecated. Remove it from base now; the sysutils/am-utils port is
still available if necessary.
Discussed with: cy
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
log daemon facility now that daemon(8) has syslog support which defaults to
daemon facility, info priority
Reviewed by: bapt
Approved by: bapt
Differential Revision: https://reviews.freebsd.org/D21561
- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.
- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.
- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.
- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.
PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19105
It was pointed out that a couple of the "memory leak" CIDs that I
fixed were arguably Coverity errors rather than errors in the
newsyslog code and the cure was worse than the disease. Revert both
changes. The first change, which included fixes for other Coverity
errors, will be re-worked to omit the troublesome changes and then
re-committed with the remaining fixes.
Reported by: bde
Sponsored by: Dell EMC Isilon
The result of a strdup() was stored in a global variable and not freed
before program exit. This is a follow-up to r343906. That change
attempted to plug these resource leaks but managed to miss a code path
on which the leak still occurs. Plug the leak on that path, too.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.
- CID 1007454, CID 1007453: Resource leak - The result of a strdup()
was stored in a global variable and not freed before program exit.
- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.
- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.
- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.
PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
We can no longer use sizeof() to get the path buffer's size. Apply
a straightforward fix for now with the aim of MFCing soon.
PR: 233633
Submitted by: Katsuyuki Miyoshi <katsu@miyoshi.matsuyama.ehime.jp>
MFC after: 3 days
The new default config will only include files from the following
directories which end with '.conf' and do not beginning with a '.'
character:
- /etc/newsyslog.conf.d/
- /usr/local/etc/newsyslog.conf.d/
This matches the syslog.conf(5) functionality, and also prevents '.sample' or
'.pkgnew' files being included. This is important for ports which install files
in /usr/local/etc/newsyslog.conf.d/ and also for pkgbase.
Approved by: eadler
Approved by: bapt
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D17086
Prevent some classes of foot-shooting that may result in permissions
problems.
Reviewed by: dab, delphij, vangyzen (earlier version)
Relnotes: yes (behavior change)
Sponsored by: Dell EMC Isilon
Differential Revision: D16831
This is related to pkgbase and changes these to use CONFS so that these are
tagged as config files.
Approved by: AllanJude (mentor)
Sponsored by: Essen Hackathon
Differential Revision: https://reviews.freebsd.org/D16694
- Remove the compression suffix macros and move them directly into the
compress_type array.
- Remove the hardcoded sizes on the suffix and compression args arrays.
- Simplify the compression args arrays at the expense of a __DECONST
when calling execv().
- Rewrite do_zipwork. The COMPRESS_* macros can directly index the
compress_types array, so the outer loop is not needed. Convert
fixed-length strings into asprintf or sbuf calls.
Submitted by: Dan Nelson <dnelson_1901@yahoo.com>
Reviewed by: gad
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16518
The zstd invocation constructed by newsyslog contains one more parameter
than invocations for the other supported compression utilities. However,
the maximum number of arguments was hard-coded, leading to an
out-of-bounds array access when using zstd compression.
Reuse of the index variable in two nested loops resulted in only the first
argument in the list being used (fine for gzip, not fine for zstd). Also
add tests for xz and zstd, and fix the COMPRESS_SUFFIX_MAXLEN macro.
Submitted by: dnelson_1901_yahoo.com
Differential Revision: https://reviews.freebsd.org/D16509
The RFC 5424 spec mentions that logging FQDNs over short hostnames is
preferred. Alter this code, so that the hostname doesn't get truncated
on startup. Keep track of the length of the short hostname, so that
fprintf() can do the truncation where necessary.
MFC after: 1 month
Move the 'X' and 'Y' entries into their sorted location in the list
of flags just above 'Z'.
Reviewed by: bcr
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D13904
Implement the 'p' flag for newsyslog from NetBSD. This flag results in
the first log file for a given file to not be compressed.
While here, don't change file attributes during a no-op run
PR: 162798
Submitted by: heas@shrubbery.net
MFC After: 1 month
When building the command to execute for compression, newsyslog was modifying
the generic arguments array instead of its own copy.
Meaning on the second file to compress with the same arguments, the command line
was not the one expected.
Fix it by creating one copy of the arguments per execution and modifying that
copy.
While here, print the command line executed in verbose mode.
Reported by: many
