In copy_file(), make sure the from_fd file descriptor is closed even
when the operation failed early.
Reported by: Coverity Scan
CID: 1545036
Sponsored by: The FreeBSD Foundation
Reviewed by: imp, emaste
Pull Request: https://github.com/freebsd/freebsd-src/pull/1238
Once we've successfully opened the file we've been asked to copy, check
that it's of the same type as FTS told us it was.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: allanjude, markj
Differential Revision: https://reviews.freebsd.org/D44806
Historically, BSD cp has followed symbolic links in the destination
when copying recursively, while GNU cp has not. POSIX is somewhat
vague on the topic, but both interpretations are within bounds. In
33ad990ce9, cp was changed to apply the same logic for symbolic
links in the destination as for symbolic links in the source: follow
if not recursing (which is moot, as this situation can only arise
while recursing) or if the `-L` option was given. There is no support
for this in POSIX. We can either switch back, or go all the way.
Having carefully weighed the kind of trouble you can run into by
following unexpected symlinks up against the kind of trouble you can
run into by not following symlinks you expected to follow, we choose
to go all the way.
Note that this means we need to stat the destination twice: once,
following links, to check if it is or references the same file as the
source, and a second time, not following links, to set the dne flag
and determine the destination's type.
While here, remove a needless complication in the dne logic. We don't
need to explicitly reject overwriting a directory with a non-directory,
because it will fail anyway.
Finally, add test cases for copying a directory to a symlink and
overwriting a directory with a non-directory.
MFC after: never
Relnotes: yes
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D44578
* When copying a directory, if the destination exists and is not a
directory, we would previously emit an error message and exit. The
correct behavior according to POSIX is to emit an error message and
continue without descending further into the source directory.
* When copying a directory, if the destination does not exist and we
fail to create it, we would previously emit an error message and
exit. The correct behavior according to POSIX is to emit an error
message and continue. Whether to descend further into the source
directory is explicitly left unspecified; GNU cp does not, which
seems to me to be the safer and less surprising option, so let's not
either.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D44577
Both wcount and wresid are ssize_t so this cast is not needed. Just
remove it so the code is easier to read.
Signed-off-by: Collin Funk <collin.funk1@gmail.com>
Reviewed by: emaste, zlei
Pull Request: https://github.com/freebsd/freebsd-src/pull/1116
This accidentally got left out of 0f4467ce44.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans, allanjude
Differential Revision: https://reviews.freebsd.org/D43067
If the destination file exists but we decide unlink it, set the dne
flag. This means we don't need to re-check the conditions that would
have caused us to delete the file when we later need to decide whether
to create or replace it.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43064
- The HLPR flags are grouped together at the beginning because they are
the standard flags for programs using FTS. Move the N flag out from
among them to its correct place in the sequence.
- The Pflag variable isn't used outside main(), but moving it out lets
us skip initialization and keeps it with its friends H, L and R.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43063
This test case tests two different things: first, that copying a symlink
results in a file with the same contents as the target of the symlink,
rather than a second symlink, and second, that cp will refuse to copy a
file to itself, or to a link to itself, or a link to its target. Leave
the first part in basic_symlink, move the second part to a new test case
named samefile, and slightly expand both cases.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43062
Rewrite `copy_file()` so the lflag and sflag are handled as early as
possible instead of constantly checking that they're not set and then
handling them at the end. This also opens the door to changing the
failure logic at some future point (for instance, we might decide to
fall back to copying if `errno` indicates that the file system does not
support links).
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans, allanjude
Differential Revision: https://reviews.freebsd.org/D43055
From NetBSD's utils.c 1.5 importing importing BSDI change, with light
formatting changes:
Author: cgd <cgd@NetBSD.org>
Date: Wed Feb 26 14:40:51 1997 +0000
Patch from BSDI (via Keith Bostic):
>NFS doesn't support chflags; ignore errors unless there's reason
>to believe we're losing bits. (Note, this still won't be right
>if the server supports flags and we were trying to *remove* flags
>on a file that we copied, i.e., that we didn't create.)
CVS Info: utils.c 1.6
Obtained from: NetBSD
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D42674
Add -N to supress copying of file flags when -p is specified (explicitly
or implicitly). Often times we don't care about the flags or wish to be
able to copy to NFS, and this comes in handy for that. FreeBSD's and
NetBSD's cp are somewhat different, so I had to reimplement all but one
of the patch hunks...
Obtained from: NetBSD (cp.1 1.25, cp.c 1.37, utils.c 1.28 by elad)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D42673
Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.
Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/
Sponsored by: Netflix
We've ifdef'd out the copyright strings for some time now. Go ahead and
remove the ifdefs. Plus whatever other detritis was left over from other
recent removals. These copyright strings are present in the comments and
are largely from CSRG's attempt at adding their copyright to every
binary file (which modern interpretations of the license doesn't
require).
Sponsored by: Netflix
Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.
Sponsored by: Netflix
In most cases, usage does not return, so mark them as __dead2. For the
cases where they do return, they have not been marked __dead2.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/735
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
This reverts commit 6433365490.
The error is not valid per api contract, it showed up as a regression
after 15f0b8c30915f0b8c309 ("zfs: merge openzfs/zfs@9cd71c860 (master)") and was
subsequently in d012836fb6 ("zfs: fix up EXDEV handling for
clone_range").
Sponsored by: Rubicon Communications, LLC ("Netgate")
* The sparsity check was ineffective: it compared the apparent size in bytes to the actual size in blocks. Instead, write a tool that reliably detects sparseness.
* Some of the seq commands were missing an argument.
* Based on empirical evidence, 1 MB holes are not necessarily large enough to be preserved by the underlying filesystem. Increase the hole size to 16 MB.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: cracauer
Differential Revision: https://reviews.freebsd.org/D38414
* Fix includes in utils.c, cf. style(9).
* Fix type mismatch: readlink(2) returns ssize_t, not int.
* It is not necessary to set errno to 0 as fts_read(3) already does it.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D38369
* The allocated buffer is only used in the fallback case, so move it
there. The argument for passing it in from the caller was that if
malloc(3) were to fail, we'd want it to fail before we started
copying anything, but firstly, it was already not in the right place
to ensure that, and secondly, malloc(3) never fails (except in very
contrived circumstances, such as an unreasonable RLIMIT_AS or
RLIMIT_DATA).
* Remove the mmap(2) option. It is almost never beneficial,
especially when the alternative is copy_file_range(2), and it adds
needless complexity and indentation.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: rmacklem, mav
Differential Revision: https://reviews.freebsd.org/D38291
The correct logic is a lot simpler than the previous iteration. We
record the base fts_name to avoid having to worry about whether we
needed the root symlink name or not (as applicable), then we can simply
shift all of that logic to after path translation to make it less
fragile.
If we're copying to DNE, then we'll have swapped out the NULL root_stat
pointer and then attempted to recurse on it. The previously nonexistent
directory shouldn't exist at all in the new structure, so just back out
from that tree entirely and move on.
The tests have been amended to indicate our expectations better with
subdirectory recursion. If we copy A to A/B, then we expect to copy
everything from A/B/* into A/B/A/B, with exception to the A that we
create in A/B.
Reviewed by: bapt
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34655
According to POSIX, cp should allow the `-P` flag to work whether `-R`
is specified or not. Currently, the `-P` option only works along with
`-R`.
PR: 199466
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D30012
The traversal was previously not properly honoring -H/-L/-P. Notably,
we should not have been resolving symlinks encountered during traversal
when either -H or -P are specified.
Sponsored by: Klara, Inc.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D34063
As noted in the PR, cp -R has some surprising behavior. Typically, when
you `cp -R foo bar` where both foo and bar exist, foo is cleanly copied
to foo/bar. When you `cp -R foo foo` (where foo clearly exists), cp(1)
goes a little off the rails as it creates foo/foo, then discovers that
and creates foo/foo/foo, so on and so forth, until it eventually fails.
POSIX doesn't seem to disallow this behavior, but it isn't very useful.
GNU cp(1) will detect the recursion and squash it, but emit a message in
the process that it has done so.
This change seemingly follows the GNU behavior, but it currently doesn't
warn about the situation -- the author feels that the final product is
about what one might expect from doing this and thus, doesn't need a
warning. The author doesn't feel strongly about this.
PR: 235438
Reviewed by: bapt
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D33944
PR252358 reported a serious performance problem when
copying a large non-sparse file on a UFS file system.
This problem seems to have been caused by a large
number of SEEK_HOLE operations, with one done
for each copy_file_range(2) call.
This patch modifies cp(1) to use a large (SSIZE_MAX)
len argument, reducing the number of system calls
and resolving the performance issue.
While here, convert the type of the "rcount" from "int"
to "ssize_t" so that it is consistent with that returned
by both read(2) and copy_file_range(2).
PR: 252358
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D27937
This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set. That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.
As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it. Otherwise we might see
a random error from one of the iterations.
gzip is ignoring most errors and could be improved separately.
Reviewed by: vangyzen
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27184
There are some tests available in the NetBSD test suite, but we don't
currently pass all of those; further investigation will go into that. For
now, just add a basic test as well as a test that copies from /dev/null to a
file.
The /dev/null test confirms that the file gets created if it's empty, then
that it truncates the file if it's non-empty. This matches some usage that
was previously employed in the build and was replaced in r366042 by a
simpler shell construct.
I will also plan on coming back to expand these in due time.
MFC after: 1 week
Even though copy_file_range has a file-system agnostic version, it still
fails on devfs (perhaps because the file descriptor is non-seekable?) In
that case, fallback to old-fashioned read/write. Fixes
"cp /dev/null /tmp/null"
PR: 249248
Reported by: Michael Butler
Reviewed by: mjg
MFC-With: 365549
Differential Revision: https://reviews.freebsd.org/D26395
This has three advantages over write(2)/read(2):
* Fewer context switches and data copies
* Mostly preserves a file's sparseness
* On some file systems (currently NFS 4.2) the file system will perform the
copy in an especially efficient way.
Reviewed by: rmacklem
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D26377
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
