Commit graph

302 commits

Author SHA1 Message Date
Junio C Hamano
b1b489f4cc Merge branch 'kk/p4-client-name-encoding-fix' into maint
"git p4" did not handle non-ASCII client name well, which has been
corrected.
source: <pull.1285.v3.git.git.1658394440.gitgitgadget@gmail.com>

* kk/p4-client-name-encoding-fix:
  git-p4: refactoring of p4CmdList()
  git-p4: fix bug with encoding of p4 client name
2022-08-10 21:52:33 -07:00
Kilian Kilger
d205483695 git-p4: refactoring of p4CmdList()
The function p4CmdList executes a Perforce command and
decodes the marshalled python dictionary. Special care has to be
taken for certain dictionary values which contain non-unicode characters.
The old handling contained separate hacks for each of the corresponding
dictionary keys. This commit tries to refactor the coding to handle the
special cases uniformely.

Signed-off-by: Kilian Kilger <kkilger@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-21 09:47:06 -07:00
Moritz Baumann
4d35f74421 git-p4: fix CR LF handling for utf16 files
Perforce silently replaces LF with CR LF for "utf16" files if the client
is a native Windows client. Since git's autocrlf logic does not undo
this transformation for UTF-16 encoded files, git-p4 replaces CR LF with
LF during the sync if the file type "utf16" is detected and the Perforce
client platform indicates that this conversion is performed.

Windows only runs on little-endian architectures, therefore the encoding
of the byte stream received from the Perforce client is UTF-16-LE and
the relevant byte sequence is 0D 00 0A 00.

Signed-off-by: Moritz Baumann <moritz.baumann@sap.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-20 11:38:06 -07:00
Kilian Kilger
34f67c9619 git-p4: fix bug with encoding of p4 client name
The Perforce client name can contain arbitrary characters
which do not decode to UTF-8. Use the fallback strategy
implemented in metadata_stream_to_writable_bytes() also
for the client name.

Signed-off-by: Kilian Kilger <kkilger@gmail.com>
Reviewed-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-08 07:59:12 -07:00
Junio C Hamano
3af1df0415 Merge branch 'tk/p4-metadata-coding-strategies'
"git p4" updates.

* tk/p4-metadata-coding-strategies:
  git-p4: improve encoding handling to support inconsistent encodings
2022-05-20 15:27:00 -07:00
Junio C Hamano
586f23705c Merge branch 'kf/p4-multiple-remotes'
"git p4" update.

* kf/p4-multiple-remotes:
  git-p4: fix issue with multiple perforce remotes
2022-05-20 15:26:55 -07:00
Junio C Hamano
af3a3205d1 Merge branch 'tk/p4-with-explicity-sync'
"git p4" update.

* tk/p4-with-explicity-sync:
  git-p4: support explicit sync of arbitrary existing git-p4 refs
2022-05-20 15:26:55 -07:00
Junio C Hamano
804ec0301f Merge branch 'tk/p4-utf8-bom'
"git p4" update.

* tk/p4-utf8-bom:
  git-p4: preserve utf8 BOM when importing from p4 to git
2022-05-20 15:26:54 -07:00
Tao Klerks
f7b5ff607f git-p4: improve encoding handling to support inconsistent encodings
git-p4 is designed to run correctly under python2.7 and python3, but
its functional behavior wrt importing user-entered text differs across
these environments:

Under python2, git-p4 "naively" writes the Perforce bytestream into git
metadata (and does not set an "encoding" header on the commits); this
means that any non-utf-8 byte sequences end up creating invalidly-encoded
commit metadata in git.

Under python3, git-p4 attempts to decode the Perforce bytestream as utf-8
data, and fails badly (with an unhelpful error) when non-utf-8 data is
encountered.

Perforce clients (especially p4v) encourage user entry of changelist
descriptions (and user full names) in OS-local encoding, and store the
resulting bytestream to the server unmodified - such that different
clients can end up creating mutually-unintelligible messages. The most
common inconsistency, in many Perforce environments, is likely to be utf-8
(typical in linux) vs cp-1252 (typical in windows).

Make the changelist-description- and user-fullname-handling code
python-runtime-agnostic, introducing three "strategies" selectable via
config:
- 'passthrough', behaving as previously under python2,
- 'strict', behaving as previously under python3, and
- 'fallback', favoring utf-8 but supporting a secondary encoding when
utf-8 decoding fails, and finally escaping high-range bytes if the
decoding with the secondary encoding also fails.

Keep the python2 default behavior as-is ('legacy' strategy), but switch
the python3 default strategy to 'fallback' with default fallback encoding
'cp1252'.

Also include tests exercising these encoding strategies, documentation for
the new config, and improve the user-facing error messages when decoding
does fail.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:30:01 -07:00
Tao Klerks
fbe5f6b804 git-p4: preserve utf8 BOM when importing from p4 to git
Perforce has a file type "utf8" which represents a text file with
explicit BOM. utf8-encoded files *without* BOM are stored as
regular file type "text". The "utf8" file type behaves like text
in all but one important way: it is stored, internally, without
the leading 3 BOM bytes.

git-p4 has historically imported utf8-with-BOM files (files stored,
in Perforce, as type "utf8") the same way as regular text files -
losing the BOM in the process.

Under most circumstances this issue has little functional impact,
as most systems consider the BOM to be optional and redundant, but
this *is* a correctness failure, and can have lead to practical
issues for example when BOMs are explicitly included in test files,
for example in a file encoding test suite.

Fix the handling of utf8-with-BOM files when importing changes from
p4 to git, and introduce a test that checks it is working correctly.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06 12:59:58 -07:00
Tao Klerks
17f273ffba git-p4: support explicit sync of arbitrary existing git-p4 refs
With the --branch argument of the "sync" subcommand, git-p4 enables
you to import a perforce branch/path to an arbitrary git ref, using
a full ref path, or to refs/remotes/p4/* or refs/heads/p4/*,
depending on --import-local, using a short ref name.

However, when you later want to explicitly sync such a given ref to
pick up subsequent p4 changes, it only works if the ref was placed
in the p4 path *and* has only one path component (no "/").

This limitation results from a bad assumption in the
existing-branch sync logic, and also means you cannot individually
sync branches detected by --detect-branches, as these also get a
"/" in their names.

Fix "git p4 sync --branch", when called with an existing ref, so
that it works correctly regardless of whether the ref is in the p4
path or not, and (in the case of refs in the p4 path) regardless of
whether it has a "/" in its short name or not.

Also add tests to validate that these branch-specific syncs work
as expected.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06 12:59:40 -07:00
Joel Holdsworth
4ff0108d9e git-p4: sort imports
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
e8f8b3b2a3 git-p4: seperate multiple statements onto seperate lines
PEP8 discourages the use of compound statements where there are multiple
statements on a single line in the "Other Recommendations" section:

https://www.python.org/dev/peps/pep-0008/#other-recommendations

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
4768af2088 git-p4: move inline comments to line above
PEP8 recommends that all inline comments should be separated from code
by two spaces, in the "Inline Comments" section:

https://www.python.org/dev/peps/pep-0008/#inline-comments

However, because all instances of these inline comments extended to an
excessive line length, they have been moved onto a seprate line.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
77956b9de5 git-p4: only seperate code blocks by a single empty line
PEP8 recommends that blank lines should be used sparingly to separate
sections in the "Blank Lines" section:

https://www.python.org/dev/peps/pep-0008/#blank-lines

This patch replaces all double blank-line separations with a single
blank line.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
da0134f653 git-p4: compare to singletons with "is" and "is not"
PEP8 recommends that comparisons with singletons such as None should be
done with "is" and "is not", and never equality operators.

This guideline is described here:
https://www.python.org/dev/peps/pep-0008/#programming-recommendations

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
7a3e83d0bd git-p4: normalize indentation of lines in conditionals
PEP8 recommends that when wrapping the arguments of conditional
statements, an extra level of indentation should be added to distinguish
arguments from the body of the statement.

This guideline is described here:
https://www.python.org/dev/peps/pep-0008/#indentation

This patch either adds the indentation, or removes unnecessary wrapping.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
6febb9f843 git-p4: ensure there is a single space around all operators
PEP8 requires that binary operators such as assignment and comparison
operators should always be surrounded by a pair of single spaces, and
recommends that all other binary operators should typically be surround
by single spaces.

The recommendation is given here in the "Other Recommendations"
section

https://www.python.org/dev/peps/pep-0008/#other-recommendations

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
c785e2029c git-p4: ensure every comment has a single #
PEP8 recommends that every comment should begin with a single '#'
character.

This guideline is described here:
https://www.python.org/dev/peps/pep-0008/#comments

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
2bcf611088 git-p4: remove spaces between dictionary keys and colons
PEP8 makes no specific recommendation about spaces preceding colons in
dictionary declarations, but all the code examples contained with it
declare dictionaries with a single space after the colon, and none
before.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
968e29e16b git-p4: remove redundant backslash-continuations inside brackets
PEP8 recommends that backslash line continuations should only be used
for line-breaks outside parentheses.

This recommendation is described in the "Maximum Line Length" section:

https://www.python.org/dev/peps/pep-0008/#maximum-line-length

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:44 -07:00
Joel Holdsworth
843d847ff7 git-p4: remove extraneous spaces before function arguments
PEP8 recommends that there should be no spaces before function arguments
in the in the "Pet Peeves" section:

https://www.python.org/dev/peps/pep-0008/#pet-peeves

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
12a77f5b7e git-p4: place a single space after every comma
This patch improves consistency across git-p4 by ensuring all command
separated arguments to function invocations, tuples and lists are
separated by commas with a single space following.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
0874bb016a git-p4: removed brackets when assigning multiple return values
In several places, git-p4 contains code of the form:

(a, b) = foo()

In each case, multiple values are returned through a tuple or a list and
bound into multiple values.

The brackets around the assigned variables are redundant and can be
removed:

a, b = foo()

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
57fe2ce0e1 git-p4: remove spaces around default arguments
PEP8 recommends that there should be no spaces around the = sign of
default argument values of functions.

This guideline is described here:
https://www.python.org/dev/peps/pep-0008/#other-recommendations

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
84af8b8544 git-p4: remove padding from lists, tuples and function arguments
PEP8 discourages use of extraneous padding inside any parenthesis,
brackets or braces in the "Pet Peeves" section:

https://www.python.org/dev/peps/pep-0008/#pet-peeves

This patch removes all cases of these.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
794bb28d2a git-p4: sort and de-duplcate pylint disable list
git-p4 contains configuration commands for pylint embedded in the header
comment.

Previously, these were combined onto single lines and not alphabetically
sorted. This patch breaks each disable command onto a separate line to
give cleaner diffs, removed duplicate entries, and sorts the list
alphabetically.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
9084961b2a git-p4: remove commented code
Previously, the script contained commented code including Python 2 print
statements. Presumably, these were used as a developer aid at some point
in history. However, the commented code is generally undesirable, and
this commented code serves no useful purpose. Therefore this patch
removes it.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
522e914f65 git-p4: convert descriptive class and function comments into docstrings
Previously, a small number of functions, methods and classes were
documented using comments. This patch improves consistency by converting
these into docstrings similar to those that already exist in the script.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
59ef3fc104 git-p4: improve consistency of docstring formatting
This patch attempts to improve the consistency of the docstrings by
making the following changes:

  - Rewraps all docstrings to a 79-character column limit.
  - Adds a full stop at the end of every docstring.
  - Removes any spaces after the opening triple-quotes of all
    docstrings.
  - Sets the hanging indent of multi-line docstrings to 3-spaces.
  - Ensures that the closing triple-quotes of multi-line docstrings are
    always on a new line indented by a 3-space indent.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
812ee74ea0 git-p4: indent with 4-spaces
PEP8 recommends that all code should be indented in 4-space units. This
guideline is described here:

https://www.python.org/dev/peps/pep-0008/#indentation

Previously git-p4 had multiple cases where code was indented with a
non-multiple of 4-spaces. This patch fixes each of these.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
990547aa2b git-p4: remove unneeded semicolons from statements
Python allows the usage of compound statements where multiple statements
are written on a single line separared by semicolons. It is also
possible to add a semicolon after a single statement, however this is
generally considered to be untidy, and is unnecessary.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Joel Holdsworth
adf159b441 git-p4: add blank lines between functions and class definitions
In the PEP8 style guidelines, top-level functions and class definitions
should be separated by two blank lines. Methods should be surrounded by
a single blank line.

This guideline is described here in the "Blank Lines" section:
https://www.python.org/dev/peps/pep-0008/#blank-lines

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 13:15:43 -07:00
Kirill Frolov
944db25c60 git-p4: fix issue with multiple perforce remotes
Single perforce branch might be sync'ed multiple times with different
revision numbers, so it will be seen to Git as complete different
commits. This can be done by the following command:

  git p4 sync --branch=NAME //perforce/path...

It is assumed, that this command applied multiple times and
peforce repository changes between command invocations.

In such situation, git p4 will see multiple perforce branches with
same name and different revision numbers. The problem is that to make
a shelve, git-p4 script will try to find "origin" branch, if not
specified in command line explicitly. And previously script selected
any branch with same name and don't mention particular revision number.
Later this may cause failure of the command "git diff-tree -r $rev^ $rev",
so shelve can't be created (due to wrong origin branch/commit).

This commit fixes the heuristic by which git p4 selects origin branch:
first it tries to select branch with same perforce path and perforce
revision, and if it fails, then selects branch with only same perforce
path (ignoring perforce revision number).

Signed-off-by: Kirill Frolov <k.frolov@samsung.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-21 14:03:36 -07:00
Junio C Hamano
c70bc338e9 Merge branch 'ab/config-based-hooks-2'
More "config-based hooks".

* ab/config-based-hooks-2:
  run-command: remove old run_hook_{le,ve}() hook API
  receive-pack: convert push-to-checkout hook to hook.h
  read-cache: convert post-index-change to use hook.h
  commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  git-p4: use 'git hook' to run hooks
  send-email: use 'git hook run' for 'sendemail-validate'
  git hook run: add an --ignore-missing flag
  hooks: convert worktree 'post-checkout' hook to hook library
  hooks: convert non-worktree 'post-checkout' hook to hook library
  merge: convert post-merge to use hook.h
  am: convert applypatch-msg to use hook.h
  rebase: convert pre-rebase to use hook.h
  hook API: add a run_hooks_l() wrapper
  am: convert {pre,post}-applypatch to use hook.h
  gc: use hook library for pre-auto-gc hook
  hook API: add a run_hooks() wrapper
  hook: add 'run' subcommand
2022-02-09 14:21:00 -08:00
Junio C Hamano
391d85d78d Merge branch 'jh/p4-spawning-external-commands-cleanup'
* jh/p4-spawning-external-commands-cleanup:
  git-p4: don't print shell commands as python lists
  git-p4: pass command arguments as lists instead of using shell
  git-p4: don't select shell mode using the type of the command argument
2022-02-05 09:42:30 -08:00
Junio C Hamano
8b9c425629 Merge branch 'jh/p4-fix-use-of-process-error-exception'
* jh/p4-fix-use-of-process-error-exception:
  git-p4: fix instantiation of CalledProcessError
2022-02-05 09:42:29 -08:00
Junio C Hamano
9dbb375f94 Merge branch 'jh/p4-remove-unused'
Remove a few commands from "git p4" that aren't very useful.

* jh/p4-remove-unused:
  git-p4: remove "rollback" verb
  git-p4: remove "debug" verb
2022-01-10 11:52:50 -08:00
Junio C Hamano
66f6c18e5b Merge branch 'jh/p4-human-unit-numbers'
The way "git p4" shows file sizes in its output has been updated to
use human-readable units.

* jh/p4-human-unit-numbers:
  git-p4: show progress as an integer
  git-p4: print size values in appropriate units
2022-01-10 11:52:49 -08:00
Emily Shaffer
0c8ac06b53 git-p4: use 'git hook' to run hooks
Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.

We're dropping the "verbose" handling added in 9f59ca4d6a (git-p4:
create new function run_git_hook, 2020-02-11), those who want
diagnostic output about how hooks are run are now able to get that via
e.g. the trace2 facility and GIT_TRACE=1.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-07 15:19:35 -08:00
Joel Holdsworth
40e7cfdd46 git-p4: fix instantiation of CalledProcessError
CalledProcessError is an exception class from the subprocess namespace.
When raising this exception, git-p4 would instantiate CalledProcessError
objects without properly referencing the subprocess namespace causing
the script to fail.

Resolves the issue by replacing CalledProcessError with
subprocess.CalledProcessError.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-06 15:05:22 -08:00
Joel Holdsworth
727e6ea350 git-p4: don't print shell commands as python lists
Previously the git-p4 script would log commands as stringified
representations of the command parameter, leading to output such as
this:

Reading pipe: ['git', 'config', '--bool', 'git-p4.useclientspec']

Now that all commands are list objects, this patch instead joins the
elements of the list into a single string so the output now looks more
readable:

Reading pipe: git config --bool git-p4.useclientspec

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-06 14:55:12 -08:00
Joel Holdsworth
8a470599f3 git-p4: pass command arguments as lists instead of using shell
In the majority of the subprocess calls where shell=True was used, it
was only needed to parse command arguments by spaces. In each of these
cases, the commands are now being passed in as lists instead of strings.

This change aids the comprehensibility of the code. Constucting commands
and arguments using strings risks bugs from unsanitized inputs, and the
attendant complexity of properly quoting and escaping command arguments.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-06 14:55:12 -08:00
Joel Holdsworth
3d8a3038bc git-p4: don't select shell mode using the type of the command argument
Previously, the script would invoke subprocess functions setting the
shell argument True if the command argument was a string, setting it
False otherwise.

This patch replaces this implicit type-driven behaviour with explicit
shell arguments specified by the caller.

The apparent motive for the implict behaviour is that the subprocess
functions do not divide command strings into args. Invoking
subprocess.call("echo hello") will attempt to execute a program by the
name "echo hello". With subprocess.call("echo hello", shell=True), sh
-c "echo hello" will be executed instead, which will cause the command
and args to be divided by spaces.

Eventually, all usage of shell=True, that is not necessary for some
purpose beyond parsing command strings, should be removed. For now,
this patch makes the usage of shells explicit.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-06 14:55:12 -08:00
Joel Holdsworth
0fe3df45f2 git-p4: remove "rollback" verb
The "rollback" verb implements a simple algorithm which takes the set of
remote perforce tracker branches, or optionally, the complete collection
of local branches in a git repository, and deletes commits from these
branches until there are no commits left with a perforce change number
greater than than a user-specified change number. If the base of a git
branch has a newer change number than the user-specified maximum, then
the branch is deleted.

In future, there might be an argument for the addition of some kind of
"reset this branch back to a given perforce change number" verb for
git-p4. However, in its current form it is unlikely to be useful to
users for the following reasons:

  * The verb is completely undocumented. The only description provided
    contains the following text: "A tool to debug the multi-branch
    import. Don't use :)".

  * The verb has a very narrow purpose in that it applies the rollback
    operation to fixed sets of branches - either all remote p4 branches,
    or all local branches. There is no way for users to specify branches
    with more granularity, for example, allowing users to specify a
    single branch or a set of branches. The utility of the current
    implementation is therefore a niche within a niche.

Given these shortcomings, this patch removes the verb from git-p4.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 13:43:32 -08:00
Joel Holdsworth
fb8dfc1ccb git-p4: remove "debug" verb
The git-p4 "debug" verb is described as "A tool to debug the output of
p4 -G".

The verb is not documented in any detail, but implements a function
which executes an arbitrary p4 command with the -G flag, which causes
perforce to format all output as marshalled Python dictionary objects.

The verb was implemented early in the history of git-p4, and may once
have served a useful purpose to the authors in the early stages of
development. However, the "debug" verb is no longer being used by the
current developers (and users) of git-p4, and whatever purpose the verb
previously offered is easily replaced by invoking p4 directly.

This patch therefore removes the verb from git-p4.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22 13:43:32 -08:00
Joel Holdsworth
0f829620e6 git-p4: show progress as an integer
When importing files from Perforce, git-p4 periodically logs the
progress of file transfers as a percentage. However, the value is
printed as a float with an excessive number of decimal places.

For example a typical update might contain the following message:

Importing revision 12345 (26.199617677553135%)

This patch simply rounds the value down to the nearest integer
percentage value, greatly improving readability.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 12:55:26 -08:00
Joel Holdsworth
ae9b9509a7 git-p4: print size values in appropriate units
The git-p4 script reports file sizes in various log messages.
Previously, in each case the script would print them as the number of
bytes divided by 1048576 i.e. the size in mebibytes, rounded down to an
integer.  This resulted in small files being described as having a size
of "0 MB".

This patch replaces the existing behaviour with a new helper function:
format_size_human_readable, which takes a number of bytes (or any other
quantity), and computes the appropriate prefix to use: none, Ki, Mi, Gi,
Ti, Pi, Ei, Zi, Yi.

For example, a size of 123456 will now be printed as "120.6 KiB" greatly
improving the readability of the log output.

Large valued prefixes such as pebi, exbi, zebi and yobi are included for
completeness, though they not expected to appear in any real-world
Perforce repository!

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 12:55:09 -08:00
Joel Holdsworth
70c0d55349 git-p4: resolve RCS keywords in bytes not utf-8
RCS keywords are strings that are replaced with information from
Perforce. Examples include $Date$, $Author$, $File$, $Change$ etc.

Perforce resolves these by expanding them with their expanded values
when files are synced, but Git's data model requires these expanded
values to be converted back into their unexpanded form.

Previously, git-p4.py would implement this behaviour through the use of
regular expressions. However, the regular expression substitution was
applied using decoded strings i.e. the content of incoming commit diffs
was first decoded from bytes into UTF-8, processed with regular
expressions, then converted back to bytes.

Not only is this behaviour inefficient, but it is also a cause of a
common issue caused by text files containing invalid UTF-8 data. For
files created in Windows, CP1252 Smart Quote Characters (0x93 and 0x94)
are seen fairly frequently. These codes are invalid in UTF-8, so if the
script encountered any file containing them, on Python 2 the symbols
will be corrupted, and on Python 3 the script will fail with an
exception.

This patch replaces this decoding/encoding with bytes object regular
expressions, so that the substitution is performed directly upon the
source data with no conversions.

A test for smart quote handling has been added to the
t9810-git-p4-rcs.sh test suite.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-16 14:06:36 -08:00
Joel Holdsworth
4cf67ae1b6 git-p4: open temporary patch file for write only
The patchRCSKeywords method creates a temporary file in which to store
the patched output data. Previously this file was opened in "w+" mode
(write and read), but the code never reads the contents of the file
while open, so it only needs to be opened in "w" mode (write-only).

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-16 14:06:36 -08:00