Check we can talk to the remote host before starting the git-fastimport
subchild.
Otherwise we fail to connect, and then exit, leaving git-fastimport
still running since we did not wait() for it.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After an error, git-p4 calls die(). This just exits, and leaves child
processes still running.
Instead of calling die(), raise an exception and catch it where the
child process(es) (git-fastimport) are created.
This was analyzed in detail here:
https://public-inbox.org/git/20190227094926.GE19739@szeder.dev/
This change does not address the particular issue of p4CmdList()
invoking a subchild and not waiting for it on error.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes it easier to try/catch around this block of code to ensure
cleanup following p4 failures is handled properly.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pylint is incredibly useful for finding bugs, but git-p4 has never used
it, so there are a lot of warnings that while important, don't actually
result in bugs.
Let's turn those off for now, so we can get some useful output.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently when there is a P4 error, git-p4 calls die() which just exits.
This then leaves the git-fast-import process still running, and can even
leave p4 itself still running.
As a result, git-p4 fails to exit cleanly. This is a particular problem
for people running the unit tests in regression.
Use this exception to report errors upwards, cleaning up as the error
propagates.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ensure that we can safely call self.closeStreams() multiple times, and
can also call it even if there is no git fast-import stream at all.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Miscellaneous small UX improvements on "git-p4".
* bk/p4-misc-usability:
git-p4: show detailed help when parsing options fail
git-p4: yes/no prompts should sanitize user text
When a user provides invalid parameters to git-p4, the program
reports the failure but does not provide the correct command syntax.
Add an exception handler to the command-line argument parser to display
the command's specific command line parameter syntax when an exception
is thrown. Rethrow the exception so the current behavior is retained.
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When prompting the user interactively for direction, the tests are
not forgiving of user input format.
For example, the first query asks for a yes/no response. If the user
enters the full word "yes" or "no" or enters a capital "Y" the test
will fail.
Create a new function, prompt(prompt_text) where
* prompt_text is the text prompt for the user
* returns a single character where valid return values are
found by inspecting prompt_text for single characters
surrounded by square brackets
This new function must prompt the user for input and sanitize it by
converting the response to a lower case string, trimming leading and
trailing spaces, and checking if the first character is in the list
of choices. If it is, return the first letter.
Change the current references to raw_input() to use this new function.
Since the method requires the returned text to be one of the available
choices, remove the loop from the calling code that handles response
verification.
Thanks-to: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git lfs" allows users to specify the custom storage location with
the configuration variable `lfs.storage`, but when interacting with
GitLFS pointers, "git p4" always uses the hardcoded default that is
the `.git/lfs/` directory, without paying attention to the
configuration.
Use the value configured in `lfs.storage`, if exists, as all the
"git" operations do, for consistency.
Signed-off-by: r.burenkov <panzercheg@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Avoid double-open exceptions on Windows platform when
calculating for lfs compressed size threshold
(git-p4.largeFileCompressedThreshold) comparisons.
Take new approach using the NamedTemporaryFile()
file-like object as input to the ZipFile() which
auto-deletes after implicit close leaving with scope.
Original code had double-open exception on Windows
platform because file still open from NamedTemporaryFile()
using generated filename instead of object.
Thanks to Andrey for patiently suggesting several
iterations on this change for avoiding exceptions!
Also print error details after resulting IOError to make
debugging cause of exception less mysterious when it has
nothing to do with "git version recent enough."
Signed-off-by: Philip.McGraw <Philip.McGraw@bentley.com>
Reviewed-by: Andrey Mazo <ahippo@yandex.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" update.
* am/p4-branches-excludes:
git-p4: respect excluded paths when detecting branches
git-p4: add failing test for "git-p4: respect excluded paths when detecting branches"
git-p4: don't exclude other files with same prefix
git-p4: add failing test for "don't exclude other files with same prefix"
git-p4: don't groom exclude path list on every commit
git-p4: match branches case insensitively if configured
git-p4: add failing test for "git-p4: match branches case insensitively if configured"
git-p4: detect/prevent infinite loop in gitCommitByP4Change()
The command line to invoke a "git cat-file" command from inside
"git p4" was not properly quoted to protect a caret and running a
broken command on Windows, which has been corrected.
* mm/p4-unshelve-windows-fix:
p4 unshelve: fix "Not a valid object name HEAD0" on Windows
git p4 unshelve was failing with these errors:
fatal: Not a valid object name HEAD0
Command failed: git cat-file commit HEAD^0
(git version 2.21.0.windows.1, python 2.7.16)
The pOpen call used by git-p4 to invoke the git command can take either a
string or an array as a first argument. The array form is preferred
because platform-specific escaping of special characters will be
handled automatically.(https://docs.python.org/2/library/subprocess.html)
The extractLogMessageFromGitCommit method was, however, using the string
form and so the caret (^) character in the HEAD^0 argument was not being
escaped on Windows. The caret happens to be the escape character, which
is why the git command was receiving HEAD0.
The behaviour can be confirmed by typing ECHO HEAD^0 at the command-
prompt, which emits HEAD0.
The solution is simply to use the array format of passing the command to
fOpen, which is recommended and used in other parts of this code anyway.
Signed-off-by: Mike Mueller <mike.mueller@moodys.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When unshelving a changelist, git-p4 tries to work out the appropriate
parent commit in a given branch (default: HEAD). To do this, it looks
at the state of any pre-existing files in the target Perforce branch,
omitting files added in the shelved changelist. Currently, only files
added (or move targets) are classed as new. However, files integrated
from other branches (i.e. a 'branch' action) also need to be considered
as added, for this purpose.
Signed-off-by: Simon Williams <simon@no-dns-yet.org.uk>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, excluded paths are only handled in the following cases:
* no branch detection;
* branch detection with using clientspec.
However, excluded paths are not respected in case of
branch detection without using clientspec.
Fix this by consulting the list of excluded paths
when splitting files across branches.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make sure not to exclude files unintentionally
if exclude paths are specified without a trailing /.
I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file".
Do this by ensuring that paths without a trailing "/" are only matched completely.
Also, abort path search on the first match as a micro-optimization.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, `cloneExclude` array is being groomed (by removing trailing "...")
on every changeset.
(since `extractFilesFromCommit()` is called on every imported changeset)
As a micro-optimization, do it once while parsing arguments.
Also, prepend "/" and remove trailing "..." at the same time.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 knows how to handle case insensitivity in file paths
if core.ignorecase is set.
However, when determining a branch for a file,
it still does a case-sensitive prefix match.
This may result in some file changes to be lost on import.
For example, given the following commits
1. add //depot/main/file1
2. add //depot/DirA/file2
3. add //depot/dira/file3
4. add //depot/DirA/file4
and "branchList = main:DirA" branch mapping,
commit 3 will be lost.
So, do branch search case insensitively if running with core.ignorecase set.
Teach splitFilesIntoBranches() to use the p4PathStartsWith() function
for path prefix matches instead of always case-sensitive match.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" failed to update a shelved change when there were moved
files, which has been corrected.
* ld/git-p4-shelve-update-fix:
git-p4: handle update of moved/copied files when updating a shelve
git-p4: add failing test for shelved CL update involving move/copy
Perforce requires a complete list of files being operated on. If
git is updating an existing shelved changelist, then any files
which are moved or copied were not being added to this list.
Signed-off-by: Luke Diamand <luke@diamand.org>
Acked-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a perforce setup where login is not required, communication fails
because p4_check_access does not understand the response from the p4
client. Fixed by detecting and ignoring the "info" response.
Signed-off-by: Peter Osterlund <peterosterlund2@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous git-p4 unshelve support would check for changes
in Perforce to the files being unshelved since the original
shelve, and would complain if any were found.
This was to ensure that the user wouldn't end up with both the
shelved change delta, and some deltas from other changes in their
git commit.
e.g. given fileA:
the
quick
brown
fox
change1: s/the/The/ <- p4 shelve this change
change2: s/fox/Fox/ <- p4 submit this change
git p4 unshelve 1 <- FAIL
This change teaches the P4Unshelve class to always create a parent
commit which matches the P4 tree (for the files being unshelved) at
the point prior to the P4 shelve being created (which is reported
in the p4 description for a shelved changelist).
That then means git-p4 can always create a git commit matching the
P4 shelve that was originally created, without any extra deltas.
The user might still need to use the --origin option though - there
is no way for git-p4 to work out the versions of all of the other
*unchanged* files in the shelve, since this information is not recorded
by Perforce.
Additionally this fixes handling of shelved 'move' operations.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The branch detection code looks for branches under refs/remotes/p4/...
and can end up getting confused if there are unshelved changes in
there as well. This happens in the function p4BranchesInGit().
Instead, put the unshelved changes into refs/remotes/p4-unshelved/<N>.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If deleting or moving a file, sometimes P4 doesn't report the file size.
The code handles this just fine but some logging crashes. Stop this
happening.
There was some earlier discussion on the list about this:
https://public-inbox.org/git/xmqq1sqpp1vv.fsf@gitster.mtv.corp.google.com/
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `p4-pre-submit` hook is executed before git-p4 submits code.
If the hook exits with non-zero value, submit process not start.
Signed-off-by: Chen Bin <chenbin.sh@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace calls to print ... with the function form, print(...), to
allow use with python3 as well as python2.x.
Converted using 2to3 (and some hand-editing).
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In Python3, basestring no longer exists, so use this workaround.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Backticks around a variable are a deprecated alias for repr().
This has been removed in python3, so just use the string
representation instead, which is equivalent.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Python3 does not have the dict.has_key() function, so replace all
such calls with "k in dict". This will still work with python2.6
and python2.7.
Converted using 2to3 (plus some hand-editing)
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The <> string inequality operator (which doesn't seem to be even
documented) no longer exists in python3. Replace with !=.
This still works with python2.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" updates.
* ld/git-p4-updates:
git-p4: auto-size the block
git-p4: narrow the scope of exceptions caught when parsing an int
git-p4: raise exceptions from p4CmdList based on error from p4 server
git-p4: better error reporting when p4 fails
git-p4: add option to disable syncing of p4/master with p4
git-p4: disable-rebase: allow setting this via configuration
git-p4: add options --commit and --disable-rebase
git-p4 originally would fetch changes in one query. On large repos this
could fail because of the limits that Perforce imposes on the number of
items returned and the number of queries in the database.
To fix this, git-p4 learned to query changes in blocks of 512 changes,
However, this can be very slow - if you have a few million changes,
with each chunk taking about a second, it can be an hour or so.
Although it's possible to tune this value manually with the
"--changes-block-size" option, it's far from obvious to ordinary users
that this is what needs doing.
This change alters the block size dynamically by looking for the
specific error messages returned from the Perforce server, and reducing
the block size if the error is seen, either to the limit reported by the
server, or to half the current block size.
That means we can start out with a very large block size, and then let
it automatically drop down to a value that works without error, while
still failing correctly if some other error occurs.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current code traps all exceptions around some code which parses an
integer, and then talks to Perforce.
That can result in errors from Perforce being ignored. Change the code
to only catch the integer conversion exceptions.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This change lays some groundwork for better handling of rowcount errors
from the server, where it fails to send us results because we requested
too many.
It adds an option to p4CmdList() to return errors as a Python exception.
The exceptions are derived from P4Exception (something went wrong),
P4ServerException (the server sent us an error code) and
P4RequestSizeException (we requested too many rows/results from the
server database).
This makes the code that handles the errors a bit easier.
The default behavior is unchanged; the new code is enabled with a flag.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently when p4 fails to run, git-p4 just crashes with an obscure
error message.
For example, if the P4 ticket has expired, you get:
Error: Cannot locate perforce checkout of <path> in client view
This change checks whether git-p4 can talk to the Perforce server when
the first P4 operation is attempted, and tries to print a meaningful
error message if it fails.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add an option to the git-p4 submit command to disable syncing
with Perforce.
This is useful for the case where a git-p4 mirror has been setup
on a server somewhere, running from (e.g.) cron, and developers
then clone from this. Having the local cloned copy also sync
from Perforce just isn't useful.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This just lets you set the --disable-rebase option with the
git configuration options git-p4.disableRebase. If you're
using this option, you probably want to set it all the time
for a given repo.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On a daily work with multiple local git branches, the usual way to
submit only a specified commit was to cherry-pick the commit on
master then run git-p4 submit. It can be very annoying to switch
between local branches and master, only to submit one commit. The
proposed new way is to select directly the commit you want to
submit.
Add option --commit to command 'git-p4 submit' in order to submit
only specified commit(s) in p4.
On a daily work developping software with big compilation time, one
may not want to rebase on his local git tree, in order to avoid long
recompilation.
Add option --disable-rebase to command 'git-p4 submit' in order to
disable rebase after submission.
Thanks-to: Cedric Borgese <cedric.borgese@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Romain Merland <merlorom@yahoo.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This can be used to "unshelve" a shelved P4 commit into
a git commit.
For example:
$ git p4 unshelve 12345
The resulting commit ends up in the branch:
refs/remotes/p4/unshelved/12345
If that branch already exists, it is renamed - for example
the above branch would be saved as p4/unshelved/12345.1.
git-p4 checks that the shelved changelist is based on files
which are at the same Perforce revision as the origin branch
being used for the unshelve (HEAD by default). If they are not,
it will refuse to unshelve. This is to ensure that the unshelved
change does not contain other changes mixed-in.
The reference branch can be changed manually with the "--origin"
option.
The change adds a new Unshelve command class. This just runs the
existing P4Sync code tweaked to handle a shelved changelist.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This was the only occurrence of "commitish" in the tree, but as the
log will reveal we've had others in the past. Fixes up code added in
00ad6e3182 ("git-p4: work with a detached head", 2015-11-21).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
--update-shelve can now be specified multiple times on the
command-line, to update multiple shelved changelists in a single
submit.
This then means that a git patch series can be mirrored to a
sequence of shelved changelists, and (relatively easily) kept in
sync as changes are made in git.
Note that Perforce does not really support overlapping shelved
changelists where one change touches the files modified by
another. Trying to do this will result in merge conflicts.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Follow the Oxford style, which says to use "up-to-date" before the noun,
but "up to date" after it. Don't change plumbing (specifically
send-pack.c, but transport.c (git push) also has the same string).
This was produced by grepping for "up-to-date" and "up to date". It
turned out we only had to edit in one direction, removing the hyphens.
Fix a typo in Documentation/git-diff-index.txt while we're there.
Reported-by: Jeffrey Manian <jeffrey.manian@gmail.com>
Reported-by: STEVEN WHITE <stevencharleswhitevoices@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function p4CmdList accepts a new argument: skip_info. When set to
True it ignores any 'code':'info' entry (skip_info=False by default).
That allows us to fix some of the tests in t9831-git-p4-triggers.sh
known to be broken with verobse p4 triggers
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The option -G of p4 (python marshal output) gives more context about the
data being output. That's useful when using the command "change -o" as
we can distinguish between warning/error line and real change description.
This fixes the case where a p4 trigger for "p4 change" is set and the command git-p4 submit is run.
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 was using "git name-rev" to find out the current branch.
That is not safe, since if multiple branches or tags point at
the same revision, the result obtained might not be what is
expected.
Instead use "git symbolic-ref".
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
