Commit graph

196 commits

Author SHA1 Message Date
Junio C Hamano 57cbc53d3e Merge branch 'js/vsts-ci'
Prepare to run test suite on Azure Pipeline.

* js/vsts-ci: (22 commits)
  test-date: drop unused parameter to getnanos()
  ci: parallelize testing on Windows
  ci: speed up Windows phase
  tests: optionally skip bin-wrappers/
  t0061: workaround issues with --with-dashes and RUNTIME_PREFIX
  tests: add t/helper/ to the PATH with --with-dashes
  mingw: try to work around issues with the test cleanup
  tests: include detailed trace logs with --write-junit-xml upon failure
  tests: avoid calling Perl just to determine file sizes
  README: add a build badge (status of the Azure Pipelines build)
  mingw: be more generous when wrapping up the setitimer() emulation
  ci: use git-sdk-64-minimal build artifact
  ci: add a Windows job to the Azure Pipelines definition
  Add a build definition for Azure DevOps
  ci/lib.sh: add support for Azure Pipelines
  tests: optionally write results as JUnit-style .xml
  test-date: add a subcommand to measure times in shell scripts
  ci: use a junction on Windows instead of a symlink
  ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
  ci/lib.sh: encapsulate Travis-specific things
  ...
2019-02-06 22:05:26 -08:00
Johannes Schindelin 72d63b2f45 mingw: be more generous when wrapping up the setitimer() emulation
Every once in a while, the Azure Pipeline fails with some semi-random

	error: timer thread did not terminate timely

This error message means that the thread that is used to emulate the
setitimer() function did not terminate within 1,000 milliseconds.

The most likely explanation (and therefore the one we should assume to
be true, according to Occam's Razor) is that the timeout of one second
is simply not enough because we try to run so many tasks in parallel.

So let's give it ten seconds instead of only one. That should be enough.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:46 -08:00
Johannes Schindelin 9e9da23c27 mingw: special-case arguments to sh
The MSYS2 runtime does its best to emulate the command-line wildcard
expansion and de-quoting which would be performed by the calling Unix
shell on Unix systems.

Those Unix shell quoting rules differ from the quoting rules applying to
Windows' cmd and Powershell, making it a little awkward to quote
command-line parameters properly when spawning other processes.

In particular, git.exe passes arguments to subprocesses that are *not*
intended to be interpreted as wildcards, and if they contain
backslashes, those are not to be interpreted as escape characters, e.g.
when passing Windows paths.

Note: this is only a problem when calling MSYS2 executables, not when
calling MINGW executables such as git.exe. However, we do call MSYS2
executables frequently, most notably when setting the use_shell flag in
the child_process structure.

There is no elegant way to determine whether the .exe file to be
executed is an MSYS2 program or a MINGW one. But since the use case of
passing a command line through the shell is so prevalent, we need to
work around this issue at least when executing sh.exe.

Let's introduce an ugly, hard-coded test whether argv[0] is "sh", and
whether it refers to the MSYS2 Bash, to determine whether we need to
quote the arguments differently than usual.

That still does not fix the issue completely, but at least it is
something.

Incidentally, this also fixes the problem where `git clone \\server\repo`
failed due to incorrect handling of the backslashes when handing the path
to the git-upload-pack process.

Further, we need to take care to quote not only whitespace and
backslashes, but also curly brackets. As aliases frequently go through
the MSYS2 Bash, and as aliases frequently get parameters such as
HEAD@{yesterday}, this is really important. As an early version of this
patch broke this, let's make sure that this does not regress by adding a
test case for that.

Helped-by: Kim Gybels <kgybels@infogroep.be>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 13:12:14 -08:00
Junio C Hamano 25d90d1cb7 Merge branch 'tb/use-common-win32-pathfuncs-on-cygwin'
Cygwin update.

* tb/use-common-win32-pathfuncs-on-cygwin:
  git clone <url> C:\cygwin\home\USER\repo' is working (again)
2019-01-14 15:29:32 -08:00
Torsten Bögershausen 1cadad6f65 git clone <url> C:\cygwin\home\USER\repo' is working (again)
A regression for cygwin users was introduced with commit 05b458c,
 "real_path: resolve symlinks by hand".

In the the commit message we read:
  The current implementation of real_path uses chdir() in order to resolve
    symlinks.  Unfortunately this isn't thread-safe as chdir() affects a
      process as a whole...

The old (and non-thread-save) OS calls chdir()/pwd() had been
replaced by a string operation.
The cygwin layer "knows" that "C:\cygwin" is an absolute path,
but the new string operation does not.

"git clone <url> C:\cygwin\home\USER\repo" fails like this:
fatal: Invalid path '/home/USER/repo/C:\cygwin\home\USER\repo'

The solution is to implement has_dos_drive_prefix(), skip_dos_drive_prefix()
is_dir_sep(), offset_1st_component() and convert_slashes() for cygwin
in the same way as it is done in 'Git for Windows' in compat/mingw.[ch]

Extract the needed code into compat/win32/path-utils.[ch] and use it
for cygwin as well.

Reported-by: Steven Penny <svnpenn@gmail.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-26 15:26:17 -08:00
Junio C Hamano b8a9209193 Merge branch 'js/mingw-msdn-url'
The URL to an MSDN page in a comment has been updated.

* js/mingw-msdn-url:
  mingw: replace an obsolete link with the superseding one
2018-11-18 18:23:58 +09:00
Johannes Schindelin 2ef2ae2917 mingw: replace an obsolete link with the superseding one
The MSDN documentation has been superseded by Microsoft Docs (which is
backed by a repository on GitHub containing many, many files in Markdown
format).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-16 14:57:54 +09:00
Johannes Schindelin 1e1a876bb2 mingw: use CreateHardLink() directly
The function `CreateHardLink()` is available in all supported Windows
versions (even since Windows XP), so there is no more need to resolve it
at runtime.

Helped-by: Max Kirillov <max@max630.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 14:41:15 +09:00
Junio C Hamano 0474cd19ef Merge branch 'js/mingw-utf8-env'
Windows fix.

* js/mingw-utf8-env:
  mingw: reencode environment variables on the fly (UTF-16 <-> UTF-8)
  t7800: fix quoting
2018-11-13 22:37:21 +09:00
Junio C Hamano 6c268fdda9 Merge branch 'js/mingw-perl5lib'
Windows fix.

* js/mingw-perl5lib:
  mingw: unset PERL5LIB by default
  config: move Windows-specific config settings into compat/mingw.c
  config: allow for platform-specific core.* config settings
  config: rename `dummy` parameter to `cb` in git_default_config()
2018-11-13 22:37:20 +09:00
Junio C Hamano b9c3f062b6 Merge branch 'js/mingw-ns-filetime'
Windows port learned to use nano-second resolution file timestamps.

* js/mingw-ns-filetime:
  mingw: implement nanosecond-precision file times
  mingw: replace MSVCRT's fstat() with a Win32-based implementation
  mingw: factor out code to set stat() data
2018-11-06 15:50:21 +09:00
Johannes Schindelin 0e218f91c2 mingw: unset PERL5LIB by default
Git for Windows ships with its own Perl interpreter, and insists on
using it, so it will most likely wreak havoc if PERL5LIB is set before
launching Git.

Let's just unset that environment variables when spawning processes.

To make this feature extensible (and overrideable), there is a new
config setting `core.unsetenvvars` that allows specifying a
comma-separated list of names to unset before spawning processes.

Reported by Gabriel Fuhrmann.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31 12:46:32 +09:00
Johannes Schindelin bdfbb0ea93 config: move Windows-specific config settings into compat/mingw.c
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31 12:46:27 +09:00
Johannes Schindelin 70fc5793df config: allow for platform-specific core.* config settings
In the Git for Windows project, we have ample precendent for config
settings that apply to Windows, and to Windows only.

Let's formalize this concept by introducing a platform_core_config()
function that can be #define'd in a platform-specific manner.

This will allow us to contain platform-specific code better, as the
corresponding variables no longer need to be exported so that they can
be defined in environment.c and be set in config.c

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31 12:46:21 +09:00
Johannes Schindelin fe21c6b285 mingw: reencode environment variables on the fly (UTF-16 <-> UTF-8)
On Windows, the authoritative environment is encoded in UTF-16. In Git
for Windows, we convert that to UTF-8 (because UTF-16 is *such* a
foreign idea to Git that its source code is unprepared for it).

Previously, out of performance concerns, we converted the entire
environment to UTF-8 in one fell swoop at the beginning, and upon
putenv() and run_command() converted it back.

Having a private copy of the environment comes with its own perils: when
a library used by Git's source code tries to modify the environment, it
does not really work (in Git for Windows' case, libcurl, see
https://github.com/git-for-windows/git/compare/bcad1e6d58^...bcad1e6d58^2
for a glimpse of the issues).

Hence, it makes our environment handling substantially more robust if we
switch to on-the-fly-conversion in `getenv()`/`putenv()` calls. Based
on an initial version in the MSVC context by Jeff Hostetler, this patch
makes it so.

Surprisingly, this has a *positive* effect on speed: at the time when
the current code was written, we tested the performance, and there were
*so many* `getenv()` calls that it seemed better to convert everything
in one go. In the meantime, though, Git has obviously been cleaned up a
bit with regards to `getenv()` calls so that the Git processes spawned
by the test suite use an average of only 40 `getenv()`/`putenv()` calls
over the process lifetime.

Speaking of the entire test suite: the total time spent in the
re-encoding in the current code takes about 32.4 seconds (out of 113
minutes runtime), whereas the code introduced in this patch takes only
about 8.2 seconds in total. Not much, but it proves that we need not be
concerned about the performance impact introduced by this patch.

Helped-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31 11:43:14 +09:00
Junio C Hamano 97ffca6cc7 Merge branch 'js/mingw-load-sys-dll'
The way DLLs are loaded on the Windows port has been improved.

* js/mingw-load-sys-dll:
  mingw: load system libraries the recommended way
2018-10-30 15:43:48 +09:00
Junio C Hamano cc6748720f Merge branch 'js/mingw-getcwd'
The way the Windows port figures out the current directory has been
improved.

* js/mingw-getcwd:
  mingw: fix getcwd when the parent directory cannot be queried
  mingw: ensure `getcwd()` reports the correct case
2018-10-30 15:43:48 +09:00
Junio C Hamano 16ce0b92bd Merge branch 'js/mingw-default-ident'
The logic to select the default user name and e-mail on Windows has
been improved.

* js/mingw-default-ident:
  mingw: use domain information for default email
  getpwuid(mingw): provide a better default for the user name
  getpwuid(mingw): initialize the structure only once
2018-10-26 14:22:15 +09:00
Anton Serbulov 4745feeb8b mingw: fix getcwd when the parent directory cannot be queried
`GetLongPathName()` function may fail when it is unable to query
the parent directory of a path component to determine the long name
for that component. It happens, because it uses `FindFirstFile()`
function for each next short part of path. The `FindFirstFile()`
requires `List Directory` and `Synchronize` desired access for a calling
process.

In case of lacking such permission for some part of path,
the `GetLongPathName()` returns 0 as result and `GetLastError()`
returns ERROR_ACCESS_DENIED.

`GetFinalPathNameByHandle()` function can help in such cases, because
it requires `Read Attributes` and `Synchronize` desired access to the
target path only.

The `GetFinalPathNameByHandle()` function was introduced on
`Windows Server 2008/Windows Vista`. So we need to load it dynamically.

`CreateFile()` parameters:
    `lpFileName` = path to the current directory
    `dwDesiredAccess` = 0 (it means `Read Attributes` and `Synchronize`)
    `dwShareMode` = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE
                    (it prevents `Sharing Violation`)
    `lpSecurityAttributes` = NULL (default security attributes)
    `dwCreationDisposition` = OPEN_EXISTING
                              (required to obtain a directory handle)
    `dwFlagsAndAttributes` = FILE_FLAG_BACKUP_SEMANTICS
                             (required to obtain a directory handle)
    `hTemplateFile` = NULL (when opening an existing file or directory,
                            `CreateFile` ignores this parameter)

The string that is returned by `GetFinalPathNameByHandle()` function
uses the \\?\ syntax. To skip the prefix and convert backslashes
to slashes, the `normalize_ntpath()` mingw function will be used.

Note: `GetFinalPathNameByHandle()` function returns a final path.
It is the path that is returned when a path is fully resolved.
For example, for a symbolic link named "C:\tmp\mydir" that points to
"D:\yourdir", the final path would be "D:\yourdir".

Signed-off-by: Anton Serbulov <aserbulov@plesk.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-24 14:48:57 +09:00
Johannes Schindelin 937974fc65 mingw: ensure getcwd() reports the correct case
When switching the current working directory, say, in PowerShell, it is
quite possible to use a different capitalization than the one that is
recorded on disk. While doing the same in `cmd.exe` adjusts the
capitalization magically, that does not happen in PowerShell so that
`getcwd()` returns the current directory in a different way than is
recorded on disk.

Typically this creates no problems except when you call

	git log .

in a subdirectory called, say, "GIT/" but you switched to "Git/" and
your `getcwd()` reports the latter, then Git won't understand that you
wanted to see the history as per the `GIT/` subdirectory but it thinks you
wanted to see the history of some directory that may have existed in the
past (but actually never did).

So let's be extra careful to adjust the capitalization of the current
directory before working with it.

Reported by a few PowerShell power users ;-)

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-24 14:48:38 +09:00
Johannes Schindelin c6f050a434 mingw: load system libraries the recommended way
When we access IPv6-related functions, we load the corresponding system
library using the `LoadLibrary()` function, which is not the recommended
way to load system libraries.

In practice, it does not make a difference: the `ws2_32.dll` library
containing the IPv6 functions is already loaded into memory, so
LoadLibrary() simply reuses the already-loaded library.

Still, recommended way is recommended way, so let's use that instead.

While at it, also adjust the code in contrib/ that loads system libraries.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-24 14:48:00 +09:00
Karsten Blees d7e8c87421 mingw: implement nanosecond-precision file times
We no longer use any of MSVCRT's stat-functions, so there's no need to
stick to a CRT-compatible 'struct stat' either.

Define and use our own POSIX-2013-compatible 'struct stat' with nanosecond-
precision file times.

Note: This can cause performance issues when using Git variants with
different file time resolutions, as the timestamps are stored in the Git
index: after updating the index with a Git variant that uses
second-precision file times, a nanosecond-aware Git will think that
pretty much every single file listed in the index is out of date.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-24 13:18:14 +09:00
Karsten Blees d75e697353 mingw: replace MSVCRT's fstat() with a Win32-based implementation
fstat() is the only stat-related CRT function for which we don't have a
full replacement yet (and thus the only reason to stick with MSVCRT's
'struct stat' definition).

Fully implement fstat(), in preparation of implementing a POSIX 2013
compatible 'struct stat' with nanosecond-precision file times.

This allows us also to implement some clever code to handle pipes and
character devices in our own way.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-24 13:18:14 +09:00
Johannes Schindelin 7bf1983886 mingw: factor out code to set stat() data
In our fstat() emulation, we convert the file metadata from Win32 data
structures to an emulated POSIX structure.

To structure the code better, let's factor that part out into its own
function.

Note: it would be tempting to try to unify this code with the part of
do_lstat() that does the same thing, but they operate on different data
structures: BY_HANDLE_FILE_INFORMATION vs WIN32_FILE_ATTRIBUTE_DATA. So
unfortunately, they cannot be unified.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-24 13:18:14 +09:00
Johannes Schindelin 501afcb8b0 mingw: use domain information for default email
When a user is registered in a Windows domain, it is really easy to
obtain the email address. So let's do that.

Suggested by Lutz Roeder.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16 12:59:57 +09:00
Johannes Schindelin 564be791f3 getpwuid(mingw): provide a better default for the user name
We do have the excellent GetUserInfoEx() function to obtain more
detailed information of the current user (if the user is part of a
Windows domain); Let's use it.

Suggested by Lutz Roeder.

To avoid the cost of loading Secur32.dll (even lazily, loading DLLs
takes a non-neglibile amount of time), we use the established technique
to load DLLs only when, and if, needed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16 12:59:57 +09:00
Johannes Schindelin 55b6513e73 getpwuid(mingw): initialize the structure only once
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16 12:59:57 +09:00
Jeff Hostetler eeaf7ddac7 mingw: fix mingw_open_append to work with named pipes
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:54:54 -07:00
Johannes Sixt d641097589 mingw: enable atomic O_APPEND
The Windows CRT implements O_APPEND "manually": on write() calls, the
file pointer is set to EOF before the data is written. Clearly, this is
not atomic. And in fact, this is the root cause of failures observed in
t5552-skipping-fetch-negotiator.sh and t5503-tagfollow.sh, where
different processes write to the same trace file simultanously; it also
occurred in t5400-send-pack.sh, but there it was worked around in
71406ed4d6 ("t5400: avoid concurrent writes into a trace file",
2017-05-18).

Fortunately, Windows does support atomic O_APPEND semantics using the
file access mode FILE_APPEND_DATA. Provide an implementation that does.

This implementation is minimal in such a way that it only implements
the open modes that are actually used in the Git code base. Emulation
for other modes can be added as necessary later. To become aware of
the necessity early, the unusal error ENOSYS is reported if an
unsupported mode is encountered.

Diagnosed-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Helped-by: Jeff Hostetler <git@jeffhostetler.com>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-13 15:22:19 -07:00
Johannes Schindelin 86e254584b mingw/msvc: use the new-style RUNTIME_PREFIX helper
This change also allows us to stop overriding argv[0] with the absolute
path of the executable, allowing us to preserve e.g. the case of the
executable's file name.

This fixes https://github.com/git-for-windows/git/issues/1496 partially.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11 18:10:28 +09:00
Johannes Schindelin 9ee0540a40 mingw: abort on invalid strftime formats
On Windows, strftime() does not silently ignore invalid formats, but
warns about them and then returns 0 and sets errno to EINVAL.

Unfortunately, Git does not expect such a behavior, as it disagrees
with strftime()'s semantics on Linux. As a consequence, Git
misinterprets the return value 0 as "I need more space" and grows the
buffer. As the larger buffer does not fix the format, the buffer grows
and grows and grows until we are out of memory and abort.

Ideally, we would switch off the parameter validation just for
strftime(), but we cannot even override the invalid parameter handler
via _set_thread_local_invalid_parameter_handler() using MINGW because
that function is not declared. Even _set_invalid_parameter_handler(),
which *is* declared, does not help, as it simply does... nothing.

So let's just bite the bullet and override strftime() for MINGW and
abort on an invalid format string. While this does not provide the
best user experience, it is the best we can do.

See https://msdn.microsoft.com/en-us/library/fe06s4ak.aspx for more
details.

This fixes https://github.com/git-for-windows/git/issues/863

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-19 10:53:17 -07:00
Johannes Schindelin 1a172e4ac1 mingw: optionally redirect stderr/stdout via the same handle
The "2>&1" notation in Powershell and in Unix shells implies that stderr
is redirected to the same handle into which stdout is already written.

Let's use this special value to allow the same trick with
GIT_REDIRECT_STDERR and GIT_REDIRECT_STDOUT: if the former's value is
`2>&1`, then stderr will simply be written to the same handle as stdout.

The functionality was suggested by Jeff Hostetler.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-02 11:19:43 +09:00
Johannes Schindelin 3f944424ac mingw: add experimental feature to redirect standard handles
Particularly when calling Git from applications, such as Visual Studio's
Team Explorer, it is important that stdin/stdout/stderr are closed
properly. However, when spawning processes on Windows, those handles
must be marked as inheritable if we want to use them, but that flag is a
global flag and may very well be used by other spawned processes which
then do not know to close those handles.

Let's introduce a set of environment variables (GIT_REDIRECT_STDIN and
friends) that specify paths to files, or even better, named pipes (which
are similar to Unix sockets) and that are used by the spawned Git
process.  This helps work around above-mentioned issue: those named
pipes will be opened in a non-inheritable way upon startup, and no
handles are passed around (and therefore no inherited handles need to be
closed by any spawned child).

This feature shipped with Git for Windows (marked as experimental) since
v2.11.0(2), so it has seen some serious testing in the meantime.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-02 11:19:41 +09:00
Junio C Hamano b9a7d55d93 Merge branch 'nd/fopen-errors'
We often try to open a file for reading whose existence is
optional, and silently ignore errors from open/fopen; report such
errors if they are not due to missing files.

* nd/fopen-errors:
  mingw_fopen: report ENOENT for invalid file names
  mingw: verify that paths are not mistaken for remote nicknames
  log: fix memory leak in open_next_file()
  rerere.c: move error_errno() closer to the source system call
  print errno when reporting a system call error
  wrapper.c: make warn_on_inaccessible() static
  wrapper.c: add and use fopen_or_warn()
  wrapper.c: add and use warn_on_fopen_errors()
  config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
  config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
  clone: use xfopen() instead of fopen()
  use xfopen() in more places
  git_fopen: fix a sparse 'not declared' warning
2017-06-13 13:47:09 -07:00
Johannes Sixt e5b313442a mingw_fopen: report ENOENT for invalid file names
On Windows, certain characters are prohibited in file names, most
prominently the colon. When fopen() is called with such an invalid file
name, the underlying Windows API actually reports a particular error,
but since there is no suitable errno value, this error is translated
to EINVAL. Detect the case and report ENOENT instead.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-02 10:40:04 +09:00
René Scharfe e0ca1ca20a mingw: simplify PATH handling
On Windows the environment variable PATH contains a semicolon-separated
list of directories to search for, in order, when looking for the
location of a binary to run.  get_path_split() parses it and returns an
array of string copies, which is iterated by path_lookup(), which in
turn passes each entry to lookup_prog().

Change lookup_prog() to take the directory name as a length-limited
string instead of as a NUL-terminated one and parse PATH directly in
path_lookup().  This avoids memory allocations, simplifying the code.

Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 21:44:38 +09:00
Junio C Hamano 13e11ff707 Merge branch 'js/no-html-bypass-on-windows'
On Windows, help.browser configuration variable used to be ignored,
which has been corrected.

* js/no-html-bypass-on-windows:
  Revert "display HTML in default browser using Windows' shell API"
2016-08-25 13:55:06 -07:00
Johannes Schindelin 6db5967d4e Revert "display HTML in default browser using Windows' shell API"
Since 4804aab (help (Windows): Display HTML in default browser using
Windows' shell API, 2008-07-13), Git for Windows used to call
`ShellExecute()` to launch the default Windows handler for `.html`
files.

The idea was to avoid going through a shell script, for performance
reasons.

However, this change ignores the `help.browser` config setting. Together
with browsing help not being a performance-critical operation, let's
just revert that patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-19 13:47:28 -07:00
Johannes Schindelin 0767172b90 mingw: let the build succeed with DEVELOPER=1
The recently introduced developer flags identified a couple of
old-style function declarations in the Windows-specific code where
the parameter list was left empty instead of specifying "void"
explicitly. Let's just fix them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-20 12:12:12 -07:00
Junio C Hamano e29300d69f Merge branch 'js/windows-dotgit' into maint
On Windows, .git and optionally any files whose name starts with a
dot are now marked as hidden, with a core.hideDotFiles knob to
customize this behaviour.

* js/windows-dotgit:
  mingw: remove unnecessary definition
  mingw: introduce the 'core.hideDotFiles' setting
2016-05-26 13:17:23 -07:00
Johannes Schindelin f30afdabbf mingw: introduce the 'core.hideDotFiles' setting
On Unix (and Linux), files and directories whose names start with a dot
are usually not shown by default. This convention is used by Git: the
.git/ directory should be left alone by regular users, and only accessed
through Git itself.

On Windows, no such convention exists. Instead, there is an explicit flag
to mark files or directories as hidden.

In the early days, Git for Windows did not mark the .git/ directory (or
for that matter, any file or directory whose name starts with a dot)
hidden. This lead to quite a bit of confusion, and even loss of data.

Consequently, Git for Windows introduced the core.hideDotFiles setting,
with three possible values: true, false, and dotGitOnly, defaulting to
marking only the .git/ directory as hidden.

The rationale: users do not need to access .git/ directly, and indeed (as
was demonstrated) should not really see that directory, either. However,
not all dot files should be hidden by default, as e.g. Eclipse does not
show them (and the user would therefore be unable to see, say, a
.gitattributes file).

In over five years since the last attempt to bring this patch into core
Git, a slightly buggy version of this patch has served Git for Windows'
users well: no single report indicated problems with the hidden .git/
directory, and the stream of problems caused by the previously non-hidden
.git/ directory simply stopped. The bugs have been fixed during the
process of getting this patch upstream.

Note that there is a funny quirk we have to pay attention to when
creating hidden files: we use Win32's _wopen() function which
transmogrifies its arguments and hands off to Win32's CreateFile()
function. That latter function errors out with ERROR_ACCESS_DENIED (the
equivalent of EACCES) when the equivalent of the O_CREAT flag was passed
and the file attributes (including the hidden flag) do not match an
existing file's. And _wopen() accepts no parameter that would be
transmogrified into said hidden flag. Therefore, we simply try again
without O_CREAT.

A slightly different method is required for our fopen()/freopen()
function as we cannot even *remove* the implicit O_CREAT flag.
Therefore, we briefly mark existing files as unhidden when opening them
via fopen()/freopen().

The ERROR_ACCESS_DENIED error can also be triggered by opening a file
that is marked as a system file (which is unlikely to be tracked in
Git), and by trying to create a file that has *just* been deleted and is
awaiting the last open handles to be released (which would be handled
better by the "Try again?" logic, a story for a different patch series,
though). In both cases, it does not matter much if we try again without
the O_CREAT flag, read: it does not hurt, either.

For details how ERROR_ACCESS_DENIED can be triggered, see
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858

Original-patch-by: Erik Faye-Lund <kusmabite@gmail.com>
Initial-Test-By: Pat Thoyts <patthoyts@users.sourceforge.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-11 13:54:53 -07:00
Johannes Sixt 8e9b20804a Windows: shorten code by re-using convert_slashes()
Make a few more spots more readable by using the recently introduced,
Windows-specific helper.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-04 18:03:02 -07:00
Junio C Hamano 6dd0a37c34 Merge branch 'jk/tighten-alloc'
* jk/tighten-alloc:
  compat/mingw: brown paper bag fix for 50a6c8e
2016-03-04 13:46:25 -08:00
Jeff King 8d5b3325e7 compat/mingw: brown paper bag fix for 50a6c8e
Commit 50a6c8e (use st_add and st_mult for allocation size
computation, 2016-02-22) fixed up many xmalloc call-sites
including ones in compat/mingw.c.

But I screwed up one of them, which was half-converted to
ALLOC_ARRAY, using a very early prototype of the function.
And I never caught it because I don't build on Windows.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-29 11:04:23 -08:00
Junio C Hamano 11529ecec9 Merge branch 'jk/tighten-alloc'
Update various codepaths to avoid manually-counted malloc().

* jk/tighten-alloc: (22 commits)
  ewah: convert to REALLOC_ARRAY, etc
  convert ewah/bitmap code to use xmalloc
  diff_populate_gitlink: use a strbuf
  transport_anonymize_url: use xstrfmt
  git-compat-util: drop mempcpy compat code
  sequencer: simplify memory allocation of get_message
  test-path-utils: fix normalize_path_copy output buffer size
  fetch-pack: simplify add_sought_entry
  fast-import: simplify allocation in start_packfile
  write_untracked_extension: use FLEX_ALLOC helper
  prepare_{git,shell}_cmd: use argv_array
  use st_add and st_mult for allocation size computation
  convert trivial cases to FLEX_ARRAY macros
  use xmallocz to avoid size arithmetic
  convert trivial cases to ALLOC_ARRAY
  convert manual allocations to argv_array
  argv-array: add detach function
  add helpers for allocating flex-array structs
  harden REALLOC_ARRAY and xcalloc against size_t overflow
  tree-diff: catch integer overflow in combine_diff_path allocation
  ...
2016-02-26 13:37:16 -08:00
Jeff King 50a6c8efa2 use st_add and st_mult for allocation size computation
If our size computation overflows size_t, we may allocate a
much smaller buffer than we expected and overflow it. It's
probably impossible to trigger an overflow in most of these
sites in practice, but it is easy enough convert their
additions and multiplications into overflow-checking
variants. This may be fixing real bugs, and it makes
auditing the code easier.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 14:51:09 -08:00
Jeff King b32fa95fd8 convert trivial cases to ALLOC_ARRAY
Each of these cases can be converted to use ALLOC_ARRAY or
REALLOC_ARRAY, which has two advantages:

  1. It automatically checks the array-size multiplication
     for overflow.

  2. It always uses sizeof(*array) for the element-size,
     so that it can never go out of sync with the declared
     type of the array.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 14:51:09 -08:00
Junio C Hamano 4b589e5b28 Merge branch 'js/mingw-tests'
Test scripts have been updated to remove assumptions that are not
portable between Git for POSIX and Git for Windows, or to skip ones
with expectations that are not satisfiable on Git for Windows.

* js/mingw-tests: (21 commits)
  gitignore: ignore generated test-fake-ssh executable
  mingw: do not bother to test funny file names
  mingw: skip a test in t9130 that cannot pass on Windows
  mingw: handle the missing POSIXPERM prereq in t9124
  mingw: avoid illegal filename in t9118
  mingw: mark t9100's test cases with appropriate prereqs
  t0008: avoid absolute path
  mingw: work around pwd issues in the tests
  mingw: fix t9700's assumption about directory separators
  mingw: skip test in t1508 that fails due to path conversion
  tests: turn off git-daemon tests if FIFOs are not available
  mingw: disable mkfifo-based tests
  mingw: accomodate t0060-path-utils for MSYS2
  mingw: fix t5601-clone.sh
  mingw: let lstat() fail with errno == ENOTDIR when appropriate
  mingw: try to delete target directory before renaming
  mingw: prepare the TMPDIR environment variable for shell scripts
  mingw: factor out Windows specific environment setup
  Git.pm: stop assuming that absolute paths start with a slash
  mingw: do not trust MSYS2's MinGW gettext.sh
  ...
2016-02-17 10:13:29 -08:00
Junio C Hamano 07be1da216 Merge branch 'js/dirname-basename' into maint
dirname() emulation has been added, as Msys2 lacks it.

* js/dirname-basename:
  mingw: avoid linking to the C library's isalpha()
  t0060: loosen overly strict expectations
  t0060: verify that basename() and dirname() work as expected
  compat/basename.c: provide a dirname() compatibility function
  compat/basename: make basename() conform to POSIX
  Refactor skipping DOS drive prefixes
2016-02-05 14:54:17 -08:00
Junio C Hamano e01c6b15c9 Merge branch 'js/dirname-basename'
dirname() emulation has been added, as Msys2 lacks it.

* js/dirname-basename:
  mingw: avoid linking to the C library's isalpha()
  t0060: loosen overly strict expectations
  t0060: verify that basename() and dirname() work as expected
  compat/basename.c: provide a dirname() compatibility function
  compat/basename: make basename() conform to POSIX
  Refactor skipping DOS drive prefixes
2016-02-03 14:16:06 -08:00