-
Notifications
You must be signed in to change notification settings - Fork 162
Last preparations before upstreaming Git for Windows' symlink support #2017
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: js/test-symlink-windows
Are you sure you want to change the base?
Last preparations before upstreaming Git for Windows' symlink support #2017
Conversation
Ever since fe53bbc (Git.pm: Always set Repository to absolute path if autodetecting, 2009-05-07), the t9700 test _must_ fail on Windows because of that age-old Unix paths vs Windows paths problem. The underlying root cause is that Git cannot run with a regular Win32 variant of Perl, the assumption that every path is a Unix path is just too strong in Git's Perl code. As a consequence, Git for Windows is basically stuck with using the MSYS2 variant of Perl which uses a POSIX emulation layer (which is a friendly fork of Cygwin) _and_ a best-effort Unix <-> Windows paths conversion whenever crossing the boundary between MSYS2 and regular Win32 processes. It is best effort only, though, using heuristics to automagically convert correctly in most cases, but not in all cases. In the context of this here patch, this means that asking `git.exe` for the absolute path of the `.git/` directory will return a Win32 path because `git.exe` is a regular Win32 executable that has no idea about Unix-ish paths. But above-mentioned commit introduced a test that wants to verify that this path is identical to the one that the Git Perl module reports (which refuses to use Win32 paths and uses Unix-ish paths instead). Obviously, this must fail because no heuristics can kick in at that layer. This test failure has not even been caught when Git introduced Windows support in its CI definition in 2e90484 (ci: add a Windows job to the Azure Pipelines definition, 2019-01-29), as all tests relying on Perl had to be disabled even from the start (because the CI runs would otherwise have resulted in prohibitively long runtimes, not because Windows is super slow per se, but because Git's test suite keeps insisting on using technology that requires a POSIX emulation layer, which _is_ super slow on Windows). To work around this failure, let's use the `cygpath` utility to convert the absolute `gitdir` path into the form that the Perl code expects. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When 0482c32 (apply: ignore working tree filemode when !core.filemode, 2023-12-26) fixed `git apply` to stop warning about executable files, it inadvertently changed the code flow also for symbolic links and directories. Let's narrow the scope of the special `!trust_executable_git` code path to apply only to regular files. This is needed to let t4115.5(symlink escape when creating new files) pass on Windows when symbolic link support is enabled in the MSYS2 runtime. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `_wopen()` function would gladly follow a symbolic link to a non-existent file and create it when given above-mentioned flags. Git expects the `open()` call to fail, though. So let's add yet another work-around to pretend that Windows behaves like Linux. This is required to let t4115.8(--reject removes .rej symlink if it exists) pass on Windows when enabling the MSYS2 runtime's symbolic link support. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test case 're-init to move gitdir symlink' wants to compare the contents of `newdir/.git`, which is a symbolic link pointing to a file. However, `git diff --no-index`, which is used by `test_cmp` on Windows, does not resolve symlinks; It shows the symlink _target_ instead (with a file mode of 120000). That is totally unexpected by the test case, which as a consequence fails, meaning that it's a bug in the test case itself. Co-authored-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Just like 0fdcfa2 (t0301: fixes for windows compatibility, 2021-09-14) explained, we should not call `mkdir -m<mode>` in the test suite because that would fail on Windows (because Windows has a much more powerful permission system that cannot be mapped into the simpler user/group/other read/write/execute model). There was one forgotten instance of this which was hidden by a `SYMLINK` prerequisite. Currently, this prevents this test case from being executed on Windows, but with the upcoming support for symbolic links, it would become a problem. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'symref transaction supports symlinks' test case is guarded by the
`SYMLINK` prerequisite because `core.prefersymlinkrefs = true` requires
symbolic links to be supported.
However, the `preferSymlinkRefs` feature is not supported on Windows,
therefore this test case needs the `MINGW` prerequisite, too.
There's a couple more cases where we set this config key:
- In a subsequent test in t0600, but there we explicitly set it to
"false". So this would naturally be supported by Windows.
- In t7201 we set the value to `yes`, but we never verify that the
written reference is a symbolic link in the first place. I guess
that we could rather remove setting the configuration value here, as
we are about to deprecate support for symrefs via symbolic links in
the first place. But that's certainly outside of the scope of this
patch.
- In t9903 we do the same, but likewise, we don't check whether the
written file is a symbolic link.
Therefore this seems to be the only instance where the tests actually
need to be adapted.
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The MSYS2 runtime (which inherits this trait from the Cygwin runtime, and which is used by Git for Windows' Bash to emulate POSIX functionality on Windows, the same Bash that is also used to run Git's test suite on Windows) has a mode where it can create native symbolic links on Windows. Naturally, this is a bit of a strange feature, given that Cygwin goes out of its way to support Unix-like paths even if no Win32 program understands those, and the symbolic links have to use Win32 paths instead (which Win32 programs understand very well). As a consequence, the symbolic link targets get normalized before the links are created. This results in certain quirks that Git's test suite is ill equipped to accommodate (because Git's test suite expects to be able to use Unix-like paths even on Windows). The test script t1006-cat-file.sh contains two prime examples, two test cases that need to skip a couple assertions because they are simply wrong in the context of Git for Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
In Git for Windows, the gitdir is canonicalized so that even when the gitdir is specified via a symbolic link, the `gitdir:` conditional include will only match the real directory path. Unfortunately, t1305 codifies a different behavior in two test cases, which are hereby skipped on Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The device `/dev/null` does not exist on Windows, it's called `NUL` there. Calling `ln -s /dev/null my-symlink` in a symlink-enabled MSYS2 Bash will therefore literally link to a file or directory called `null` that is supposed to be in the current drive's top-level `dev` directory. Which typically does not exist. The test, however, really wants the created symbolic link to point to the NUL device. Let's instead use the `mklink` utility on Windows to perform that job, and keep using `ln -s /dev/null <target>` on non-Windows platforms. While at it, add the missing `SYMLINKS` prereq because this test _still_ would not pass on Windows before support for symbolic links is upstreamed from Git for Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git's test suite's relies on Unix shell scripting, which is understandable, of course, given Git's firm roots (and indeed, ongoing focus) on Linux. This fact, combined with Unix shell scripting's natural habitat -- which is, naturally... *drumroll*... Unix -- often has unintended side effects, where developers expect the test suite to run in a Unix environment, which is an incorrect assumption. One instance of this problem can be observed in the 'difftool --dir-diff handles modified symlinks' test case in `t7800-difftool.sh`, which assumes that all absolute paths start with a forward slash. That assumption is incorrect in general, e.g. on Windows, where absolute paths have many shapes and forms, none of which starts with a forward slash. The only saving grace is that this test case is currently not run on Windows because of the `SYMLINK` prerequisite. However, I am currently working towards upstreaming symbolic link support from Git for Windows to upstream Git, which will put a crack into that saving grace. Let's change that test case so that it does not rely on absolute paths (which are passed to the "external command" `ls` as parameters and are therefore part of its output, and which the test case wants to filter out before verifying that the output is as expected) starting with a forward slash. Let's instead rely on the much more reliable fact that `ls` will output the path in a line that ends in a colon, and simply filter out those lines by matching said colon instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
As pointed out in git-for-windows#1676, the `git rev-parse --is-inside-work-tree` command currently fails when the current directory's path contains symbolic links. The underlying reason for this bug is that `getcwd()` is supposed to resolve symbolic links, but our `mingw_getcwd()` implementation did not. We do have all the building blocks for that, though: the `GetFinalPathByHandleW()` function will resolve symbolic links. However, we only called that function if `GetLongPathNameW()` failed, for historical reasons: the latter function was supported for a long time, but the former API function was introduced only with Windows Vista, and we used to support also Windows XP. With that support having been dropped, we are free to call the symbolic link-resolving function right away. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In Git for Windows, `has_symlinks` is set to 0 by default. Therefore, we need to parse the config setting `core.symlinks` to know if it has been set to `true`. In `git init`, we must do that before copying the templates because they might contain symbolic links. Even if the support for symbolic links on Windows has not made it to upstream Git yet, we really should make sure that all the `core.*` settings are parsed before proceeding, as they might very well change the behavior of `git init` in a way the user intended. This fixes git-for-windows#3414 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `strbuf_readlink()` function calls `readlink()`` twice if the hint argument specifies the exact size of the link target (e.g. by passing stat.st_size as returned by `lstat()`). This is necessary because `readlink(..., hint) == hint` could mean that the buffer was too small. Use `hint + 1` as buffer size to prevent this. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `strbuf_readlink()` function refuses to read link targets that exceed PATH_MAX (even if a sufficient size was specified by the caller). As some platforms (*cough* Windows *cough*) support longer paths, remove this restriction (similar to `strbuf_getcwd()`). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Currently, this function hard-codes the directory separator as the forward slash. However, on Windows the backslash character is valid, too. And we want to call this function in the upcoming support for symlinks on Windows with the symlink targets (which naturally use the canonical directory separator on Windows, which is _not_ the forward slash). Prepare that function to be useful also in that context. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
|
/submit |
|
Submitted as pull.2017.git.1765899229.gitgitgadget@gmail.com To fetch this version into To fetch this version to local tag |
4c01018 to
a6a50b0
Compare
|
This patch series was integrated into seen via git@d4817df. |
77dfd22 to
a4c7170
Compare
|
This branch is now known as |
|
There was a status update in the "New Topics" section about the branch Further preparation to upstream symbolic link support on Windows. Comments? source: <pull.2017.git.1765899229.gitgitgadget@gmail.com> |
| #endif | ||
|
|
||
| char *mingw_getcwd(char *pointer, int len) | ||
| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Patrick Steinhardt wrote (reply to this):
On Tue, Dec 16, 2025 at 03:33:45PM +0000, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/compat/mingw.c b/compat/mingw.c
> index ba1b7b6dd1..7215b127cc 100644
> --- a/compat/mingw.c
> +++ b/compat/mingw.c
> @@ -1251,18 +1251,16 @@ char *mingw_getcwd(char *pointer, int len)
> {
> wchar_t cwd[MAX_PATH], wpointer[MAX_PATH];
> DWORD ret = GetCurrentDirectoryW(ARRAY_SIZE(cwd), cwd);
> + HANDLE hnd;
>
> if (!ret || ret >= ARRAY_SIZE(cwd)) {
> errno = ret ? ENAMETOOLONG : err_win_to_posix(GetLastError());
> return NULL;
> }
> - ret = GetLongPathNameW(cwd, wpointer, ARRAY_SIZE(wpointer));
> - if (!ret && GetLastError() == ERROR_ACCESS_DENIED) {
> - HANDLE hnd = CreateFileW(cwd, 0,
> - FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL,
> - OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL);
> - if (hnd == INVALID_HANDLE_VALUE)
> - return NULL;
> + hnd = CreateFileW(cwd, 0,
> + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL,
> + OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL);
> + if (hnd != INVALID_HANDLE_VALUE) {
> ret = GetFinalPathNameByHandleW(hnd, wpointer, ARRAY_SIZE(wpointer), 0);
> CloseHandle(hnd);
> if (!ret || ret >= ARRAY_SIZE(wpointer))
Okay. Due to the change we now also try calling `GetFileAttributesW()`
in case `CreateFileW()` fails, which wasn't the case before. But I'd
consider that to be a win -- if we cannot figure out the final path
name, then we can at least return the unresolved current working
directory.
Patrick
> @@ -1271,13 +1269,11 @@ char *mingw_getcwd(char *pointer, int len)
> return NULL;
> return pointer;
> }
> - if (!ret || ret >= ARRAY_SIZE(wpointer))
> - return NULL;
> - if (GetFileAttributesW(wpointer) == INVALID_FILE_ATTRIBUTES) {
> + if (GetFileAttributesW(cwd) == INVALID_FILE_ATTRIBUTES) {
> errno = ENOENT;
> return NULL;
> }
> - if (xwcstoutf(pointer, wpointer, len) < 0)
> + if (xwcstoutf(pointer, cwd, len) < 0)
> return NULL;
> convert_slashes(pointer);
> return pointer;|
User |
| string = ep; | ||
| } | ||
|
|
||
| return (current & ~negative) | positive; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Patrick Steinhardt wrote (reply to this):
On Tue, Dec 16, 2025 at 03:33:46PM +0000, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/setup.c b/setup.c
> index 7086741e6c..42e4e7a690 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -2611,7 +2611,7 @@ int init_db(const char *git_dir, const char *real_git_dir,
> * have set up the repository format such that we can evaluate
> * includeIf conditions correctly in the case of re-initialization.
> */
> - repo_config(the_repository, platform_core_config, NULL);
> + repo_config(the_repository, git_default_core_config, NULL);
>
> safe_create_dir(the_repository, git_dir, 0);
Two lines further down we call `create_default_files()`, and there we
end up calling `repo_config(the_repository, git_default_config, NULL)`
as one of the first things. We do so after copying templates though, so
indeed this comes too late.
We also cannot really merge these two calls: we need to re-parse the
configuration after having copied over the template, as the template may
contain a gitconfig file itself.
Furthermore, `git_default_core_config()` already knows to call
`platform_core_config()`, as well. So we're not losing any of that
information, either.
All to say that this change makes sense to me and should be safe, as we
don't end up parsing _more_ configuration keys, we only parse a subset
of it a bit earlier.
Patrick|
|
||
| ssize_t strbuf_write(struct strbuf *sb, FILE *f) | ||
| { | ||
| return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Patrick Steinhardt wrote (reply to this):
On Tue, Dec 16, 2025 at 03:33:48PM +0000, Karsten Blees via GitGitGadget wrote:
> diff --git a/strbuf.c b/strbuf.c
> index 44a8f6a554..fa4e30f112 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -566,8 +566,6 @@ ssize_t strbuf_write(struct strbuf *sb, FILE *f)
> return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0;
> }
>
> -#define STRBUF_MAXLINK (2*PATH_MAX)
> -
> int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint)
> {
> size_t oldalloc = sb->alloc;
> @@ -575,7 +573,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint)
> if (hint < 32)
> hint = 32;
>
> - while (hint < STRBUF_MAXLINK) {
> + for (;;) {
> ssize_t len;
>
> strbuf_grow(sb, hint + 1);
This makes me wonder whether we have a better way to figure out the
actual size of the buffer that we ultimately need to allocate. But
reading through readlink(3p) doesn't indicate anything, and I'm not sure
whether we can always rely on lstat(3p) to return the correct size for
symlink contents on all platforms.
One thing that _is_ noted though is that calling the function with a
buffer size larger than SSIZE_MAX is implementation-defined. It does
make me a bit uneasy in that light to grow indefinitely.
Which makes me wonder whether Windows has a limit for the symlink
contents that we could enforce in theory so that we can reasonably turn
this into a bounded loop again?
Patrick
After preparing Git's test suite for the upcoming support for symlinks on Windows, this patch series touches up a couple of code paths that might not seem to be related at first, but need to be adjusted for the symlink support to work as expected.
This is based on
js/test-symlink-windows.cc: Patrick Steinhardt ps@pks.im