Support symbolic links on Windows #2018
Open
+507
−156
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The Win32 API function `GetFileAttributes()` cannot handle paths with trailing dir separators. The current `mingw_stat()`/`mingw_lstat()` implementation calls `GetFileAttributes()` twice if the path has trailing slashes (first with the original path that was passed as function parameter, and and a second time with a path copy with trailing '/' removed). With the conversion to wide Unicode, we get the length of the path for free, and also have a (wide char) buffer that can be modified. This makes it easy to avoid that extraneous Win32 API call. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
|
There are issues in commit ca10b27: |
With respect to symlinks, the current `mingw_stat()` implementation is almost identical to `mingw_lstat()`: except for the file type (`st_mode & S_IFMT`), it returns information about the link rather than the target. Implement `mingw_stat()` by opening the file handle requesting minimal permissions, and then calling `GetFileInformationByHandle()` on it. This way, all links are resolved by the Windows file system layer. If symlinks are disabled, use `mingw_lstat()` as before, but fail with `ELOOP` if a symlink would have to be resolved. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With the new `mingw_stat()` implementation, `do_lstat()` is only called from `mingw_lstat()` (with the function parameter `follow == 0`). Remove the extra function and the old `mingw_stat()`-specific (`follow == 1`) logic. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When obtaining lstat information for reparse points, we need to call `FindFirstFile()` in addition to `GetFileInformationEx()` to obtain the type of the reparse point (symlink, mount point etc.). However, currently there is no error handling whatsoever if `FindFirstFile()` fails. Call `FindFirstFile()` before modifying the `stat *buf` output parameter and error out if the call fails. Note: The `FindFirstFile()` return value includes all the data that we get from `GetFileAttributesEx()`, so we could replace `GetFileAttributesEx()` with `FindFirstFile()`. We don't do that because `GetFileAttributesEx()` is about twice as fast for single files. I.e. we only pay the extra cost of calling `FindFirstFile()` in the rare case that we encounter a reparse point. Please also note that the indentation the remaining reparse point code changed, and hence the best way to look at this diff is with `--color-moved -w`. That code was _not_ moved because a subsequent commit will move it to an altogether different function, anyway. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Move the `S_IFLNK` detection to `file_attr_to_st_mode()`. Implement `DT_LNK` detection in dirent.c's `readdir()` function. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
POSIX specifies that upon successful return from `lstat()`: "the value of the st_size member shall be set to the length of the pathname contained in the symbolic link not including any terminating null byte". Git typically doesn't trust the `stat.st_size` member of symlinks (e.g. see `strbuf_readlink()`). Therefore, it is tempting to save on the extra overhead of opening and reading the reparse point merely to calculate the exact size of the link target. This is, in fact, what Git for Windows did, from May 2015 to May 2020. At least almost: some functions take shortcuts if `st_size` is 0 (e.g. `diff_populate_filespec()`), hence Git for Windows hard-coded the length of all symlinks to MAX_PATH. This did cause problems, though, specifically in Git repositories that were also accessed by Git for Cygwin or Git for WSL. For example, doing `git reset --hard` using Git for Windows would update the size of symlinks in the index to be MAX_PATH; at a later time Git for Cygwin or Git for WSL would find that symlinks have changed size during `git status` and update the index. And then Git for Windows would think that the index needs to be updated. Even if the symlinks did not, in fact, change. To avoid that, the correct size must be determined. Signed-off-by: Bill Zissimopoulos <billziss@navimatics.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In several places, Git's Windows-specific code follows the pattern where it tries to perform an operation, and retries several times when that operation fails, sleeping an increasing amount of time, before finally giving up and asking the user whether to rety (after, say, closing an editor that held a handle to a file, preventing the operation from succeeding). This logic is a bit hard to use, and inconsistent: `mingw_unlink()` and `mingw_rmdir()` duplicate the code to retry, and both of them do so incompletely. They also do not restore `errno` if the user answers 'no'. Introduce a `retry_ask_yes_no()` helper function that handles retry with small delay, asking the user, and restoring `errno`. Note that in `mingw_unlink()`, we include the `_wchmod()` call in the retry loop (which may fail if the file is locked exclusively). In `mingw_rmdir()`, we include special error handling in the retry loop. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Symlinks on Windows don't work the same way as on Unix systems. For example, there are different types of symlinks for directories and files, and unless using a recent-ish Windows version in Developer Mode, creating symlinks requires administrative privileges. By default, disable symlink support on Windows. That is, users explicitly have to enable it with `git config [--system|--global] core.symlinks true`; For convenience, `git init` (and `git clone`) will perform a test whether the current setup allows creating symlinks and will configure that setting in the repository config. The test suite ignores system / global config files. Allow testing *with* symlink support by checking if native symlinks are enabled in MSYS2 (via setting the special environment variable `MSYS=winsymlinks:nativestrict` to ask the MSYS2 runtime to enable creating symlinks). Note: This assumes that Git's test suite is run in MSYS2's Bash, which is true for the time being (an experiment to switch to BusyBox-w32 failed due to the experimental nature of BusyBox-w32). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The Win32 API calls do not set `errno`; Instead, error codes for failed operations must be obtained via the `GetLastError()` function. Git would not know what to do with those error values, though, which is why Git's Windows compatibility layer translates them to `errno` values. Let's handle a couple of symlink-related error codes that will become relevant with the upcoming support for symlinks on Windows. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `_wunlink()` and `DeleteFileW()` functions refuse to delete symlinks to directories on Windows; The error code woutl be `ERROR_ACCESS_DENIED` in that case. Take that error code as an indicator that we need to try `_wrmdir()` as well. In the best case, it will remove a symlink. In the worst case, it will fail with the same error code again. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Older MSVCRT's `_wrename()` function cannot rename symlinks over existing files: it returns success without doing anything. Newer MSVCR*.dll versions probably do not share this problem: according to CRT sources, they just call `MoveFileEx()` with the `MOVEFILE_COPY_ALLOWED` flag. Avoid the `_wrename()` call, and go with directly calling `MoveFileEx()`, with proper error handling of course. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
If symlinks are enabled, resolve all symlinks when changing directories, as required by POSIX. Note: Git's `real_path()` function bases its link resolution algorithm on this property of `chdir()`. Unfortunately, the current directory on Windows is limited to only MAX_PATH (260) characters. Therefore using symlinks and long paths in combination may be problematic. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Implement `readlink()` by reading NTFS reparse points via the `read_reparse_point()` function that was introduced earlier to determine the length of symlink targets. Works for symlinks and directory junctions. If symlinks are disabled, fail with `ENOSYS`. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Implement `symlink()`. This implementation always creates _file_ symlinks (remember: Windows discerns between symlinks pointing to directories and those pointing to files). Support for directory symlinks will be added in a subseqeuent commit. This implementation fails with `ENOSYS` if symlinks are disabled or unsupported. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Symlinks on Windows have a flag that indicates whether the target is a file or a directory. Symlinks of wrong type simply don't work. This even affects core Win32 APIs (e.g. `DeleteFile()` refuses to delete directory symlinks). However, `CreateFile()` with FILE_FLAG_BACKUP_SEMANTICS does work. Check the target type by first creating a tentative file symlink, opening it, and checking the type of the resulting handle. If it is a directory, recreate the symlink with the directory flag set. It is possible to create symlinks before the target exists (or in case of symlinks to symlinks: before the target type is known). If this happens, create a tentative file symlink and postpone the directory decision: keep a list of phantom symlinks to be processed whenever a new directory is created in `mingw_mkdir()`. Limitations: This algorithm may fail if a link target changes from file to directory or vice versa, or if the target directory is created in another process. It's the best Git can do, though. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
As of Windows 10 Build 14972 in Developer Mode, a new flag is supported by `CreateSymbolicLink()` to create symbolic links even when running outside of an elevated session (which was previously required). This new flag is called `SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE` and has the numeric value 0x02. Previous Windows 10 versions will not understand that flag and return an `ERROR_INVALID_PARAMETER`, therefore we have to be careful to try passing that flag only when the build number indicates that it is supported. For more information about the new flag, see this blog post: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/ Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When creating directories via `safe_create_leading_directories()`, we might encounter an already-existing directory which is not readable by the current user. To handle that situation, Git's code calls `stat()` to determine whether we're looking at a directory. In such a case, `CreateFile()` will fail, though, no matter what, and consequently `mingw_stat()` will fail, too. But POSIX semantics seem to still allow `stat()` to go forward. So let's call `mingw_lstat()` to the rescue if we fail to get a file handle due to denied permission in `mingw_stat()`, and fill the stat info that way. We need to be careful to not allow this to go forward in case that we're looking at a symbolic link: to resolve the link, we would still have to create a file handle, and we just found out that we cannot. Therefore, `stat()` still needs to fail with `EACCES` in that case. This fixes git-for-windows#2531. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In git-for-windows#2637, we fixed a bug where symbolic links' target path sizes were recorded incorrectly in the index. The downside of this fix was that every user with tracked symbolic links in their checkouts would see them as modified in `git status`, but not in `git diff`, and only a `git add <path>` (or `git add -u`) would "fix" this. Let's do better than that: we can detect that situation and simply pretend that a symbolic link with a known bad size (or a size that just happens to be that bad size, a _very_ unlikely scenario because it would overflow our buffers due to the trailing NUL byte) means that it needs to be re-checked as if we had just checked it out. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Member
Author
|
/submit |
|
Submitted as pull.2018.git.1765980535.gitgitgadget@gmail.com To fetch this version into To fetch this version to local tag |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This finally upstreams Git for Windows' support for Windows' branch of symbolic links, which has been maturing since 2015. It is based off of
js/prep-symlink-windows.